GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — Add coverage benchmark by kumaraditya303 · Pull Request #213 · python/pyperformance

Conversation

@kumaraditya303

@kumaraditya303 kumaraditya303 commented Jun 6, 2022 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Contributor

Add a coverage benchmark to help benchmarking and avoid regressions such as python/cpython#93516.
Benchmark result 3.10 vs 3.12 main branch.

Benchmark 3.10 3.12
coverage 96.8 ms 204 ms: 2.11x slower

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
All reactions
kumaraditya303 added 2 commits June 6, 2022 08:07
@kumaraditya303
Add coverage benchmark
7fbe6c4
@kumaraditya303
docstring
f2b683d
@kumaraditya303 kumaraditya303 mentioned this pull request Jun 6, 2022
Use PyCode_GetCode coveragepy/coveragepy#1394
Merged
@kumaraditya303
kumaraditya303 requested a review from gvanrossum June 6, 2022 08:35
@kumaraditya303
whitespace
26b9450
gvanrossum
gvanrossum reviewed Jun 6, 2022
View reviewed changes

@gvanrossum gvanrossum left a comment

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks for doing something about this.

Question for @kumaraditya303 -- Have you actually tried this before and after the regression you mentioned as example?

Question for @ericsnowcurrently -- Do you agree this is useful?

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
All reactions
@kumaraditya303

kumaraditya303 commented Jun 6, 2022

Copy link
Copy Markdown
Contributor Author

Have you actually tried this before and after the regression you mentioned as example?

@gvanrossum: The regression I linked above 1 is not caused by a single change but rather accumulation of multiple slowdown caused by multiple changes of which the most significant ones being python/cpython#91666 and python/cpython#27077. Since there is no benchmark in pyperformance to exercise coverage performance, the slowdown was unnoticed. Once this benchmark is added, it will help us pinpoint which commits caused the most slowdown with a simple git bisect between 3.10 and main branches.

Footnotes

  1. https://github.com/python/pyperformance/pull/213#issue-1261503016 ↩
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
@gvanrossum

gvanrossum commented Jun 6, 2022

Copy link
Copy Markdown
Member

Nevertheless it would be interesting to see the numbers of running this BM on the state before and after said commits to demonstrate its usefulness.
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
@kumaraditya303

kumaraditya303 commented Jun 6, 2022

Copy link
Copy Markdown
Contributor Author

I am running the benchmarks now, will post soon.
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
@kumaraditya303

kumaraditya303 commented Jun 6, 2022

Copy link
Copy Markdown
Contributor Author

@gvanrossum Benchmarking before and after python/cpython@944fffe commit

Results:

Benchmark before after
coverage 161 ms 213 ms: 1.32x slower
gvanrossum reacted with thumbs up emoji
All reactions
  • 👍 1 reaction

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
@gvanrossum
gvanrossum merged commit 01d3e81 into python:main Jun 6, 2022
@ericsnowcurrently

ericsnowcurrently commented Jun 6, 2022

Copy link
Copy Markdown
Member

Question for @ericsnowcurrently -- Do you agree this is useful?

Yeah, it's a helpful indicator of impact.
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
@Fidget-Spinner Fidget-Spinner mentioned this pull request Jun 12, 2022
Severe performance degradation for tracing under 3.11 python/cpython#93516
Open
@kumaraditya303
kumaraditya303 deleted the coverage branch June 12, 2022 16:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@gvanrossum gvanrossum gvanrossum left review comments

Uh oh!

There was an error while loading. Please reload this page.

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Uh oh!

There was an error while loading. Please reload this page.

3 participants

@kumaraditya303 @gvanrossum @ericsnowcurrently
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
You are visitor 01402503 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.