GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — gh-108416: Mark slow but not CPU bound test methods with requires_resource('walltime') by serhiy-storchaka · Pull Request #108480 · python/cpython

Conversation

@serhiy-storchaka

@serhiy-storchaka serhiy-storchaka commented Aug 25, 2023 •
edited by bedevere-bot
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Member
  • Issue: Mark slow test methods with @requires_resource('cpu') #108416

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
@serhiy-storchaka
pythongh-108416: Mark slow but not CPU bound test methods with requir…
f144852
…es_resource('walltime')
@serhiy-storchaka serhiy-storchaka added tests Tests in the Lib/test dir skip news needs backport to 3.11 only security fixes needs backport to 3.12 only security fixes labels Aug 25, 2023
@serhiy-storchaka
serhiy-storchaka requested review from ambv and gpshead August 25, 2023 11:33
@serhiy-storchaka
serhiy-storchaka requested review from a team and vsajip as code owners August 25, 2023 11:33
@bedevere-bot bedevere-bot mentioned this pull request Aug 25, 2023
Mark slow test methods with @requires_resource('cpu') #108416
Closed
@bedevere-bot bedevere-bot added the awaiting core review label Aug 25, 2023
@serhiy-storchaka
Merge branch 'main' into mark-slow-tests-walltime
de5f051
@serhiy-storchaka
serhiy-storchaka removed request for a team and vsajip August 25, 2023 12:47
@serhiy-storchaka serhiy-storchaka mentioned this pull request Sep 2, 2023
gh-108416: Mark slow test methods with @requires_resource('cpu') #108421
Merged
@serhiy-storchaka

serhiy-storchaka commented Sep 2, 2023

Copy link
Copy Markdown
Member Author

These are mostly tests which contain several sleep()s or call a function which expected to fail by timeout. There are few tests which depend on slow external network service.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@serhiy-storchaka
Merge branch 'main' into mark-slow-tests-walltime
58480c3
gpshead
gpshead approved these changes Sep 3, 2023
View reviewed changes
@bedevere-bot bedevere-bot added awaiting merge and removed awaiting core review labels Sep 3, 2023
@serhiy-storchaka
Merge branch 'main' into mark-slow-tests-walltime
18dd932
@serhiy-storchaka

serhiy-storchaka commented Sep 3, 2023

Copy link
Copy Markdown
Member Author
$ ./python -m test -uall,-cpu -j4
...
Total duration: 7 min 27 sec
Total tests: run=42,127 skipped=1,077
$ ./python -m test -uall,-cpu,-walltime -j4
...
Total duration: 6 min 26 sec
Total tests: run=42,127 skipped=1,139
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@serhiy-storchaka serhiy-storchaka mentioned this pull request Sep 5, 2023
Support selecting tests by labels #108828
Open
@serhiy-storchaka
serhiy-storchaka merged commit 1e0d627 into python:main Sep 5, 2023
@miss-islington

miss-islington commented Sep 5, 2023

Copy link
Copy Markdown
Contributor

Thanks @serhiy-storchaka for the PR 🌮🎉.. I'm working now to backport this PR to: 3.11, 3.12.
🐍🍒⛏🤖

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@bedevere-bot bedevere-bot removed the awaiting merge label Sep 5, 2023
@serhiy-storchaka
serhiy-storchaka deleted the mark-slow-tests-walltime branch September 5, 2023 14:56
@miss-islington

miss-islington commented Sep 5, 2023

Copy link
Copy Markdown
Contributor

Sorry, @serhiy-storchaka, I could not cleanly backport this to 3.11 due to a conflict.
Please backport using cherry_picker on command line.
cherry_picker 1e0d62793a84001e92f1c80b511d3a212b435acc 3.11

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

miss-islington pushed a commit to miss-islington/cpython that referenced this pull request Sep 5, 2023
@serhiy-storchaka @miss-islington
pythongh-108416: Mark slow but not CPU bound test methods with requir…
7cd8441
…es_resource('walltime') (pythonGH-108480)

(cherry picked from commit 1e0d627)

Co-authored-by: Serhiy Storchaka <[email protected]>
@miss-islington miss-islington assigned serhiy-storchaka Sep 5, 2023
@bedevere-bot

bedevere-bot commented Sep 5, 2023

Copy link
Copy Markdown

GH-108923 is a backport of this pull request to the 3.12 branch.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@bedevere-bot bedevere-bot removed the needs backport to 3.12 only security fixes label Sep 5, 2023
serhiy-storchaka added a commit to serhiy-storchaka/cpython that referenced this pull request Sep 5, 2023
@serhiy-storchaka
[3.11] pythongh-108416: Mark slow but not CPU bound test methods with…
65b5218
… requires_resource('walltime') (pythonGH-108480).

(cherry picked from commit 1e0d627)

Co-authored-by: Serhiy Storchaka <[email protected]>
@bedevere-bot

bedevere-bot commented Sep 5, 2023

Copy link
Copy Markdown

GH-108924 is a backport of this pull request to the 3.11 branch.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@bedevere-bot bedevere-bot removed the needs backport to 3.11 only security fixes label Sep 5, 2023
serhiy-storchaka added a commit that referenced this pull request Sep 5, 2023
@serhiy-storchaka
[3.11] gh-108416: Mark slow but not CPU bound test methods with requi…
cf19e8e
…res_resource('walltime') (GH-108480) (GH-108924)

(cherry picked from commit 1e0d627)
vstinner pushed a commit to vstinner/cpython that referenced this pull request Sep 5, 2023
@serhiy-storchaka @vstinner
pythongh-108416: Mark slow but not CPU bound test methods with requir…
2cf86ef
…es_resource('walltime') (pythonGH-108480)

(cherry picked from commit 1e0d627)
@vstinner vstinner mentioned this pull request Sep 5, 2023
[3.12] gh-108834: Sync libregrtest with the main branch #108966
Merged
Yhg1s pushed a commit that referenced this pull request Sep 8, 2023
@vstinner @serhiy-storchaka
[3.12] gh-108834: Sync libregrtest with the main branch (#108966)
0855b2c
* gh-108834: regrtest reruns failed tests in subprocesses (#108839)

When using --rerun option, regrtest now re-runs failed tests
in verbose mode in fresh worker processes to have more
deterministic behavior. So it can write its final report even
if a test killed a worker progress.

Add --fail-rerun option to regrtest: exit with non-zero exit code
if a test failed pass passed when re-run in verbose mode (in a
fresh process). That's now more useful since tests can pass
when re-run in a fresh worker progress, whereas they failed
when run after other tests when tests are run sequentially.

Rename --verbose2 option (-w) to --rerun. Keep --verbose2 as a
deprecated alias.

Changes:

* Fix and enhance statistics in regrtest summary. Add "(filtered)"
  when --match and/or --ignore options are used.
* Add RunTests class.
* Add TestResult.get_rerun_match_tests() method
* Rewrite code to serialize/deserialize worker arguments as JSON
  using a new WorkerJob class.
* Fix stats when a test is run with --forever --rerun.
* If failed test names cannot be parsed, log a warning and don't
  filter tests.
* test_regrtest.test_rerun_success() now uses a marker file, since
  the test is re-run in a separated process.
* Add tests on normalize_test_name() function.
* Add test_success() and test_skip() tests to test_regrtest.

(cherry picked from commit 31c2945)

* gh-108834: regrtest --fail-rerun exits with code 5 (#108896)

When the --fail-rerun option is used and a test fails and then pass,
regrtest now uses exit code 5 ("rerun) instead of 2 ("bad test").

(cherry picked from commit 1170d5a)

* gh-108416: Mark slow but not CPU bound test methods with requires_resource('walltime') (GH-108480)

(cherry picked from commit 1e0d627)

* Manually sync Lib/test/libregrtest/ from main

---------

Co-authored-by: Serhiy Storchaka <[email protected]>
@serhiy-storchaka serhiy-storchaka removed their assignment Sep 8, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@gpshead gpshead gpshead approved these changes
@ambv ambv Awaiting requested review from ambv

Uh oh!

There was an error while loading. Please reload this page.

Assignees

No one assigned

Labels

skip news tests Tests in the Lib/test dir

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Uh oh!

There was an error while loading. Please reload this page.

4 participants

@serhiy-storchaka @miss-islington @bedevere-bot @gpshead
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
You are visitor 01402503 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.