GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — gh-130363: add a test resource to mark tests that need an idle system by rossburton · Pull Request #130508 · python/cpython

Conversation

@rossburton

@rossburton rossburton commented Feb 24, 2025 •
edited by bedevere-app Bot
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Contributor

Some tests are very sensitive to timing and will fail on a loaded system, typically because there are small windows for timeouts to trigger in.

Some of these are poorly implemented tests which can be improved, but others may genuinely have strict timing requirements.

Add a test resource so that these tests can be marked as such, and only ran when the system is known to be idle.

  • Issue: Annotate test cases which are timing sensitive #130363

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
@bedevere-app bedevere-app Bot added the awaiting review label Feb 24, 2025
@bedevere-app bedevere-app Bot mentioned this pull request Feb 24, 2025
Annotate test cases which are timing sensitive #130363
Open
@rossburton
rossburton force-pushed the idle branch from 9654152 to c3b2f58 Compare February 24, 2025 14:13
@rossburton
pythongh-130363: add a test resource to mark tests that need an idle …
9dfbe1e
…system

Some tests are very sensitive to timing and will fail on a loaded system,
typically because there are small windows for timeouts to trigger in.

Some of these are poorly implemented tests which can be improved, but
others may genuinely have strict timing requirements.

Add a test resource so that these tests can be marked as such, and only
ran when the system is known to be idle.
@rossburton
rossburton force-pushed the idle branch from c3b2f58 to 9dfbe1e Compare February 24, 2025 14:22
@vstinner

vstinner commented Feb 24, 2025 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Member

Some tests are very sensitive to timing and will fail on a loaded system

Which tests? Do you have examples?

The Python test suite is reliable on busy / heavy loaded systems.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@rossburton

rossburton commented Feb 24, 2025

Copy link
Copy Markdown
Contributor Author

The latest example which triggered this:

FAIL: test_timerfd_initval (test.test_os.TimerfdTests.test_timerfd_initval)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib/python3.13/test/test_os.py", line 4166, in test_timerfd_initval
    self.assertAlmostEqual(next_expiration, initial_expiration, places=self.CLOCK_RES_PLACES)
    ~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AssertionError: 0.24403756000000001 != 0.25 within 3 places (0.0059624399999999855 difference)

The test is checking how accurately the timer is.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@vstinner

vstinner commented Feb 24, 2025

Copy link
Copy Markdown
Member

Would be be possible to tune TimerfdTests to accept a large time difference, rather than skipping the test?

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@rossburton

rossburton commented Feb 24, 2025 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Contributor Author

Isn't the point of the test to check the timings though? Would it be considered a pass if the request was a 0.1 second timer and a bug meant it waited for 0.1 hours instead?

Found an existing bug for that specific test on another Linux system: #126112.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@mhsmith

mhsmith commented Feb 24, 2025

Copy link
Copy Markdown
Member

Found an existing bug for that specific test on another Linux system: #126112

These are two different timerfd tests. The one quoted above isn't really testing the timer at all – it never expires. It's really testing that a few lines of Python code can run within 1 millisecond. This has already been increased to 10 ms on Android emulators (see the comment at the top of the test class), and I think it wouldn't be unreasonable to extend that to all platforms.

By comparison, the test mentioned in #126112 has a much wider timing margin – 125 ms if I understand correctly – so either it's caused by something else, or my understanding is wrong.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@mhsmith mhsmith mentioned this pull request Feb 24, 2025
test_os.TimerfdTests is flaky #126112
Closed
@github-actions

github-actions Bot commented Apr 21, 2026

Copy link
Copy Markdown

This PR is stale because it has been open for 30 days with no activity.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@github-actions github-actions Bot added the stale Stale PR or inactive for long period of time. label Apr 21, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Uh oh!

There was an error while loading. Please reload this page.

Assignees

No one assigned

Labels

awaiting review stale Stale PR or inactive for long period of time.

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Uh oh!

There was an error while loading. Please reload this page.

3 participants

@rossburton @vstinner @mhsmith
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
You are visitor 01402503 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.