GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — [3.10] bpo-41710: PyThread_acquire_lock_timed() uses sem_clockwait() by vstinner · Pull Request #28671 · python/cpython

Conversation

@vstinner

@vstinner vstinner commented Oct 1, 2021 •
edited by bedevere-bot
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Member

On Unix, if the sem_clockwait() function is available in the C
library (glibc 2.30 and newer), the threading.Lock.acquire() method
now uses the monotonic clock (time.CLOCK_MONOTONIC) for the timeout,
rather than using the system clock (time.CLOCK_REALTIME), to not be
affected by system clock changes.

configure now checks if the sem_clockwait() function is available.

https://bugs.python.org/issue41710

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
@vstinner
bpo-41710: PyThread_acquire_lock_timed() uses sem_clockwait()
3112358
On Unix, if the sem_clockwait() function is available in the C
library (glibc 2.30 and newer), the threading.Lock.acquire() method
now uses the monotonic clock (time.CLOCK_MONOTONIC) for the timeout,
rather than using the system clock (time.CLOCK_REALTIME), to not be
affected by system clock changes.

configure now checks if the sem_clockwait() function is available.
@bedevere-bot bedevere-bot added the awaiting core review label Oct 1, 2021
@the-knights-who-say-ni the-knights-who-say-ni added the CLA signed label Oct 1, 2021
@vstinner

vstinner commented Oct 1, 2021

Copy link
Copy Markdown
Member Author

@gpshead @corona10: Do you think that it's worth it to backport the sem_clockwait() change from main the 3.9 and 3.10? I see it as a bugfix for https://bugs.python.org/issue41710 bug.


In the main branch, I wrote pytime_add() and _PyTime_AsTimespec_clamp() to handle timeout and clock values close to the C type limits. It should make the code safer. I made tons of changes in pytime.c for that. I don't think that it's worth it to backport these enhancements. Just don't use crazy timeouts :-D The current code doesn't clamp values, so this change doesn't add any regression.

In Modules/_threadmodule.c, lock_acquire_parse_args() makes sure that the timeout is less than PY_TIMEOUT_MAX. Moreover, PyThread_acquire_lock_timed() has a sanity check:

    if (microseconds > PY_TIMEOUT_MAX) {
        Py_FatalError("Timeout larger than PY_TIMEOUT_MAX");
    }

In main, I removed the Py_FatalError() call thanks to the new _PyTime_AsTimespec_clamp() function.

By the way, I documented pytime.c in pytime.h ;-) (in the main branch)

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@corona10

corona10 commented Oct 1, 2021

Copy link
Copy Markdown
Member

Do you think that it's worth it to backport the sem_clockwait() change from main the 3.9 and 3.10?

I am +0 on this.
Since using sem_clockwait can be a new feature from some views even though that feature can fix the bug.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@gpshead

gpshead commented Oct 1, 2021

Copy link
Copy Markdown
Member

I think it is reasonable as a bugfix. sem_clockwait appears to be a pure libc only function, not a new syscall requiring new os kernel support so the risk seems low. When compiled on a system with headers defining it the binary already requires that min. library version to run. So it should be fine.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

gpshead
gpshead approved these changes Oct 1, 2021
View reviewed changes
@bedevere-bot bedevere-bot added awaiting merge and removed awaiting core review labels Oct 1, 2021
@vstinner

vstinner commented Oct 1, 2021

Copy link
Copy Markdown
Member Author

I think it is reasonable as a bugfix. sem_clockwait appears to be a pure libc only function, not a new syscall requiring new os kernel support so the risk seems low. When compiled on a system with headers defining it the binary already requires that min. library version to run. So it should be fine.

Yeah, when I wrote my change for the main branch, I read the sem_clockwait() source code in the glibc. It shares code with other semaphore functions and it doesn't seem to be a new syscall. At least, I failed to find a specific syscall call in the code or using strace.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@vstinner
vstinner merged commit 6df8c32 into python:3.10 Oct 1, 2021
@bedevere-bot bedevere-bot removed the awaiting merge label Oct 1, 2021
@vstinner
vstinner deleted the sem_clockwait310 branch October 1, 2021 16:22
@vstinner vstinner added the needs backport to 3.9 label Oct 1, 2021
@miss-islington

miss-islington commented Oct 1, 2021

Copy link
Copy Markdown
Contributor

Thanks @vstinner for the PR 🌮🎉.. I'm working now to backport this PR to: 3.9.
🐍🍒⛏🤖

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@bedevere-bot

bedevere-bot commented Oct 1, 2021

Copy link
Copy Markdown

GH-28683 is a backport of this pull request to the 3.9 branch.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@bedevere-bot bedevere-bot removed the needs backport to 3.9 label Oct 1, 2021
miss-islington pushed a commit to miss-islington/cpython that referenced this pull request Oct 1, 2021
@vstinner @miss-islington
bpo-41710: PyThread_acquire_lock_timed() uses sem_clockwait() (python…
ba3654a
…GH-28671)

On Unix, if the sem_clockwait() function is available in the C
library (glibc 2.30 and newer), the threading.Lock.acquire() method
now uses the monotonic clock (time.CLOCK_MONOTONIC) for the timeout,
rather than using the system clock (time.CLOCK_REALTIME), to not be
affected by system clock changes.

configure now checks if the sem_clockwait() function is available.
(cherry picked from commit 6df8c32)

Co-authored-by: Victor Stinner <[email protected]>
vstinner added a commit that referenced this pull request Oct 1, 2021
@miss-islington @vstinner
bpo-41710: PyThread_acquire_lock_timed() uses sem_clockwait() (GH-28671…
0e1aeab
…) (GH-28683)

On Unix, if the sem_clockwait() function is available in the C
library (glibc 2.30 and newer), the threading.Lock.acquire() method
now uses the monotonic clock (time.CLOCK_MONOTONIC) for the timeout,
rather than using the system clock (time.CLOCK_REALTIME), to not be
affected by system clock changes.

configure now checks if the sem_clockwait() function is available.
(cherry picked from commit 6df8c32)

Co-authored-by: Victor Stinner <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@gpshead gpshead gpshead approved these changes

Uh oh!

There was an error while loading. Please reload this page.

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Uh oh!

There was an error while loading. Please reload this page.

6 participants

@vstinner @corona10 @gpshead @miss-islington @bedevere-bot @the-knights-who-say-ni
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
You are visitor 01402503 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.