GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — gh-116738: Test uuid module for free threading by yoney · Pull Request #140068 · python/cpython

Conversation

@yoney

@yoney yoney commented Oct 13, 2025 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Contributor

This PR is more about explaining the reasoning behind the changes than about the code itself, which is actually quite minimal.


Starting with Windows (the easy one):

  • On Windows, the _uuid module uses UuidCreateSequential(). In the current GIL-enabled build, the GIL is already released while this function is called.
  • Although the documentation doesn’t explicitly state that UuidCreateSequential() is thread-safe, it does mention that if the computer doesn’t have an Ethernet or token ring address, the generated UUID is still a valid identifier and is guaranteed to be unique among all UUIDs generated on that machine. So, at the very least, we have local uniqueness.
  • Since the GIL is released (in the GIL-enabled build), I believe calling UuidCreateSequential() also be safe in a free-threaded build. I’ve added a test for it.

For the rest of the systems (Linux, macOS, Solaris, AIX, FreeBSD, etc.):

  • On these platforms, we call one of three functions: uuid_generate_time_safe(), uuid_create(), or uuid_generate_time().
  1. libuuid (uuid_generate_time_safe() and uuid_generate_time()):

    • On Linux, macOS, and Solaris, CPython uses libuuid. Although I couldn’t find any explicit statement in the documentation confirming that libuuid is thread-safe, I reviewed the source code and it seems to be thread-safe.
    • Additionally, I found a discussion where the author of libuuid explains why they introduced THREAD_LOCAL while addressing a threading bug. I checked the current version of libuuid, and that threading bug has been fixed.
    • The Solaris man page also marks these functions as MT-Safe—this is the only explicit mention of thread-safety I could find.
    • I also implemented a test to generate UUIDs using libuuid and ran it repeatedly on Linux (with 48 threads) and macOS (with 10 threads) for several hours. I wasn’t able to produce any collisions or observe any threading issues. On Linux, I tested both with and without the uuidd daemon running.

    Based on all of the above, I changed the uuid module to release the GIL for these libuuid functions, since everything I found indicates they are thread-safe. I did come across some old reported issues, but those have either been fixed in the library itself or addressed in the distributions. However, if there are still concerns about thread safety, to be on the safe side we can remove Py_{BEGIN,END}_ALLOW_THREADS and add a mutex to protect these functions in the free-threading build.

  2. FreeBSD, OpenBSD and AIX (uuid_create())

    • Unfortunately, the AIX documentation doesn’t mention anything about thread safety, and I don’t have access to an AIX machine to test it myself. So, the situation for AIX is unclear.
    • I checked the recent version of the FreeBSD code, and uuid_create() calls the uuidgen() system call which uses a uuid_mutex for synchronization. So, it should be thread-safe.

    I changed the uuid module to release the GIL for create_uuid(). However, the situation on AIX is still unclear, and I’m not sure if there are any other systems that use create_uuid(). It might be safer to remove Py_{BEGIN,END}_ALLOW_THREADS and protect create_uuid() with a mutex in the free-threading build.


Finally, just a note regarding libuuid: Even though the functions seem thread-safe, that doesn’t always mean the UUIDs will be unique. To guarantee uniqueness of UUIDs, libuuid first tries to use a global clock state file (if the process has permission) and/or the uuidd daemon (if it’s running or can be started). If those aren’t available, it falls back to getrandom(), then /dev/{u,}random, and finally rand(). In the worst case, it’s theoretically possible for two processes running at the same time to generate the same UUID.


cc: @mpage @colesbury @Yhg1s @kumaraditya303 @vstinner

  • Issue: Audit all built-in modules for thread safety #116738

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
@yoney
pythongh-116738: Make uuid module thread-safe
3cf26a3
@bedevere-app bedevere-app Bot mentioned this pull request Oct 13, 2025
Audit all built-in modules for thread safety #116738
Open
@yoney
yoney marked this pull request as ready for review October 13, 2025 23:21
@bedevere-app bedevere-app Bot added the awaiting review label Oct 13, 2025
ashm-dev added a commit to ashm-dev/cpython that referenced this pull request Oct 14, 2025
@ashm-dev
pythongh-140068: Fix memory leak in subinterpreter creation
5a8ac36
The memcpy() call in _PyInterpreterState_New() was overwriting the
_malloced pointer that was set by alloc_interpreter(), causing a memory
leak when subinterpreters were destroyed.

Fixed by preserving the _malloced pointer across the memcpy().
@vstinner

vstinner commented Oct 14, 2025

Copy link
Copy Markdown
Member

cc @picnixz

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@picnixz
picnixz self-requested a review October 14, 2025 11:09
@mpage
mpage requested review from Yhg1s, colesbury, kumaraditya303 and mpage October 14, 2025 16:14
@mpage

mpage commented Oct 14, 2025

Copy link
Copy Markdown
Contributor

First of all, thanks a bunch for the detailed and thorough research!

Based on all of the above, I changed the uuid module to release the GIL for these libuuid functions, since everything I found indicates they are thread-safe ... However, if there are still concerns about thread safety, to be on the safe side we can remove Py_{BEGIN,END}_ALLOW_THREADS and add a mutex to protect these functions in the free-threading build.

I would lean towards not modifying the existing behavior in the with-GIL builds (either by adding a mutex in the free-threaded build, or by leaving the code unmodified, thereby allowing the builds to diverge). Curious to see what others think.

yoney reacted with thumbs up emoji
All reactions
  • 👍 1 reaction

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@colesbury

colesbury commented Oct 14, 2025

Copy link
Copy Markdown
Contributor

I would lean towards not modifying the existing behavior in the with-GIL builds (either by adding a mutex in the free-threaded build, or by leaving the code unmodified, thereby allowing the builds to diverge).

Yes, I agree

yoney reacted with thumbs up emoji
All reactions
  • 👍 1 reaction

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@kumaraditya303

kumaraditya303 commented Oct 14, 2025

Copy link
Copy Markdown
Contributor

I would lean towards not modifying the existing behavior in the with-GIL builds (either by adding a mutex in the free-threaded build, or by leaving the code unmodified, thereby allowing the builds to diverge).

Agreed, also if the functions do not block then releasing and acquiring the GIL would cause a slowdown in the gil-enabled build.

yoney reacted with thumbs up emoji
All reactions
  • 👍 1 reaction

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@yoney
pythongh-116738: Remove Py_{BEGIN,END}_ALLOW_THREADS
5dab3ed
@yoney yoney changed the title gh-116738: Make uuid module thread-safe gh-116738: Test uuid module for free threading Oct 15, 2025
kumaraditya303
kumaraditya303 reviewed Oct 15, 2025
View reviewed changes
Comment thread Lib/test/test_free_threading/test_uuid.py Resolved

Uh oh!

There was an error while loading. Please reload this page.

kumaraditya303
kumaraditya303 reviewed Oct 15, 2025
View reviewed changes
Comment thread Lib/test/test_free_threading/test_uuid.py Resolved

Uh oh!

There was an error while loading. Please reload this page.

@yoney

yoney commented Oct 15, 2025

Copy link
Copy Markdown
Contributor Author

@mpage @colesbury @kumaraditya303 thanks for the comments!

I’ve removed the Py_{BEGIN,END}_ALLOW_THREADS lines and left everything else as it was. With this approach, we’re saying we believe these three functions are thread-safe, so there’s no need to add extra locking. This also covers subinterpreters, so we don’t need to do anything special for them.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@kumaraditya303 kumaraditya303 added topic-free-threading needs backport to 3.14 bugs and security fixes skip news labels Oct 15, 2025
@yoney
pythongh-116738: Remove redundant skipUnless
442c082
vstinner
vstinner reviewed Oct 15, 2025
View reviewed changes
Comment thread Lib/test/test_free_threading/test_uuid.py Resolved

Uh oh!

There was an error while loading. Please reload this page.

vstinner
vstinner reviewed Oct 15, 2025
View reviewed changes
Comment thread Lib/test/test_free_threading/test_uuid.py Resolved

Uh oh!

There was an error while loading. Please reload this page.

@yoney
pythongh-116738: Improve readability
e73b23d
colesbury
colesbury approved these changes Oct 15, 2025
View reviewed changes
@bedevere-app bedevere-app Bot added awaiting merge and removed awaiting review labels Oct 15, 2025
vstinner
vstinner approved these changes Oct 16, 2025
View reviewed changes

@vstinner vstinner left a comment

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
kumaraditya303
kumaraditya303 approved these changes Oct 16, 2025
View reviewed changes
@kumaraditya303
kumaraditya303 merged commit 9a87ce8 into python:main Oct 16, 2025
4 checks passed

Uh oh!

There was an error while loading. Please reload this page.

@miss-islington-app

miss-islington-app Bot commented Oct 16, 2025

Copy link
Copy Markdown

Thanks @yoney for the PR, and @kumaraditya303 for merging it 🌮🎉.. I'm working now to backport this PR to: 3.14.
🐍🍒⛏🤖

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@bedevere-app bedevere-app Bot removed the awaiting merge label Oct 16, 2025
miss-islington pushed a commit to miss-islington/cpython that referenced this pull request Oct 16, 2025
@yoney @miss-islington
pythongh-116738: test uuid module thread safety in free-threading (p…
2677697
…ythonGH-140068)

(cherry picked from commit 9a87ce8)

Co-authored-by: Alper <[email protected]>
@bedevere-app

bedevere-app Bot commented Oct 16, 2025

Copy link
Copy Markdown

GH-140229 is a backport of this pull request to the 3.14 branch.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@bedevere-app bedevere-app Bot removed the needs backport to 3.14 bugs and security fixes label Oct 16, 2025
kumaraditya303 pushed a commit that referenced this pull request Oct 17, 2025
@miss-islington @yoney
[3.14] gh-116738: test uuid module thread safety in free-threading (G…
e07b78c
…H-140068) (#140229)

gh-116738: test `uuid` module thread safety in free-threading (GH-140068)
(cherry picked from commit 9a87ce8)

Co-authored-by: Alper <[email protected]>
@yoney
yoney deleted the ft_uuid branch October 19, 2025 03:03
StanFromIreland pushed a commit to StanFromIreland/cpython that referenced this pull request Dec 6, 2025
@yoney @StanFromIreland
pythongh-116738: test uuid module thread safety in free-threading (p…
6572ad9
…ython#140068)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@vstinner vstinner vstinner approved these changes
@colesbury colesbury colesbury approved these changes
@kumaraditya303 kumaraditya303 kumaraditya303 approved these changes
@picnixz picnixz Awaiting requested review from picnixz
@mpage mpage Awaiting requested review from mpage
@Yhg1s Yhg1s Awaiting requested review from Yhg1s

Uh oh!

There was an error while loading. Please reload this page.

Assignees

No one assigned

Labels

skip news topic-free-threading

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Uh oh!

There was an error while loading. Please reload this page.

5 participants

@yoney @vstinner @mpage @colesbury @kumaraditya303
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
You are visitor 01402503 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.