GeezerVison is showing github.com — gh-115999: Enable BINARY_SUBSCR_GETITEM for free-threaded build by corona10 · Pull Request #127737 · python/cpython
Conversation
corona10
commented
Dec 8, 2024
•
edited by bedevere-app
Bot
Loading
Uh oh!
There was an error while loading. Please reload this page.
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
corona10
changed the title
[WIP] gh-115999: Enable BINARY_SUBSCR_GETITEM for free-threaded build
gh-115999: Enable BINARY_SUBSCR_GETITEM for free-threaded build
Dec 8, 2024
corona10
commented
Dec 8, 2024
•
edited
Loading
Uh oh!
There was an error while loading. Please reload this page.
|
@mpage, @colesbury
To enable BINARY_SUBSCR_GETITEM, we should handle TypeObject and _spec_cache which are mutable by other threads.
To handle this, I have introduced PyType_GetItemFromCache, _PyType_GetItemFromCacheWithVersion, and _PyType_CacheGetItemForSpecialization to lock the type and mutate them at the same transaction.
But I am not sure this approach is proper approach since I noticed that cached items can be mutated before the interpreter gets cached objects, so I have added the logic to check whether the item is NULL or FunctionObject, but please let me know if my approach is wrong.
Anyway, this is the first working version without error. PTAL.
|
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for working on this! I left a few comments inline.
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
|
|
PyTypeObject *tp = Py_TYPE(PyStackRef_AsPyObjectBorrow(container));
|
|
|
PyHeapTypeObject *ht = (PyHeapTypeObject *)tp;
|
|
|
PyObject *getitem = ht->_spec_cache.getitem;
|
|
|
PyObject *getitem = _PyType_GetItemFromCache(tp);
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think it's probably safe to reload getitem from the cache after the guard passes. (If the getitem in the cache is replaced after the guard passes, the new getitem cannot be mutated in a way that would change the function version: that would require stopping the world). However, I'd prefer that we pass the getitem that is loaded in _BINARY_SUBSCR_CHECK_FUNC to _BINARY_SUBSCR_INIT_CALL.
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
corona10
changed the title
gh-115999: Enable BINARY_SUBSCR_GETITEM for free-threaded build
[WIP] gh-115999: Enable BINARY_SUBSCR_GETITEM for free-threaded build
Dec 15, 2024
corona10
changed the title
[WIP] gh-115999: Enable BINARY_SUBSCR_GETITEM for free-threaded build
gh-115999: Enable BINARY_SUBSCR_GETITEM for free-threaded build
Dec 16, 2024
corona10
commented
Dec 16, 2024
•
edited
Loading
Uh oh!
There was an error while loading. Please reload this page.
|
@mpage Hmm would you like to take a look again?
|
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Overall approach is looking good! I'd still prefer it if we passed the getitem that is loaded in _BINARY_SUBSCR_CHECK_FUNC to _BINARY_SUBSCR_INIT_CALL. That would also allow us to avoid introducing the DEOPT_IF in _BINARY_SUBSCR_INIT_CALL. Is there a reason you don't want to do that?
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
I can't see it. You may forget to press the comment button, I guess :)
Oops - fixed!
|
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks good! I noticed one last thing then I think we're good to go.
We need to update the opcode definition of _BINARY_SUBSCR_INIT_CALL in Python/optimizer_bytecodes.c to handle the extra output (getitem) of _BINARY_SUBSCR_CHECK_FUNC.
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
corona10 reacted with thumbs up emoji
|
Done! Even if I am not sure that this change is the correct for tier 2 because I have never handled this before :)
|
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
|
The tier2 change is correct. I should add an automatic script that checks tier2 and tier1 definitions are synced so that you don't trip over this. Thanks Matt for catching that!
|
corona10 and mpage reacted with thumbs up emoji
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
|
I kicked off a couple of benchmark runs earlier today. They should be done shortly. Let’s wait for the results. Otherwise, looks great!
|
corona10 reacted with thumbs up emoji
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks good! Benchmark results:
- ~1% improvement on free-threaded builds
- ~neutral on default builds
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️
Hi! The buildbot PPC64LE Fedora Stable 3.x has failed when building commit 48c70b8.
What do you need to do:
- Don't panic.
- Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
- Go to the page of the buildbot that failed (https://buildbot.python.org/#/builders/90/builds/5364) and take a look at the build logs.
- Check if the failure is related to this commit (48c70b8) or if it is a false positive.
- If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.
You can take a look at the buildbot page here:
https://buildbot.python.org/#/builders/90/builds/5364
Failed tests:
Failed subtests:
- testStream - test.test_socket.ThreadedVSOCKSocketStreamTest.testStream
Summary of the results of the build (if available):
==
Click to see traceback logs
Traceback (most recent call last):
File "/home/buildbot/buildarea/3.x.cstratak-fedora-stable-ppc64le/build/Lib/test/test_socket.py", line 550, in testStream
msg = self.conn.recv(1024)
PermissionError: [Errno 13] Permission denied
|
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️
Hi! The buildbot PPC64LE Fedora Stable Clang 3.x has failed when building commit 48c70b8.
What do you need to do:
- Don't panic.
- Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
- Go to the page of the buildbot that failed (https://buildbot.python.org/#/builders/435/builds/5142) and take a look at the build logs.
- Check if the failure is related to this commit (48c70b8) or if it is a false positive.
- If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.
You can take a look at the buildbot page here:
https://buildbot.python.org/#/builders/435/builds/5142
Failed tests:
Failed subtests:
- testStream - test.test_socket.ThreadedVSOCKSocketStreamTest.testStream
Summary of the results of the build (if available):
==
Click to see traceback logs
Traceback (most recent call last):
File "/home/buildbot/buildarea/3.x.cstratak-fedora-stable-ppc64le.clang/build/Lib/test/test_socket.py", line 550, in testStream
msg = self.conn.recv(1024)
PermissionError: [Errno 13] Permission denied
|
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️
Hi! The buildbot PPC64LE Fedora Stable Clang Installed 3.x has failed when building commit 48c70b8.
What do you need to do:
- Don't panic.
- Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
- Go to the page of the buildbot that failed (https://buildbot.python.org/#/builders/451/builds/5124) and take a look at the build logs.
- Check if the failure is related to this commit (48c70b8) or if it is a false positive.
- If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.
You can take a look at the buildbot page here:
https://buildbot.python.org/#/builders/451/builds/5124
Failed tests:
Failed subtests:
- testStream - test.test_socket.ThreadedVSOCKSocketStreamTest.testStream
Summary of the results of the build (if available):
==
Click to see traceback logs
Traceback (most recent call last):
File "/home/buildbot/buildarea/3.x.cstratak-fedora-stable-ppc64le.clang-installed/build/target/lib/python3.14/test/test_socket.py", line 550, in testStream
msg = self.conn.recv(1024)
PermissionError: [Errno 13] Permission denied
|
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️
Hi! The buildbot PPC64LE Fedora Stable LTO + PGO 3.x has failed when building commit 48c70b8.
What do you need to do:
- Don't panic.
- Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
- Go to the page of the buildbot that failed (https://buildbot.python.org/#/builders/101/builds/5356) and take a look at the build logs.
- Check if the failure is related to this commit (48c70b8) or if it is a false positive.
- If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.
You can take a look at the buildbot page here:
https://buildbot.python.org/#/builders/101/builds/5356
Failed tests:
Failed subtests:
- testStream - test.test_socket.ThreadedVSOCKSocketStreamTest.testStream
Summary of the results of the build (if available):
==
Click to see traceback logs
Traceback (most recent call last):
File "/home/buildbot/buildarea/3.x.cstratak-fedora-stable-ppc64le.lto-pgo/build/Lib/test/test_socket.py", line 550, in testStream
msg = self.conn.recv(1024)
PermissionError: [Errno 13] Permission denied
|
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️
Hi! The buildbot PPC64LE Fedora Stable LTO 3.x has failed when building commit 48c70b8.
What do you need to do:
- Don't panic.
- Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
- Go to the page of the buildbot that failed (https://buildbot.python.org/#/builders/541/builds/4746) and take a look at the build logs.
- Check if the failure is related to this commit (48c70b8) or if it is a false positive.
- If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.
You can take a look at the buildbot page here:
https://buildbot.python.org/#/builders/541/builds/4746
Failed tests:
Failed subtests:
- testStream - test.test_socket.ThreadedVSOCKSocketStreamTest.testStream
Summary of the results of the build (if available):
==
Click to see traceback logs
Traceback (most recent call last):
File "/home/buildbot/buildarea/3.x.cstratak-fedora-stable-ppc64le.lto/build/Lib/test/test_socket.py", line 550, in testStream
msg = self.conn.recv(1024)
PermissionError: [Errno 13] Permission denied
|
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️
Hi! The buildbot PPC64LE Fedora Stable Refleaks 3.x has failed when building commit 48c70b8.
What do you need to do:
- Don't panic.
- Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
- Go to the page of the buildbot that failed (https://buildbot.python.org/#/builders/280/builds/1480) and take a look at the build logs.
- Check if the failure is related to this commit (48c70b8) or if it is a false positive.
- If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.
You can take a look at the buildbot page here:
https://buildbot.python.org/#/builders/280/builds/1480
Failed tests:
Failed subtests:
- testStream - test.test_socket.ThreadedVSOCKSocketStreamTest.testStream
Summary of the results of the build (if available):
==
Click to see traceback logs
Traceback (most recent call last):
File "/home/buildbot/buildarea/3.x.cstratak-fedora-stable-ppc64le.refleak/build/Lib/test/test_socket.py", line 550, in testStream
msg = self.conn.recv(1024)
PermissionError: [Errno 13] Permission denied
|
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
Looks good! Benchmark results:
- ~1% improvement on free-threaded builds
- ~neutral on default builds
I don't think a 0.3-0.5% slower on average with a 13% slowdown on the gc_traversal benchmark is "neutral".
It might be just noise, but the large slowdown on the gc_traversal benchmark suggests a real issue.
Could we revert this PR until we have a better understanding of what is going on here?
|
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
|
The gc_traversal benchmark results sure looks like noise to me. That benchmark is dominated by list_traverse and functions in the GC, none of which have changed.
How about you run the benchmarks in the faster-cpython infrastructure and see if you get the same results?
|
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
DavidChoi76
pushed a commit
to realtime-dam-operation-using-AI/ras-commander
that referenced
this pull request
Nov 9, 2026
…s violation
watch_h5py_reads() assigned and restored h5py.AttributeManager.__getitem__ on
every context entry (several times per routed test). After ~1000 such class
modifications in one worker, CPython 3.13 runs out of type versions for the
class (MAX_VERSIONS_PER_CLASS); PyType_Modified() then returns early and no
longer clears the BINARY_SUBSCR_GETITEM cache, a borrowed pointer to the
per-context wrapper closure. Once that closure was freed, `obj.attrs[k]` ran
freed code (silently recording into a dead list) or faulted in
PyEval_EvalFrameDefault (cmp [rbx+88h] = func_version of the freed function).
This is the intermittent "access violation" in _fast_hdf_typed.read_attrs /
read_attr seen under pytest-xdist; it is not in HDF5 or the native engine.
CPython 3.14 fixed the cache (python/cpython#127737); 3.13.16 is affected.
The wrappers are now module-lifetime functions installed once per process
(reinstalled only if a restore removed them, refused if a foreign patch is
active); contexts only toggle recording. Regression tests cover wrapper
identity across contexts, nested contexts, foreign patches, and a subprocess
stress (900 contexts with allocator churn) that dies with 0xC0000005 on the
old guard.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Uh oh!
There was an error while loading. Please reload this page.
Assignees
No one assigned
Milestone
No milestone
Development
Successfully merging this pull request may close these issues.
Uh oh!
There was an error while loading. Please reload this page.
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
|