GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — GH-126910: Add GNU backtrace support for unwinding JIT frames by diegorusso · Pull Request #149104 · python/cpython

Conversation

@diegorusso

@diegorusso diegorusso commented Apr 28, 2026 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Contributor
  • Issue: Supporting stack unwinding in the JIT compiler #126910

📚 Documentation preview 📚: https://cpython-previews--149104.org.readthedocs.build/

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
@diegorusso
diegorusso requested review from a team, AA-Turner, Fidget-Spinner, ZeroIntensity, brandtbucher, emmatyping, ericsnowcurrently, erlend-aasland, markshannon and savannahostrowski as code owners April 28, 2026 16:12
@bedevere-app bedevere-app Bot added the awaiting core review label Apr 28, 2026
@bedevere-app bedevere-app Bot mentioned this pull request Apr 28, 2026
Supporting stack unwinding in the JIT compiler #126910
Open
@diegorusso
diegorusso requested a review from pablogsal April 28, 2026 16:12
@diegorusso

diegorusso commented Apr 28, 2026

Copy link
Copy Markdown
Contributor Author

Performance wise, I didn't see any hit. Numbers are in the noise.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@read-the-docs-community

read-the-docs-community Bot commented Apr 28, 2026 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown

Documentation build overview

📚 cpython-previews | 🛠️ Build #32535375 | 📁 Comparing 75b3be8 against main (2b6a137)

  🔍 Preview build  

25 files changed · ± 25 modified

± Modified

  • c-api/perfmaps.html
  • faq/programming.html
  • howto/descriptor.html
  • howto/perf_profiling.html
  • library/asyncio-task.html
  • library/compression.zstd.html
  • library/concurrency.html
  • library/email.policy.html
  • library/faulthandler.html
  • library/functions.html
  • and 15 more...
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@diegorusso
pythonGH-126910: Add GNU backtrace support for unwinding JIT frames
db967dc
@diegorusso
diegorusso force-pushed the jit-backtrace-support branch from abc1072 to db967dc Compare May 2, 2026 17:59
@diegorusso

diegorusso commented May 2, 2026

Copy link
Copy Markdown
Contributor Author

Apologies.. I had to force push.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

diegorusso
diegorusso commented May 2, 2026
View reviewed changes
Comment thread Include/internal/pycore_jit_unwind.h Resolved

Uh oh!

There was an error while loading. Please reload this page.

blurb-it Bot and others added 2 commits May 2, 2026 18:02
@blurb-it
📜🤖 Added by blurb_it.
0979429
@pablogsal
Guard JIT backtrace unwind
e8de85d
pablogsal
pablogsal reviewed May 2, 2026
View reviewed changes
Comment thread Include/internal/pycore_jit_unwind.h Outdated

#if defined(_Py_JIT) && defined(__linux__) && defined(__ELF__)
# define PY_HAVE_JIT_GDB_UNWIND
# if defined(HAVE_EXECINFO_H) && defined(HAVE_BACKTRACE)

@pablogsal pablogsal May 2, 2026

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This needs a separate configure link probe. HAVE_BACKTRACE only tells us that backtrace() is available, but this code also calls libgcc-specific symbols: __register_frame and __deregister_frame.

For example, Linux builds with clang -rtlib=compiler-rt can have execinfo.h / backtrace() while those frame registration symbols are not linkable. In that case thismacro gets enabled and the build fails later at link time.

Please add a configure check for __register_frame __deregister_frame and gate PY_HAVE_JIT_GNU_BACKTRACE_UNWIND on that result.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

diegorusso reacted with thumbs up emoji
All reactions
  • 👍 1 reaction

@pablogsal pablogsal May 2, 2026 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For example this can break in:

  • Alpine / musl + libexecinfo (no libgcc_s __register_frame)
  • clang + compiler-rt without libgcc_s
  • LLVM libunwind builds (different FDE table format)

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

diegorusso reacted with thumbs up emoji
All reactions
  • 👍 1 reaction

@diegorusso diegorusso May 2, 2026

Copy link
Copy Markdown
Contributor Author

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok, it makes sense.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

@diegorusso diegorusso May 3, 2026

Copy link
Copy Markdown
Contributor Author

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

link probe implemented here: ef9ea52

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
pablogsal
pablogsal reviewed May 2, 2026
View reviewed changes
Comment thread Lib/test/test_frame_pointer_unwind.py
@support.requires_gil_enabled("test requires the GIL enabled")
@unittest.skipIf(support.is_wasi, "test not supported on WASI")
@unittest.skipUnless(sys.platform == "linux", "GNU backtrace unwinding test requires Linux")
class GnuBacktraceUnwindTests(unittest.TestCase):

@pablogsal pablogsal May 2, 2026 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The new test only asserts python_frames > 0 and jit_frames > 0, which a stub unwinder would pass. Two things that would help:

  • Tighten to python_frames >= 10 (the recursion depth) so the count actually means something.
  • Add a test that JIT frames disappear from backtrace() after the executor is freed: that's the property the deregister code exists to guarantee and nothing covers it today.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

diegorusso reacted with thumbs up emoji
All reactions
  • 👍 1 reaction

@diegorusso diegorusso May 3, 2026

Copy link
Copy Markdown
Contributor Author

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Addressed here: 01df239

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
pablogsal
pablogsal reviewed May 2, 2026
View reviewed changes
Comment thread Include/internal/pycore_jit_publish.h
* A NULL result is valid when publication succeeded only through backends
* with no unregister step, such as perf map output.
*/
_PyJitCodeRegistration *_PyJit_RegisterCode(const void *code_addr,

@pablogsal pablogsal May 2, 2026 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

_PyJit_RegisterCode returns NULL for three different reasons (perf-only success, calloc failure, all backends failed) and the caller can't tell them apart. Could you rename registered -> any_registered and add a one-liner near the perf branch noting it's intentionally not counted? The deleted comment from jit_record_code about partial-failure being non-fatal would also be nice to restore.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

@diegorusso diegorusso May 3, 2026

Copy link
Copy Markdown
Contributor Author

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Addressed here: cf16da0

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
diegorusso
diegorusso commented May 3, 2026
View reviewed changes
Comment thread Python/jit_publish.c Outdated
#if defined(PY_HAVE_JIT_GDB_UNWIND) \
|| defined(PY_HAVE_JIT_GNU_BACKTRACE_UNWIND)
struct _PyJitCodeRegistration {
# if defined(PY_HAVE_JIT_GDB_UNWIND)

@diegorusso diegorusso May 2, 2026

Copy link
Copy Markdown
Contributor Author

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we really need to guard every field of the struct?

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

@pablogsal pablogsal May 5, 2026

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The current version is valid, but the per-field guards are probably more defensive than necessary. I added them to keep the registration object matching the enabled backends, but since the struct is private and the fields are opaque handles, we can simplify the struct and keep the feature guards around the actual register/unregister code.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
diegorusso added 4 commits May 3, 2026 01:46
@diegorusso
Link probe HAVE_LIBGCC_EH_FRAME_REGISTRATION
ef9ea52
@diegorusso
Strengthen JIT unwind tests
01df239
@diegorusso
Clarify JIT publication registration semantics
cf16da0
@diegorusso
Add Pablo
5cd2e87
@diegorusso
diegorusso requested a review from corona10 as a code owner May 3, 2026 01:06
@pablogsal
fixup! Add Pablo
75b3be8
pablogsal
pablogsal approved these changes May 5, 2026
View reviewed changes

@pablogsal pablogsal left a comment

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I reviewed this and tested it locally again and it works in all cases I can find.

Great work @diegorusso 💪

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

Fidget-Spinner and diegorusso reacted with rocket emoji
All reactions
  • 🚀 2 reactions
@bedevere-app bedevere-app Bot added awaiting merge and removed awaiting core review labels May 5, 2026
@diegorusso
diegorusso merged commit 1e5d942 into python:main May 5, 2026
95 of 96 checks passed

Uh oh!

There was an error while loading. Please reload this page.

@bedevere-app bedevere-app Bot removed the awaiting merge label May 5, 2026
@diegorusso
diegorusso deleted the jit-backtrace-support branch May 5, 2026 08:43
@bedevere-bot

bedevere-bot commented May 5, 2026

Copy link
Copy Markdown

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot s390x RHEL9 LTO + PGO 3.x (tier-3) has failed when building commit 1e5d942.

What do you need to do:

  1. Don't panic.
  2. Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
  3. Go to the page of the buildbot that failed (https://buildbot.python.org/#/builders/1578/builds/4764) and take a look at the build logs.
  4. Check if the failure is related to this commit (1e5d942) or if it is a false positive.
  5. If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/#/builders/1578/builds/4764

Failed tests:

  • test_frame_pointer_unwind

Failed subtests:

  • test_manual_unwind_respects_frame_pointers - test.test_frame_pointer_unwind.FramePointerUnwindTests.test_manual_unwind_respects_frame_pointers

Summary of the results of the build (if available):

==

Click to see traceback logs
Traceback (most recent call last):
  File "/var/lib/buildbot/worker/cstratak-rhel9-s390x/3.x.cstratak-rhel9-s390x.lto-pgo/build/Lib/test/test_frame_pointer_unwind.py", line 241, in test_manual_unwind_respects_frame_pointers
    self.assertGreaterEqual(
    ~~~~~~~~~~~~~~~~~~~~~~~^
        python_frames,
        ^^^^^^^^^^^^^^
        STACK_DEPTH,
        ^^^^^^^^^^^^
        f"expected to find Python frames on {self.machine} with env {env}",
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
AssertionError: 1 not greater than or equal to 10 : expected to find Python frames on s390x with env {'PYTHON_JIT': '0'}
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@diegorusso

diegorusso commented May 5, 2026

Copy link
Copy Markdown
Contributor Author

The buildbot failure will be fixed by: #149362

hugovk reacted with thumbs up emoji
All reactions
  • 👍 1 reaction

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@encukou

encukou commented May 5, 2026

Copy link
Copy Markdown
Member

Looks like this fails on ppc64le: https://buildbot.python.org/#/builders/451/builds/6989

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@encukou

encukou commented May 5, 2026

Copy link
Copy Markdown
Member

Also Raspbian failures went from

{"length": 1, "python_frames": 0, "jit_frames": 0, "other_frames": 1, "jit_backend": null}

to

{"length": 0, "python_frames": 0, "jit_frames": 0, "other_frames": 0, "jit_backend": null, "unwinder": "gnu_backtrace_unwind"}

(even with #149362 applied)

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@diegorusso

diegorusso commented May 5, 2026

Copy link
Copy Markdown
Contributor Author

Looks like this fails on ppc64le: https://buildbot.python.org/#/builders/451/builds/6989

No, the GNU backtrace works on that platform. The one failing is test.test_frame_pointer_unwind.FramePointerUnwindTests.test_manual_unwind_respects_frame_pointers and very likely because the way we walk the fp chain.
On PPC64 the stack is defined like this: https://refspecs.linuxfoundation.org/ELF/ppc64/PPC-elf64abi-1.9.html?utm_source=openai#STACK
The ABI stack layout explicitly has Back chain (SP + 0), CR save area (SP + 8), and LR save area (SP + 16).

We need to update @pablogsal PR (#149362) to include the shape for ppc64.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@diegorusso diegorusso mentioned this pull request May 5, 2026
gh-149202: Fix frame pointer unwinding on s390x and ARM #149362
Merged
@stratakis stratakis mentioned this pull request May 6, 2026
gh-149202: Fix frame pointer unwinding on ppc64le and armv7/clang #149409
Merged
@encukou

encukou commented May 7, 2026

Copy link
Copy Markdown
Member

ppc64le is fixed now; but on Raspbian it is GNU backtrace that's failing -- see #149409 (comment)

hugovk reacted with hooray emoji
All reactions
  • 🎉 1 reaction

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@pablogsal pablogsal pablogsal approved these changes
@markshannon markshannon Awaiting requested review from markshannon markshannon is a code owner
@Fidget-Spinner Fidget-Spinner Awaiting requested review from Fidget-Spinner Fidget-Spinner is a code owner
@brandtbucher brandtbucher Awaiting requested review from brandtbucher brandtbucher is a code owner
@savannahostrowski savannahostrowski Awaiting requested review from savannahostrowski
@ericsnowcurrently ericsnowcurrently Awaiting requested review from ericsnowcurrently ericsnowcurrently is a code owner
@ZeroIntensity ZeroIntensity Awaiting requested review from ZeroIntensity
@erlend-aasland erlend-aasland Awaiting requested review from erlend-aasland erlend-aasland is a code owner
@AA-Turner AA-Turner Awaiting requested review from AA-Turner AA-Turner is a code owner
@emmatyping emmatyping Awaiting requested review from emmatyping emmatyping is a code owner
@corona10 corona10 Awaiting requested review from corona10 corona10 is a code owner

Uh oh!

There was an error while loading. Please reload this page.

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Uh oh!

There was an error while loading. Please reload this page.

4 participants

@diegorusso @bedevere-bot @encukou @pablogsal
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
You are visitor 01402503 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.