GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — GH-139922: Tail calling for MSVC (VS 2026) by chris-eibl · Pull Request #143068 · python/cpython

Conversation

@chris-eibl

@chris-eibl chris-eibl commented Dec 22, 2025 •
edited by bedevere-app Bot
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Member

So much has happened on main that I started fresh over and sqash merged the needed things from #139962 onto latest main (mostly because I wanted to use it for my own experiments on main again).

Based on @markshannon's work in #142228 I've factored out _Py_VectorCallInstrumentation_StackRefSteal, because then @Fidget-Spinner's #138115 is not needed in this PR and can be done in a follow-up. An alternative would be to keep it and pass tstate into the Py_*_StackRefSteal functions.

  • Issue: Tail calling interpreter for MSVC #139922

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
chris-eibl added 3 commits December 21, 2025 14:51
@chris-eibl
rebased + _Py_VectorCallInstrumentation_StackRefSteal
374874c
@chris-eibl
Merge commit '374874c3e6d8c4745a52b3bc22044db607b2c9e7' into msvc-tai…
e0cd0f5
…l-call-new
@chris-eibl
try to fix MSVC tail call CI
7e5555a
@bedevere-app bedevere-app Bot mentioned this pull request Dec 22, 2025
Tail calling interpreter for MSVC #139922
Closed
@chris-eibl
chris-eibl marked this pull request as ready for review December 22, 2025 14:23
@chris-eibl
chris-eibl requested review from a team, AA-Turner, ezio-melotti, hugovk and markshannon as code owners December 22, 2025 14:23
@bedevere-app bedevere-app Bot added the awaiting review label Dec 22, 2025
@chris-eibl
chris-eibl requested a review from Fidget-Spinner December 22, 2025 14:23
@chris-eibl
Alternative to restrict and conditional Py_NO_INLINE for PyMapping_Ge…
a75042a
…tOptionalItem
Fidget-Spinner
Fidget-Spinner reviewed Dec 22, 2025
View reviewed changes
Comment thread Python/bytecodes.c Resolved

Uh oh!

There was an error while loading. Please reload this page.

Fidget-Spinner
Fidget-Spinner reviewed Dec 22, 2025
View reviewed changes
Comment thread Include/abstract.h Resolved

Uh oh!

There was an error while loading. Please reload this page.

@Fidget-Spinner

Fidget-Spinner commented Dec 22, 2025

Copy link
Copy Markdown
Member

In the future, please communicate if you want to supercede a PR for other contributors. It's often considered not good etiquette to take over another person's PR unless they explicitly allow it. However, I'm more than happy to let you take over the PR in this case.

chris-eibl reacted with thumbs up emoji
All reactions
  • 👍 1 reaction

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

chris-eibl
chris-eibl commented Dec 22, 2025
View reviewed changes
Comment thread Objects/abstract.c Outdated
// MSVC fails during a tail call release build with loads of
// error C4737: Unable to perform required tail call.
// without using Py_NO_INLINE here, but PGO works fine.
#if defined(_MSC_VER) && !defined(__clang__) && _Py_TAIL_CALL_INTERP && !defined(_Py_USING_PGO)

@chris-eibl chris-eibl Dec 22, 2025

Copy link
Copy Markdown
Member Author

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It needed lots of trial and error to come up with this unintuive minimal change needed to convince MSVC that the local variable passed by reference into result here does not escape the tail call.

Using it as an return value in PyMapping_GetOptionalItem2 is more explicit, but might have other drawbacks?

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
@chris-eibl

chris-eibl commented Dec 22, 2025

Copy link
Copy Markdown
Member Author

In the future, please communicate if you want to supercede a PR for other contributors. It's often considered not good etiquette to take over another person's PR unless they explicitly allow it. However, I'm more than happy to let you take over the PR in this case.

Sorry, wasn't ment as superseding, more as a follow up and maybe just a draft for new discussions. Thought about you're gonna take whatever deems appropriate back into your original PR ...

If this here is going to fly, I'd like to include you and Brandt as authors for the commits, but don't know how atm ...

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@hugovk

hugovk commented Dec 22, 2025

Copy link
Copy Markdown
Member

If this here is going to fly, I'd like to include you and Brandt as authors for the commits, but don't know how atm ...

Co-authored-by is the way:

https://docs.github.com/en/pull-requests/committing-changes-to-your-project/creating-and-editing-commits/creating-a-commit-with-multiple-authors

chris-eibl reacted with heart emoji
All reactions
  • ❤️ 1 reaction

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@Fidget-Spinner

Fidget-Spinner commented Dec 22, 2025

Copy link
Copy Markdown
Member

If this here is going to fly, I'd like to include you and Brandt as authors for the commits, but don't know how atm ...

Oh I'm happy to be superseded. If you'd like to attribute commit history though, you can just push a trivial commit with the co-author info like Hugo pointed out, and when I squash and merge I'll use that.

chris-eibl reacted with thumbs up emoji
All reactions
  • 👍 1 reaction

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@chris-eibl @Fidget-Spinner @brandtbucher
make _PyMapping_GetOptionalItem2 private and comment scopes
fdfd8a0
Co-Authored-By: Ken Jin <[email protected]>
Co-Authored-By: Brandt Bucher <[email protected]>
@Fidget-Spinner

Fidget-Spinner commented Dec 22, 2025

Copy link
Copy Markdown
Member

Try pulling main in, I fixed a JIT bug there that should turn the JIT CI green again.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@chris-eibl
Merge with main because aarch64 jit builds started failing
058889e
@chris-eibl

chris-eibl commented Dec 22, 2025

Copy link
Copy Markdown
Member Author

Co-authored-by is the way:

Done

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

hugovk
hugovk reviewed Dec 22, 2025
View reviewed changes
Comment thread .github/workflows/tail-call.yml Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread .github/workflows/tail-call.yml Resolved

Uh oh!

There was an error while loading. Please reload this page.

chris-eibl
chris-eibl commented Dec 22, 2025
View reviewed changes
Comment thread .github/workflows/tail-call.yml Resolved

Uh oh!

There was an error while loading. Please reload this page.

chris-eibl and others added 2 commits December 22, 2025 17:44
@chris-eibl @hugovk
Apply suggestions from code review
c8d051b
Co-authored-by: Hugo van Kemenade <[email protected]>
@chris-eibl @hugovk
Update .github/workflows/tail-call.yml
ab70524
I could not add this one to batch, so have to do it separately

Co-authored-by: Hugo van Kemenade <[email protected]>
@chris-eibl

chris-eibl commented Dec 22, 2025

Copy link
Copy Markdown
Member Author

https://github.com/python/cpython/actions/runs/20437258430/job/58722460806 failed inbetween
test_extractall_none_gid (test.test_tarfile.NoneInfoExtractTests_Tar.test_extractall_none_gid) ... Windows fatal exception: access violation
(but got green when rerunning the failed test) even though pulling in main.

I've found this: #143073

Hopefully, this will fix it ...

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@chris-eibl
enable "Emulated Windows Clang (release)"
5c1c240
try to fix "Native Windows MSVC (release)"
@Fidget-Spinner

Fidget-Spinner commented Dec 22, 2025 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Member

Just ignore the JIT problems for now. The new JIT executor management has a known bug #143026

chris-eibl reacted with thumbs up emoji
All reactions
  • 👍 1 reaction

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

chris-eibl added 2 commits December 22, 2025 18:26
@chris-eibl
Merge remote-tracking branch 'upstream/main' into msvc-tail-call-new
a04824f
@chris-eibl
revert to cmd but keep Hugo's other changes
da1adaf
add --no-progress to choco
@Fidget-Spinner Fidget-Spinner added the 🔨 test-with-refleak-buildbots Test PR w/ refleak buildbots; report in status section label Dec 22, 2025
@bedevere-bot

bedevere-bot commented Dec 22, 2025

Copy link
Copy Markdown

🤖 New build scheduled with the buildbot fleet by @Fidget-Spinner for commit da1adaf 🤖

Results will be shown at:

https://buildbot.python.org/all/#/grid?branch=refs%2Fpull%2F143068%2Fmerge

If you want to schedule another build, you need to add the 🔨 test-with-refleak-buildbots label again.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@bedevere-bot bedevere-bot removed the 🔨 test-with-refleak-buildbots Test PR w/ refleak buildbots; report in status section label Dec 22, 2025
@Fidget-Spinner

Fidget-Spinner commented Dec 22, 2025 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Member

Benchmark results are within noise for the default interpreter https://github.com/facebookexperimental/free-threading-benchmarking/blob/main/results/bm-20251222-3.15.0a3%2B-da1adaf/bm-20251222-vultr-x86_64-chris%252deibl-msvc_tail_call_new-3.15.0a3%2B-da1adaf-vs-base.md

Refleak bots are green.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

Fidget-Spinner
Fidget-Spinner approved these changes Dec 22, 2025
View reviewed changes

@Fidget-Spinner Fidget-Spinner left a comment

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Once again:

  • This PR is the successor of one that has been up for 2 months already #139962 . We've had enough discussion there to get a sensing of what we want.
  • Refleak tests are green, performance results are within noise for the default (computed goto) interpreter.
  • This PR adds CI for the new configuration, which is also green.
  • I have discussed this feature with the other core devs a few months back on our bi-weekly perf meetings. Everyone was in agreement that adding scopes to MSVC is fine as it isn't that invasive.
  • This PR is the minimal changeset required to get TC working on MSVC.
  • It uses the techniques suggested by Mark in the other PR (just adding scopes and commenting on them). So I expect he should be okay with the overall design I hope. I understand he might disagree on some of the smaller changes.
  • It LGTM, from my review of the changes.

Therefore, I'm merging this PR.

Any follow-ups can be done on future PRs. Any discussion about the design can be done on the issue itself.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

chris-eibl reacted with thumbs up emoji stonebig reacted with heart emoji
All reactions
  • 👍 1 reaction
  • ❤️ 1 reaction
@bedevere-app bedevere-app Bot added awaiting merge and removed awaiting review labels Dec 22, 2025
@Fidget-Spinner
Fidget-Spinner merged commit be3c131 into python:main Dec 22, 2025
94 of 95 checks passed

Uh oh!

There was an error while loading. Please reload this page.

@bedevere-app bedevere-app Bot removed the awaiting merge label Dec 22, 2025
@hugovk hugovk mentioned this pull request Dec 23, 2025
Update pre-commit with zizmor and Ruff fixes #143095
Merged
@chris-eibl
chris-eibl deleted the msvc-tail-call-new branch December 26, 2025 17:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@hugovk hugovk hugovk left review comments
@Fidget-Spinner Fidget-Spinner Fidget-Spinner approved these changes
@markshannon markshannon Awaiting requested review from markshannon markshannon is a code owner
@ezio-melotti ezio-melotti Awaiting requested review from ezio-melotti ezio-melotti is a code owner
@AA-Turner AA-Turner Awaiting requested review from AA-Turner AA-Turner is a code owner

Uh oh!

There was an error while loading. Please reload this page.

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Uh oh!

There was an error while loading. Please reload this page.

4 participants

@chris-eibl @Fidget-Spinner @hugovk @bedevere-bot
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
You are visitor 01402503 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.