GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — gh-149584: `_remote_debugging`: Read stack chunks directly instead of through the page cache by maurycy · Pull Request #160888 · python/cpython

Conversation

@maurycy

@maurycy maurycy commented Oct 29, 2026 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Contributor

process_single_stack_chunk would read the chunk with _Py_RemoteDebug_ReadRemoteMemory instead of going through the page cache. Reading chunks through the page cache doesn't really do anything:

  • on Linux pages are 4KB, so it goes straight to process_vm_readv
  • on macOS the chunk is exactly one page, the sampler keeps its own cache and nothing else in the sample reads that page, so all the paged read adds is a retry after the first KERN_INVALID_ADDRESS

That retry can only succeed after the target has mapped a new chunk at the same address, so the sampler ends up parsing a chunk that is not the one the thread state pointed at. Invalid stack chunk size 0 is the symptom (a live chunk never has size 0):

maurycy@gimel cpython-rd-base (main 312222f) % sudo dtrace -q -Z -x dynvarsize=64m -n '
  pid$target:_remote_debugging*:copy_stack_chunks:entry { self->in_copy = 1; self->fails = 0; }
  pid$target:_remote_debugging*:copy_stack_chunks:return /self->in_copy/ { @per_copy[self->fails] = count(); self->in_copy = 0; }
  pid$target::mach_vm_read_overwrite:entry /self->in_copy/ { self->addr = arg1; }
  pid$target::mach_vm_read_overwrite:return /self->in_copy && self->addr/ { @kr[arg1] = count(); self->fails += arg1 != 0; self->addr = 0; }
  END { printa("  chunk read kr=%d: %@d\n", @kr); printa("  copies with %d failed reads: %@d\n", @per_copy); }' \
  -c "./python.exe repro_chunk_retry.py 5000"
  chunk read kr=1: 954
  chunk read kr=0: 10607
  copies with 2 failed reads: 315
  copies with 1 failed reads: 324
  copies with 0 failed reads: 4361

324 copies got KERN_INVALID_ADDRESS and then a successful read of the same address a few microseconds later.

deep_recursion oracle target at 100kHz, cache_frames=True, 3s:

error main direct chunk read
chunk read failed (size 16384, kern_return_t 1) 41575 35592
frame read failed (size 88, kern_return_t 1) 24037 37221
Incomplete sample: did not reach base frame 8659 7027
Invalid stack chunk size 0 669 17
total failed samples 74940 (25.0%) 79857 (26.6%)

On Linux perf stat counts the same number of process_vm_readv calls with both builds. The retried reads are reported as the original read failure now, so the error rate goes up on Oracle.

Not newsworthy IMO. I don't know how to create a not racy test here.

Context: I've found this one while reviving maurycy#3 and investigating some Oracle numbers when aliasing.

  • Issue: _remote_debugging: reading whole pages over and over #149584

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
@maurycy
up
da6b2c0
@maurycy
maurycy requested a review from pablogsal as a code owner October 29, 2026 13:15
@bedevere-app bedevere-app Bot added the awaiting review label Oct 29, 2026
@bedevere-app bedevere-app Bot mentioned this pull request Oct 29, 2026
_remote_debugging: reading whole pages over and over #149584
Open
pablogsal
pablogsal approved these changes Oct 29, 2026
View reviewed changes
@bedevere-app bedevere-app Bot added awaiting merge and removed awaiting review labels Oct 29, 2026
@pablogsal pablogsal added awaiting review needs backport to 3.15 bugs and security fixes and removed awaiting merge labels Oct 29, 2026
@pablogsal
pablogsal enabled auto-merge (squash) October 29, 2026 14:51
@maurycy

maurycy commented Oct 29, 2026 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Contributor Author

@pablogsal Thank you. Do we want NEWS here?

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@maurycy maurycy mentioned this pull request Oct 31, 2026
gh-149584: _remote_debugging: Map remote pages with mach_vm_remap on macOS instead of copying them #161011
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@pablogsal pablogsal pablogsal approved these changes

Uh oh!

There was an error while loading. Please reload this page.

Assignees

No one assigned

Labels

awaiting review needs backport to 3.15 bugs and security fixes

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Uh oh!

There was an error while loading. Please reload this page.

2 participants

@maurycy @pablogsal
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
You are visitor 01402520 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.