GeezerVison

Most changes to Python require a NEWS entry. Add one using the blurb_it web app or the blurb command-line tool.

If this change has little impact on Python users, wait for a maintainer to apply the skip news label instead.

All commit authors signed the Contributor License Agreement.

Most changes to Python require a NEWS entry. Add one using the blurb_it web app or the blurb command-line tool.

If this change has little impact on Python users, wait for a maintainer to apply the skip news label instead.

Additional context:

This PR is intended to fix the specific _io.BytesIO free-threaded race reported in gh-151640. The issue report focuses on the whole-buffer BytesIO.read() fast path returning self->buf directly via Py_NewRef(), while a concurrent writer may later resize that same internal bytes object.

While investigating the code path, I also found that BytesIO.getvalue() can expose the same internal buffer in a similar way. For that reason, this PR handles both whole-buffer read() and getvalue() in Py_GIL_DISABLED builds.

The chosen fix is intentionally conservative: in free-threaded builds, avoid returning the internal BytesIO buffer directly to user code, while keeping the existing fast path unchanged for regular GIL builds. This avoids changing behavior/performance for non-free-threaded builds and keeps the patch localized to _io.BytesIO.

I also considered fixing this lower in the resize/shared-buffer ownership path, but that seemed more invasive because it would require changing the interaction between SHARED_BUF(), _PyBytes_Resize(), and buffer ownership/copy-on-write behavior. If maintainers prefer preserving this optimization in free-threaded builds, I’m happy to rework the patch in that direction.

I think always copying buf is going to cause too much performance regression. In some cases the buffer is going to be large and so this copy will be significant. Tracking if the buf has been shared seems like a better solution. That sounds like your "more invasive" approach, which I think is worth the extra work and complexity. I prototyped this change here: https://github.com/nascheme/cpython/tree/gh-151640-bytesio-shared-flag

@nascheme Thanks for the prototype. I reworked the PR to follow the shared-buffer flag / copy-on-write approach, rather than always copying the buffer in free-threaded builds.

The update includes the concurrent regression test and the NEWS entry. All CI checks are now green. Local validation also included free-threaded Py_DEBUG/refleak and the targeted free-threaded ThreadSanitizer regression: 10/10 passes with no diagnostics.

The updated commit is 367b372.

I'll let @ZeroIntensity take a look too but I think this can be merged.

Thanks @pedramkarimii for the PR, and @ZeroIntensity for merging it 🌮🎉.. I'm working now to backport this PR to: 3.14, 3.15.
🐍🍒⛏🤖

Sorry, @pedramkarimii and @ZeroIntensity, I could not cleanly backport this to 3.15 due to a conflict.
Please backport using cherry_picker on command line.

cherry_picker 881a13a97c7a29b9de57913c773cd5f195270fbd 3.15

Sorry, @pedramkarimii and @ZeroIntensity, I could not cleanly backport this to 3.14 due to a conflict.
Please backport using cherry_picker on command line.

cherry_picker 881a13a97c7a29b9de57913c773cd5f195270fbd 3.14

GH-153151 is a backport of this pull request to the 3.15 branch.

GH-153153 is a backport of this pull request to the 3.14 branch.