GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — gh-143008: fix TextIOWrapper.truncate via re-entrant flush by yihong0618 · Pull Request #143041 · python/cpython

Conversation

@yihong0618

@yihong0618 yihong0618 commented Dec 21, 2025 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Contributor

This patch fix all re-entrant flush problem
by adding an attr flushing
and. refactor the file check to _textiowrapper_flush

  • Issue: Null pointer dereference in TextIOWrapper.truncate via re-entrant flush #143008

all crash problem in 143008 can be fix cc @jackfromeast can you help to check?
thank you very much!

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
@yihong0618
fix: TextIOWrapper.truncate via re-entrant flush
f7eca0d
Signed-off-by: yihong0618 <[email protected]>
@bedevere-app bedevere-app Bot mentioned this pull request Dec 21, 2025
Null pointer dereference in TextIOWrapper.truncate via re-entrant flush #143008
Closed
@bedevere-app bedevere-app Bot added the awaiting review label Dec 21, 2025
cmaloney
cmaloney reviewed Dec 21, 2025
View reviewed changes

@cmaloney cmaloney left a comment

Copy link
Copy Markdown
Contributor

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't like adding a new flushing variable here. To me it doesn't solves the core issue: Code assumes self->buffer is non-NULL because of checks which become invalid by the time it's actually dereferenced/used (TOCTOU general bug class). I think TextIOWrapper already has the information we need to be looking at / checking for (CHECK_ATTACHED macro / self->detached, self->buffer == NULL, and self->ok, ...). Those seem to be tracking very close to the same thing and should be usable to solve this group of issues.

I think it would be better here to use CHECK_ATTACHED just before self->buffer is used / passed to a call / returned; making sure there aren't any other calls to interpreter methods betwen the CHECK_ATTACHED + self->buffer usage / dereference.

I suspect BufferedIO will need a similar set of fixes: It also has a concept of an underlying stream that can be "detached" and which user code may detach during various operations.

Some comments on specifics inline, the test_textio ones I think are important in further changes, implementation details less so if not adding/using the flushing member.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

serhiy-storchaka reacted with thumbs up emoji yihong0618 reacted with heart emoji
All reactions
  • 👍 1 reaction
  • ❤️ 1 reaction
Comment thread Lib/test/test_io/test_textio.py Outdated
# instead of crashing.
wrapper = None

class BadRaw(self.RawIOBase):

@cmaloney cmaloney Dec 21, 2025

Copy link
Copy Markdown
Contributor

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There's a number of mock helpers defined in test_io/utils.py to make it so can reduce this to just the methods that matter. I think self.MockRawIO will work here

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

@yihong0618 yihong0618 Dec 22, 2025

Copy link
Copy Markdown
Contributor Author

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
Comment thread Lib/test/test_io/test_textio.py Outdated
with self.subTest(name):
wrapper = self.TextIOWrapper(EvilBuffer(BadRaw()), encoding='utf-8')
self.assertRaisesRegex(RuntimeError, "reentrant", method)
wrapper = None

@cmaloney cmaloney Dec 21, 2025

Copy link
Copy Markdown
Contributor

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can we make the gc / closing here more explicit? del wrapper?

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

@yihong0618 yihong0618 Dec 22, 2025

Copy link
Copy Markdown
Contributor Author

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
Comment thread Misc/NEWS.d/next/Library/2025-12-21-17-56-37.gh-issue-143008.aakErJ.rst Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread Modules/_io/textio.c Outdated
static inline int
_textiowrapper_flush(textio *self)
{
self->flushing = 1;

@cmaloney cmaloney Dec 21, 2025

Copy link
Copy Markdown
Contributor

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why can't this be in _io_TextIOWrapper_flush_impl?

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

@yihong0618 yihong0618 Dec 22, 2025

Copy link
Copy Markdown
Contributor Author

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it can, the design is cover it in _PyFile_Flush

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

@yihong0618 yihong0618 Dec 29, 2025

Copy link
Copy Markdown
Contributor Author

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

dropped

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
Comment thread Modules/_io/textio.c
PyObject *ret;
self->flushing = 1;
do {
ret = PyObject_CallMethodOneArg(self->buffer, &_Py_ID(write), b);

@cmaloney cmaloney Dec 21, 2025

Copy link
Copy Markdown
Contributor

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

every loop iteration / write call could execute arbitrary interpreter code that could invalidate self->buffer so likely need to check on each loop iteration.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

@yihong0618 yihong0618 Dec 22, 2025

Copy link
Copy Markdown
Contributor Author

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

will check, but I think maybe not need

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

@yihong0618 yihong0618 Dec 29, 2025

Copy link
Copy Markdown
Contributor Author

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

dropped

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
@yihong0618

yihong0618 commented Dec 21, 2025

Copy link
Copy Markdown
Contributor Author

I don't like adding a new flushing variable here. To me it doesn't solves the core issue: Code assumes self->buffer is non-NULL because of checks which become invalid by the time it's actually dereferenced/used (TOCTOU general bug class). I think TextIOWrapper already has the information we need to be looking at / checking for (CHECK_ATTACHED macro / self->detached, self->buffer == NULL, and self->ok, ...). Those seem to be tracking very close to the same thing and should be usable to solve this group of issues.

I think it would be better here to use CHECK_ATTACHED just before self->buffer is used / passed to a call / returned; making sure there aren't any other calls to interpreter methods betwen the CHECK_ATTACHED + self->buffer usage / dereference.

I suspect BufferedIO will need a similar set of fixes: It also has a concept of an underlying stream that can be "detached" and which user code may detach during various operations.

Some comments on specifics inline, the test_textio ones I think are important in further changes, implementation details less so if not adding/using the flushing member.

I think you are right but CHECK_ATTACHED in the middle of code is not a good name and easy to forget add it.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@yihong0618

yihong0618 commented Dec 21, 2025

Copy link
Copy Markdown
Contributor Author

And I can change it to my first solution or waiting for other discuss here?

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@yihong0618 @cmaloney
Update Misc/NEWS.d/next/Library/2025-12-21-17-56-37.gh-issue-143008.a…
a96757b
…akErJ.rst

Co-authored-by: Cody Maloney <[email protected]>
@cmaloney

cmaloney commented Dec 22, 2025

Copy link
Copy Markdown
Contributor

I agree CHECK_ATTACHED is easy to forget and I'd argue this bug is the result of some cases missing it.. It is a relatively small change though and matches the rest of the textio.c code style / convention. I have been working towards changing textio.c more generally, but that's too big for a likely back-ported bugfix.

Can you explain what your first solution is? I'm not understanding what you're referring to.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@yihong0618

yihong0618 commented Dec 22, 2025

Copy link
Copy Markdown
Contributor Author

I agree CHECK_ATTACHED is easy to forget and I'd argue this bug is the result of some cases missing it.. It is a relatively small change though and matches the rest of the textio.c code style / convention. I have been working towards changing textio.c more generally, but that's too big for a likely back-ported bugfix.

Can you explain what your first solution is? I'm not understanding what you're referring to.

#143008 (comment)

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

serhiy-storchaka
serhiy-storchaka reviewed Dec 23, 2025
View reviewed changes

@serhiy-storchaka serhiy-storchaka left a comment

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This will break the case when flush() was intentionally overridden.

I am not also sure that it fixes the core issue, it just makes a particular reproducer no longer working.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
@yihong0618

yihong0618 commented Dec 24, 2025

Copy link
Copy Markdown
Contributor Author

This will break the case when flush() was intentionally overridden.

I am not also sure that it fixes the core issue, it just makes a particular reproducer no longer working.

will dig it, thank you

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@yihong0618

yihong0618 commented Dec 29, 2025

Copy link
Copy Markdown
Contributor Author

I agree CHECK_ATTACHED is easy to forget and I'd argue this bug is the result of some cases missing it.. It is a relatively small change though and matches the rest of the textio.c code style / convention. I have been working towards changing textio.c more generally, but that's too big for a likely back-ported bugfix.

Can you explain what your first solution is? I'm not understanding what you're referring to.

after some check and work I do not think CHECK_ATTACHED before every is a good design will dig more
maybe I can find some better solution to fix all of them, if I failed will close this waiting for some expert to finish
this, thank you very much for review

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@yihong0618
fix: apply some suggestion
65d7dbb
Signed-off-by: yihong0618 <[email protected]>
@yihong0618

yihong0618 commented Dec 29, 2025 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Contributor Author

There's a number of mock helpers defined in test_io/utils.py to make it so can reduce this to just the methods that matter. I think self.MockRawIO will work here

I made some change but not sure its better idea, can you help to check? Thank you very much.


if you have better way to fix all of them feel free to close it~ thanks again

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@picnixz picnixz linked an issue Jan 1, 2026 that may be closed by this pull request
Null pointer dereference in TextIOWrapper.truncate via re-entrant flush #143008
Closed
@picnixz picnixz removed a link to an issue Jan 1, 2026
Null pointer dereference in TextIOWrapper.truncate via re-entrant flush #143008
Closed
@cmaloney

cmaloney commented Jan 5, 2026

Copy link
Copy Markdown
Contributor

I'm starting to think a small refactor of TextIOWrapper to add and use a helper that always checks self->ok + self->attached would be better here; wrote up more thoughts in #143008 (comment)

yihong0618 reacted with thumbs up emoji
All reactions
  • 👍 1 reaction

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@yihong0618

yihong0618 commented Jan 5, 2026

Copy link
Copy Markdown
Contributor Author

I'm starting to think a small refactor of TextIOWrapper to add and use a helper that always checks self->ok + self->attached would be better here; wrote up more thoughts in #143008 (comment)

will try to understand and learn it thank you

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

cmaloney added a commit to cmaloney/cpython that referenced this pull request Mar 15, 2026
@cmaloney @yihong0618
pythongh-143008: Safer TextIOWrapper underlying stream access
42caa08
TextIOWrapper keeps its underlying stream in a member called
`self->buffer`. That stream can be detached by user code, such as custom
`.flush` implementations resulting in `self->buffer` being set to NULL.
The implementation often checked at the start of functions if
`self->buffer` is in a good state, but did not always recheck after
other Python code was called which could modify `self->buffer`.

The cases which need to be re-checked are hard to spot so rather than
rely on reviewer effort create better safety by making all self->buffer
access go through helper functions.

Thank you yihong0618 for the test, NEWS and initial implementation in
pythongh-143041.

Co-authored-by: yihong0618 <[email protected]>
@cmaloney cmaloney mentioned this pull request Mar 15, 2026
gh-143008: Fix Null pointer dereferences in TextIOWrapper underlying stream access #145957
Merged
@github-actions

github-actions Bot commented May 4, 2026

Copy link
Copy Markdown

This PR is stale because it has been open for 30 days with no activity.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@github-actions github-actions Bot added the stale Stale PR or inactive for long period of time. label May 4, 2026
vstinner added a commit that referenced this pull request Jun 9, 2026
@cmaloney @yihong0618 @vstinner
gh-143008: Fix Null pointer dereferences in TextIOWrapper underlying …
db4b194
…stream access (#145957)

TextIOWrapper keeps its underlying stream in a member called
`self->buffer`. That stream can be detached by user code, such as custom
`.flush` implementations resulting in `self->buffer` being set to NULL.
The implementation often checked at the start of functions if
`self->buffer` is in a good state, but did not always recheck after
other Python code was called which could modify `self->buffer`.

The cases which need to be re-checked are hard to spot so rather than
rely on reviewer effort create better safety by making all self->buffer
access go through helper functions.

Thank you yihong0618 for the test, NEWS and initial implementation in
gh-143041.

Co-authored-by: yihong0618 <[email protected]>
Co-authored-by: Victor Stinner <[email protected]>
@vstinner

vstinner commented Jun 9, 2026

Copy link
Copy Markdown
Member

#145957 was merged instead. Thanks for your contribution anyway! The merged change is based on your work and you're credited in the change ;-)

cmaloney reacted with hooray emoji
All reactions
  • 🎉 1 reaction

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@vstinner vstinner closed this Jun 9, 2026
philthompson10 pushed a commit to philthompson10/cpython that referenced this pull request Jun 17, 2026
@cmaloney @yihong0618
@vstinner @philthompson10
pythongh-143008: Fix Null pointer dereferences in TextIOWrapper under…
c24e0bb
…lying stream access (python#145957)

TextIOWrapper keeps its underlying stream in a member called
`self->buffer`. That stream can be detached by user code, such as custom
`.flush` implementations resulting in `self->buffer` being set to NULL.
The implementation often checked at the start of functions if
`self->buffer` is in a good state, but did not always recheck after
other Python code was called which could modify `self->buffer`.

The cases which need to be re-checked are hard to spot so rather than
rely on reviewer effort create better safety by making all self->buffer
access go through helper functions.

Thank you yihong0618 for the test, NEWS and initial implementation in
pythongh-143041.

Co-authored-by: yihong0618 <[email protected]>
Co-authored-by: Victor Stinner <[email protected]>
vstinner added a commit that referenced this pull request Oct 16, 2026
@vstinner @cmaloney
@yihong0618 @brijkapadia @ashm-dev
[3.15] gh-157364: Fix use-after-free in io.TextIOWrapper during reent…
ea1312b
…rant detach (#159591)

* gh-143008: Fix Null pointer dereferences in TextIOWrapper underlying stream access (#145957)

TextIOWrapper keeps its underlying stream in a member called
`self->buffer`. That stream can be detached by user code, such as custom
`.flush` implementations resulting in `self->buffer` being set to NULL.
The implementation often checked at the start of functions if
`self->buffer` is in a good state, but did not always recheck after
other Python code was called which could modify `self->buffer`.

The cases which need to be re-checked are hard to spot so rather than
rely on reviewer effort create better safety by making all self->buffer
access go through helper functions.

Thank you yihong0618 for the test, NEWS and initial implementation in
gh-143041.

Co-authored-by: yihong0618 <[email protected]>
Co-authored-by: Victor Stinner <[email protected]>
(cherry picked from commit db4b194)

* gh-154523: Fix data-race in `TextIOWrapper.detach()` (#154565)

(cherry picked from commit 8440a6a)

* gh-157364: Fix use-after-free in io.TextIOWrapper during reentrant detach (#157370)

Co-authored-by: Victor Stinner <[email protected]>
(cherry picked from commit e19dc47)

---------

Co-authored-by: Cody Maloney <[email protected]>
Co-authored-by: yihong0618 <[email protected]>
Co-authored-by: Brij Kapadia <[email protected]>
Co-authored-by: Shamil <[email protected]>
@vstinner vstinner mentioned this pull request Oct 16, 2026
[3.14] gh-157364: Fix use-after-free in io.TextIOWrapper during reentrant detach #159600
Merged
vstinner added a commit that referenced this pull request Oct 16, 2026
@vstinner @cmaloney
@yihong0618 @brijkapadia @ashm-dev
[3.14] gh-157364: Fix use-after-free in io.TextIOWrapper during reent…
13fc16c
…rant detach (#159600)

[3.15] gh-157364: Fix use-after-free in io.TextIOWrapper during reentrant detach (#159591)

* gh-143008: Fix Null pointer dereferences in TextIOWrapper underlying stream access (#145957)

TextIOWrapper keeps its underlying stream in a member called
`self->buffer`. That stream can be detached by user code, such as custom
`.flush` implementations resulting in `self->buffer` being set to NULL.
The implementation often checked at the start of functions if
`self->buffer` is in a good state, but did not always recheck after
other Python code was called which could modify `self->buffer`.

The cases which need to be re-checked are hard to spot so rather than
rely on reviewer effort create better safety by making all self->buffer
access go through helper functions.

Thank you yihong0618 for the test, NEWS and initial implementation in
gh-143041.



(cherry picked from commit db4b194)

* gh-154523: Fix data-race in `TextIOWrapper.detach()` (#154565)

(cherry picked from commit 8440a6a)

* gh-157364: Fix use-after-free in io.TextIOWrapper during reentrant detach (#157370)


(cherry picked from commit e19dc47)

---------





(cherry picked from commit ea1312b)

Co-authored-by: Cody Maloney <[email protected]>
Co-authored-by: yihong0618 <[email protected]>
Co-authored-by: Brij Kapadia <[email protected]>
Co-authored-by: Shamil <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@cmaloney cmaloney cmaloney left review comments
@serhiy-storchaka serhiy-storchaka serhiy-storchaka left review comments

Uh oh!

There was an error while loading. Please reload this page.

Assignees

No one assigned

Labels

awaiting review stale Stale PR or inactive for long period of time.

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Uh oh!

There was an error while loading. Please reload this page.

4 participants

@yihong0618 @cmaloney @vstinner @serhiy-storchaka
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
You are visitor 01402503 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.