GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — gh-140798: Fix memory leak with `threading.local` when finalizer resurrects `threading_local_key` by efimov-mikhail · Pull Request #140836 · python/cpython

Conversation

@efimov-mikhail

@efimov-mikhail efimov-mikhail commented Oct 31, 2025 •
edited by bedevere-app Bot
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Member
  • Issue: Memory leak with threading.local when tracing is active #140798

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
@efimov-mikhail
Memory leak fix
b29c5bd
@efimov-mikhail
efimov-mikhail requested review from ZeroIntensity and ericsnowcurrently as code owners October 31, 2025 14:34
@bedevere-app bedevere-app Bot mentioned this pull request Oct 31, 2025
Memory leak with threading.local when tracing is active #140798
Open
@bedevere-app bedevere-app Bot added the awaiting review label Oct 31, 2025
@efimov-mikhail
efimov-mikhail requested review from vstinner and removed request for ZeroIntensity and ericsnowcurrently October 31, 2025 14:35
efimov-mikhail added 2 commits October 31, 2025 17:40
@efimov-mikhail
Test added
0843821
@efimov-mikhail
Typo
4382459
@efimov-mikhail
efimov-mikhail marked this pull request as draft October 31, 2025 14:46
@bedevere-app bedevere-app Bot removed the awaiting review label Oct 31, 2025
efimov-mikhail added 2 commits October 31, 2025 17:50
@efimov-mikhail
Add NEWS
491898d
@efimov-mikhail
Correct cleaning threading trace function in test
685c050
@efimov-mikhail efimov-mikhail added needs backport to 3.13 only security fixes needs backport to 3.14 bugs and security fixes labels Oct 31, 2025
@efimov-mikhail
efimov-mikhail marked this pull request as ready for review October 31, 2025 15:27
@bedevere-app bedevere-app Bot added the awaiting review label Oct 31, 2025
efimov-mikhail added 2 commits October 31, 2025 18:40
@efimov-mikhail
Test readability improved
f7e6162
@efimov-mikhail
Merge branch 'main' into issue-140798-thread-local-clear
9a9957f
sergey-miryanov
sergey-miryanov previously approved these changes Nov 1, 2025
View reviewed changes

@sergey-miryanov sergey-miryanov left a comment

Copy link
Copy Markdown
Contributor

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tracer handles Python calls. If a thread's locals contain an object with a Python callback (e.g., a __del__ finalizer), then tracer will handle the call.

When our thread is finalizing, the locals are cleared, as are all objects within them. In this case, the finalizer of our object's __del__ will be called and handled by the tracer. This leads to the recreation of thread locals (threading_local_key and threading_local_sentinel), which leak because we mistakenly believe that the locals were cleared and will not be recreated.

In this PR, we clear the locals after clearing the tracers, so any callbacks from finalizers will not be handled, and the locals cannot be recreated.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
@bedevere-app bedevere-app Bot added awaiting core review and removed awaiting review labels Nov 1, 2025
@efimov-mikhail

efimov-mikhail commented Nov 1, 2025 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Member Author

Thanks for the review, Sergey!

But I have to say that I've discovered one even more difficult case without tracing at all:

import _thread
import threading

local = _thread._local()

class ClassWithDel:
    def __del__(self):
        getattr(local, 'c', None)

def thread_func():
    local.c = ClassWithDel()

t = threading.Thread(target=thread_func)
t.start()
t.join()

There's a leak with ASAN since finalizer of local.c object recreates localsdict for this thread.

CC @mpage @colesbury @ambv as author and reviewers of #121655.

Btw, we can merge this as is and create another issue for this case if this will be clearer.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@colesbury

colesbury commented Nov 1, 2025

Copy link
Copy Markdown
Contributor

This does not look like the right approach to me. I think you will end up playing whac-a-mole where each time you "fix" one leak you will introduce other bugs. You need a robust approach to dealing with finalizers like the GC has.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@efimov-mikhail

efimov-mikhail commented Nov 1, 2025 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Member Author

Thanks for the comment, @colesbury .
I'd like to stress that this PR doesn't introduce new bugs, it just doesn't fix all related issues in the current state. There's a leak with this code snippet on main.

And I'll be grateful for some piece of advice, if possible.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@colesbury

colesbury commented Nov 1, 2025

Copy link
Copy Markdown
Contributor

I think you just haven't yet found the new bugs that this PR introduces. The original bug report was only found via fuzzing. I suspect that if you keep fuzzing you will find different ones with this PR.

The problems is that we have code in the form:

Py_CLEAR(tstate->var1);
Py_CLEAR(tstate->var2);
...
Py_CLEAR(tstate->varN);

The underlying problem is that when you clear tstate->var2 you may execute Python code that re-initializes tstate->var2 or variables cleared earlier like tstate->var1. If you move around the order of clearing, you change the possible bugs.

For example, if you move tstate->threading_local_key to the end, you now have potential leaks where thread local variable destructors set things like tstate->asyncio_running_loop

sergey-miryanov and efimov-mikhail reacted with thumbs up emoji
All reactions
  • 👍 2 reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@efimov-mikhail

efimov-mikhail commented Nov 1, 2025

Copy link
Copy Markdown
Member Author

In general, I agree with you, swapping clear function calls isn't good way of fixing anything.
I have some new thoughts on this topic. I'll continue to find better solution.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@efimov-mikhail
efimov-mikhail marked this pull request as draft November 1, 2025 15:18
@bedevere-app bedevere-app Bot removed the awaiting core review label Nov 1, 2025
@sergey-miryanov
sergey-miryanov dismissed their stale review November 1, 2025 18:03

It seems that we should find new solution.

@bedevere-app bedevere-app Bot added the awaiting review label Nov 1, 2025
@ZeroIntensity ZeroIntensity mentioned this pull request Nov 1, 2025
gh-140860: Clear audit hooks before destroying modules #140861
Closed
efimov-mikhail added 4 commits November 2, 2025 20:41
@efimov-mikhail
Revert pystate.c changes, comment fix
cdae2c7
@efimov-mikhail
Another fix and test, NEWS updated
6955309
@efimov-mikhail
Test improved
c4bc099
@efimov-mikhail
Remove two asserts
50bc0f6
@efimov-mikhail
efimov-mikhail requested a review from sergey-miryanov November 2, 2025 20:35
@efimov-mikhail efimov-mikhail changed the title gh-140798: Fix memory leak with threading.local when tracing is active gh-140798: Fix memory leak with threading.local when finalizer resurrects thread_local_key Nov 2, 2025
@efimov-mikhail efimov-mikhail changed the title gh-140798: Fix memory leak with threading.local when finalizer resurrects thread_local_key gh-140798: Fix memory leak with threading.local when finalizer resurrects threading_local_key Nov 2, 2025
efimov-mikhail added 2 commits November 3, 2025 00:23
@efimov-mikhail
NEWS fix
dfa5f30
@efimov-mikhail
Remove one blank line
a7d3147
@efimov-mikhail
efimov-mikhail marked this pull request as ready for review November 2, 2025 21:49
@efimov-mikhail
efimov-mikhail requested review from colesbury and mpage November 2, 2025 21:50
@efimov-mikhail

efimov-mikhail commented Nov 15, 2025

Copy link
Copy Markdown
Member Author

Friendly ping, @colesbury @vstinner @mpage

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@github-actions

github-actions Bot commented May 1, 2026

Copy link
Copy Markdown

This PR is stale because it has been open for 30 days with no activity.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@github-actions github-actions Bot added the stale Stale PR or inactive for long period of time. label May 1, 2026
@serhiy-storchaka serhiy-storchaka added the needs backport to 3.15 bugs and security fixes label May 30, 2026
@github-actions github-actions Bot removed the stale Stale PR or inactive for long period of time. label Jun 1, 2026
@github-actions

github-actions Bot commented Sep 4, 2026

Copy link
Copy Markdown

This PR is stale because it has been open for 90 days with no activity.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@github-actions github-actions Bot added the stale Stale PR or inactive for long period of time. label Sep 4, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@vstinner vstinner Awaiting requested review from vstinner
@sergey-miryanov sergey-miryanov Awaiting requested review from sergey-miryanov
@colesbury colesbury Awaiting requested review from colesbury
@mpage mpage Awaiting requested review from mpage

Uh oh!

There was an error while loading. Please reload this page.

Assignees

No one assigned

Labels

awaiting review needs backport to 3.13 only security fixes needs backport to 3.14 bugs and security fixes needs backport to 3.15 bugs and security fixes stale Stale PR or inactive for long period of time.

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Uh oh!

There was an error while loading. Please reload this page.

4 participants

@efimov-mikhail @colesbury @sergey-miryanov @serhiy-storchaka
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
You are visitor 01402520 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.