GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — gh-145615: Fix mimalloc page leak in the free-threaded build by colesbury · Pull Request #145626 · python/cpython

Conversation

@colesbury

@colesbury colesbury commented Mar 7, 2026 •
edited by bedevere-app Bot
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Contributor

Fix three issues that caused mimalloc pages to be leaked until the owning thread exited:

  1. In _PyMem_mi_page_maybe_free(), move pages out of the full queue when relying on QSBR to defer freeing the page. Pages in the full queue are never searched by mi_page_queue_find_free_ex(), so a page left there is unusable for allocations.

  2. Move _PyMem_mi_page_clear_qsbr() from _mi_page_free_collect() to _mi_page_thread_free_collect() where it only fires when all blocks on the page are free (used == 0). The previous placement was too broad: it cleared QSBR state whenever local_free was non-NULL, but _mi_page_free_collect() is called from non-allocation paths (e.g., page visiting in mi_heap_visit_blocks) where the page is not being reused.

  3. In _PyMem_mi_page_maybe_free(), use the page's heap tld to find the correct thread state for QSBR list insertion instead of PyThreadState_GET(). During stop-the-world pauses, the function may process pages belonging to other threads, so the current thread state is not necessarily the owner of the page.

  • Issue: Free threading memory leak #145615

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
@colesbury
pythongh-145615: Fix mimalloc page leak in the free-threaded build
218b6bf
Fix three issues that caused mimalloc pages to be leaked until the
owning thread exited:

1. In _PyMem_mi_page_maybe_free(), move pages out of the full queue
   when relying on QSBR to defer freeing the page. Pages in the full
   queue are never searched by mi_page_queue_find_free_ex(), so a page
   left there is unusable for allocations.

2. Move _PyMem_mi_page_clear_qsbr() from _mi_page_free_collect() to
   _mi_page_thread_free_collect() where it only fires when all blocks
   on the page are free (used == 0). The previous placement was too
   broad: it cleared QSBR state whenever local_free was non-NULL, but
   _mi_page_free_collect() is called from non-allocation paths (e.g.,
   page visiting in mi_heap_visit_blocks) where the page is not being
   reused.

3. In _PyMem_mi_page_maybe_free(), use the page's heap tld to find the
   correct thread state for QSBR list insertion instead of
   PyThreadState_GET(). During stop-the-world pauses, the function may
   process pages belonging to other threads, so the current thread
   state is not necessarily the owner of the page.
@colesbury colesbury added topic-free-threading needs backport to 3.14 bugs and security fixes labels Mar 7, 2026
@bedevere-app bedevere-app Bot mentioned this pull request Mar 7, 2026
Free threading memory leak #145615
Closed
@colesbury colesbury added the 🔨 test-with-buildbots Test PR w/ buildbots; report in status section label Mar 7, 2026
@bedevere-bot

This comment was marked as outdated.

Sign in to view
@bedevere-bot bedevere-bot removed the 🔨 test-with-buildbots Test PR w/ buildbots; report in status section label Mar 7, 2026
colesbury added 3 commits March 7, 2026 16:58
@colesbury
Fix page queue mismatch in mi_heap_page_collect
4d059c9
@colesbury
Extract tstate_from_heap helper in obmalloc.c
ebd5980
@colesbury
Fix build error in default build.
2516a1a
@colesbury colesbury added the 🔨 test-with-buildbots Test PR w/ buildbots; report in status section label Mar 7, 2026
@bedevere-bot

bedevere-bot commented Mar 7, 2026

Copy link
Copy Markdown

🤖 New build scheduled with the buildbot fleet by @colesbury for commit 2516a1a 🤖

Results will be shown at:

https://buildbot.python.org/all/#/grid?branch=refs%2Fpull%2F145626%2Fmerge

If you want to schedule another build, you need to add the 🔨 test-with-buildbots label again.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@bedevere-bot bedevere-bot removed the 🔨 test-with-buildbots Test PR w/ buildbots; report in status section label Mar 7, 2026
@colesbury
colesbury marked this pull request as ready for review March 9, 2026 14:06
@bedevere-app bedevere-app Bot added the awaiting core review label Mar 9, 2026
@colesbury
colesbury requested a review from Yhg1s March 9, 2026 14:06
@colesbury colesbury assigned mpage and unassigned mpage Mar 9, 2026
@colesbury
colesbury requested a review from mpage March 9, 2026 14:06
@mpage

mpage commented Mar 9, 2026 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Contributor

The changes look good to me, though I am not intimately familiar with mimalloc or our modifications to it. The buildbot failure seems like it might be real though. It looks like test.test_tools.test_freeze.TestFreeze.test_freeze_simple_script is consistently timing out on this PR. None of the recent runs on the buildbot failed in the same way.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@colesbury

colesbury commented Mar 9, 2026 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Contributor Author

I don't think the buildbot failure is related. The PR only touches mimalloc and Py_GIL_DISABLED code, which isn't used by that buildbot configuration. There are fairly frequent time outs at random places with that buildbot: https://buildbot.python.org/#/workers/4. The test_freeze_simple_script test is pretty slow and isn't run in GitHub CI.

mpage reacted with thumbs up emoji
All reactions
  • 👍 1 reaction

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

mpage
mpage approved these changes Mar 9, 2026
View reviewed changes
@bedevere-app bedevere-app Bot added awaiting merge and removed awaiting core review labels Mar 9, 2026
@colesbury
colesbury merged commit d76df75 into python:main Mar 9, 2026
130 of 131 checks passed

Uh oh!

There was an error while loading. Please reload this page.

@bedevere-app bedevere-app Bot removed the awaiting merge label Mar 9, 2026
@miss-islington-app

miss-islington-app Bot commented Mar 9, 2026

Copy link
Copy Markdown

Thanks @colesbury for the PR 🌮🎉.. I'm working now to backport this PR to: 3.14.
🐍🍒⛏🤖

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@colesbury
colesbury deleted the gh-145615-fix-mimalloc-page-leak branch March 9, 2026 17:24
miss-islington pushed a commit to miss-islington/cpython that referenced this pull request Mar 9, 2026
@colesbury @miss-islington
pythongh-145615: Fix mimalloc page leak in the free-threaded build (p…
2a3a642
…ythongh-145626)

Fix three issues that caused mimalloc pages to be leaked until the
owning thread exited:

1. In _PyMem_mi_page_maybe_free(), move pages out of the full queue
   when relying on QSBR to defer freeing the page. Pages in the full
   queue are never searched by mi_page_queue_find_free_ex(), so a page
   left there is unusable for allocations.

2. Move _PyMem_mi_page_clear_qsbr() from _mi_page_free_collect() to
   _mi_page_thread_free_collect() where it only fires when all blocks
   on the page are free (used == 0). The previous placement was too
   broad: it cleared QSBR state whenever local_free was non-NULL, but
   _mi_page_free_collect() is called from non-allocation paths (e.g.,
   page visiting in mi_heap_visit_blocks) where the page is not being
   reused.

3. In _PyMem_mi_page_maybe_free(), use the page's heap tld to find the
   correct thread state for QSBR list insertion instead of
   PyThreadState_GET(). During stop-the-world pauses, the function may
   process pages belonging to other threads, so the current thread
   state is not necessarily the owner of the page.
(cherry picked from commit d76df75)

Co-authored-by: Sam Gross <[email protected]>
@bedevere-app

bedevere-app Bot commented Mar 9, 2026

Copy link
Copy Markdown

GH-145691 is a backport of this pull request to the 3.14 branch.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@bedevere-app bedevere-app Bot removed the needs backport to 3.14 bugs and security fixes label Mar 9, 2026
colesbury added a commit that referenced this pull request Mar 9, 2026
@miss-islington @colesbury
[3.14] gh-145615: Fix mimalloc page leak in the free-threaded build (g…
ba1ea3a
…h-145626) (#145691)

Fix three issues that caused mimalloc pages to be leaked until the
owning thread exited:

1. In _PyMem_mi_page_maybe_free(), move pages out of the full queue
   when relying on QSBR to defer freeing the page. Pages in the full
   queue are never searched by mi_page_queue_find_free_ex(), so a page
   left there is unusable for allocations.

2. Move _PyMem_mi_page_clear_qsbr() from _mi_page_free_collect() to
   _mi_page_thread_free_collect() where it only fires when all blocks
   on the page are free (used == 0). The previous placement was too
   broad: it cleared QSBR state whenever local_free was non-NULL, but
   _mi_page_free_collect() is called from non-allocation paths (e.g.,
   page visiting in mi_heap_visit_blocks) where the page is not being
   reused.

3. In _PyMem_mi_page_maybe_free(), use the page's heap tld to find the
   correct thread state for QSBR list insertion instead of
   PyThreadState_GET(). During stop-the-world pauses, the function may
   process pages belonging to other threads, so the current thread
   state is not necessarily the owner of the page.

(cherry picked from commit d76df75)

Co-authored-by: Sam Gross <[email protected]>
@Yhg1s

Yhg1s commented Mar 10, 2026

Copy link
Copy Markdown
Member

(Late to the party but...) should the changes to mimalloc source be upstreamed?

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@colesbury

colesbury commented Mar 10, 2026

Copy link
Copy Markdown
Contributor Author

No, the PR is modifications to our mimalloc modifications / obmalloc.c code. The bugs were in code I wrote, not in upstream mimalloc.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@colesbury

colesbury commented Mar 10, 2026

Copy link
Copy Markdown
Contributor Author

I'll put a note on the issue to make that more clear.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

ljfp pushed a commit to ljfp/cpython that referenced this pull request Apr 25, 2026
@colesbury @ljfp
pythongh-145615: Fix mimalloc page leak in the free-threaded build (p…
581c0ab
…ythongh-145626)

Fix three issues that caused mimalloc pages to be leaked until the
owning thread exited:

1. In _PyMem_mi_page_maybe_free(), move pages out of the full queue
   when relying on QSBR to defer freeing the page. Pages in the full
   queue are never searched by mi_page_queue_find_free_ex(), so a page
   left there is unusable for allocations.

2. Move _PyMem_mi_page_clear_qsbr() from _mi_page_free_collect() to
   _mi_page_thread_free_collect() where it only fires when all blocks
   on the page are free (used == 0). The previous placement was too
   broad: it cleared QSBR state whenever local_free was non-NULL, but
   _mi_page_free_collect() is called from non-allocation paths (e.g.,
   page visiting in mi_heap_visit_blocks) where the page is not being
   reused.

3. In _PyMem_mi_page_maybe_free(), use the page's heap tld to find the
   correct thread state for QSBR list insertion instead of
   PyThreadState_GET(). During stop-the-world pauses, the function may
   process pages belonging to other threads, so the current thread
   state is not necessarily the owner of the page.
@weixlu weixlu mentioned this pull request Jul 6, 2026
Assertion triggered in MiMalloc (free-threaded build) #153176
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@mpage mpage mpage approved these changes
@Yhg1s Yhg1s Awaiting requested review from Yhg1s

Uh oh!

There was an error while loading. Please reload this page.

Assignees

No one assigned

Labels

topic-free-threading

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Uh oh!

There was an error while loading. Please reload this page.

4 participants

@colesbury @bedevere-bot @mpage @Yhg1s
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
You are visitor 01402503 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.