GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — GH-108362: Incremental GC implementation by markshannon · Pull Request #116206 · python/cpython

Conversation

@markshannon

@markshannon markshannon commented Mar 1, 2024 •
edited by github-actions Bot
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Member

Implements incremental cyclic GC.
Instead of traversing one generation on each collection, we traverse the young generation and the oldest part of the old generation. By traversing the old generation a chunk at a time, we keep pause times down a lot.

See faster-cpython/ideas#613 for the idea and algorithm.

  • Issue: Incremental cycle GC #108362

📚 Documentation preview 📚: https://cpython-previews--116206.org.readthedocs.build/

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
markshannon added 7 commits January 7, 2024 07:04
@markshannon
Incremental GC
4cde149
@markshannon
Merge branch 'main' into incremental-gc-3
41f7a12
@markshannon
Fix compiler warning
5361bd7
@markshannon
Fix gdb support
0951d03
@markshannon
Make cold exits immortal safely
03ea080
@markshannon
Merge young gen and increment before scanning.
49b47fa
@markshannon
Fix docs formatting
7febe8e
@bedevere-app bedevere-app Bot mentioned this pull request Mar 1, 2024
Incremental cycle GC #108362
Closed
@markshannon

markshannon commented Mar 4, 2024 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Member Author

The build failures appear to be caused by #114751, so I think this is ready for review.

I believe the reason that the earlier version of this PR needed reverting was that it relied on the young gen being empty after a young collection. However if a finalizer creates new objects, then that assumption is false.

This version merges the young gen and increment before collection, so we don't care if the young gen isn't empty afterwards.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@markshannon
Merge branch 'main' into incremental-gc-3
912ebc1
@markshannon markshannon added the 🔨 test-with-buildbots Test PR w/ buildbots; report in status section label Mar 5, 2024
@bedevere-bot

bedevere-bot commented Mar 5, 2024

Copy link
Copy Markdown

🤖 New build scheduled with the buildbot fleet by @markshannon for commit 912ebc1 🤖

If you want to schedule another build, you need to add the 🔨 test-with-buildbots label again.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@bedevere-bot bedevere-bot removed the 🔨 test-with-buildbots Test PR w/ buildbots; report in status section label Mar 5, 2024
@markshannon

markshannon commented Mar 6, 2024

Copy link
Copy Markdown
Member Author

!buildbot bigmem

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@bedevere-bot

bedevere-bot commented Mar 6, 2024

Copy link
Copy Markdown

🤖 New build scheduled with the buildbot fleet by @markshannon for commit 912ebc1 🤖

The command will test the builders whose names match following regular expression: bigmem

The builders matched are:

  • AMD64 Windows11 Bigmem PR
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@markshannon
Revert check for immortality.
3c1ef42
@markshannon markshannon added the 🔨 test-with-buildbots Test PR w/ buildbots; report in status section label Mar 7, 2024
@bedevere-bot

bedevere-bot commented Mar 7, 2024

Copy link
Copy Markdown

🤖 New build scheduled with the buildbot fleet by @markshannon for commit 3c1ef42 🤖

If you want to schedule another build, you need to add the 🔨 test-with-buildbots label again.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@bedevere-bot bedevere-bot removed the 🔨 test-with-buildbots Test PR w/ buildbots; report in status section label Mar 7, 2024
@markshannon

markshannon commented Mar 7, 2024

Copy link
Copy Markdown
Member Author

!buildbot bigmem

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@bedevere-bot

bedevere-bot commented Mar 7, 2024

Copy link
Copy Markdown

🤖 New build scheduled with the buildbot fleet by @markshannon for commit 3c1ef42 🤖

The command will test the builders whose names match following regular expression: bigmem

The builders matched are:

  • AMD64 Windows11 Bigmem PR
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@markshannon
markshannon marked this pull request as ready for review March 8, 2024 10:07
@markshannon
markshannon requested review from ericsnowcurrently, gvanrossum, kumaraditya303 and pablogsal as code owners March 8, 2024 10:07
@bedevere-app bedevere-app Bot added the awaiting core review label Mar 8, 2024
@markshannon
markshannon requested review from DinoV and removed request for kumaraditya303 March 11, 2024 09:28
@markshannon
Merge branch 'main' into incremental-gc-3
7a41258
@markshannon

markshannon commented Mar 19, 2024

Copy link
Copy Markdown
Member Author

!buildbot bigmem

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@bedevere-bot

bedevere-bot commented Mar 19, 2024

Copy link
Copy Markdown

🤖 New build scheduled with the buildbot fleet by @markshannon for commit f4f04d6 🤖

The command will test the builders whose names match following regular expression: bigmem

The builders matched are:

  • AMD64 Windows11 Bigmem PR
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

DinoV
DinoV approved these changes Mar 19, 2024
View reviewed changes

@DinoV DinoV left a comment

Copy link
Copy Markdown
Contributor

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM now!

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
@bedevere-app bedevere-app Bot added awaiting merge and removed awaiting core review labels Mar 19, 2024
@markshannon

markshannon commented Mar 20, 2024

Copy link
Copy Markdown
Member Author

Note on the test failures:

  • JIT / aarch64-unknown-linux-gnu/gcc (Debug) (pull_request) passes ~450 out of 460 tests then is timed out after an hour. I'm considering that a pass.
  • Tests / CIFuzz (memory) (pull_request) is just broken and fails on everything at the moment.
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@markshannon
markshannon merged commit 1530932 into python:main Mar 20, 2024
@bedevere-app bedevere-app Bot removed the awaiting merge label Mar 20, 2024
vstinner pushed a commit to vstinner/cpython that referenced this pull request Mar 20, 2024
@markshannon @vstinner
pythonGH-108362: Incremental Cycle GC (pythonGH-116206)
f4b2af9
@hauntsaninja hauntsaninja mentioned this pull request Mar 21, 2024
Major slowdown on large listcomp #117108
Closed
This was referenced Mar 21, 2024
pystats: #116206 (Incremental GC implementation) broke pystats build #117122
Closed
gh-117122: Fix pystats after incremental GC changes #117123
Closed
GH-117108: Change the size of the GC increment to about 1% of the total heap size. #117120
Merged
gh-116968: Reimplement Tier 2 counters #117144
Merged
@ericsnowcurrently

ericsnowcurrently commented Mar 22, 2024

Copy link
Copy Markdown
Member

This may have introduced a performance regression. See gh-117108.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@mdboom mdboom mentioned this pull request Mar 22, 2024
Investigation: Why do some benchmarks have significantly increased memory usage with the new incremental GC? faster-cpython/ideas#668
Open
gvanrossum added a commit to gvanrossum/cpython that referenced this pull request Mar 23, 2024
@gvanrossum
Revert "pythonGH-108362: Incremental Cycle GC (pythonGH-116206)"
e6480f7
Second half of the revert.

This reverts commit 1530932.
adorilson pushed a commit to adorilson/cpython that referenced this pull request Mar 25, 2024
@markshannon @adorilson
pythonGH-108362: Incremental Cycle GC (pythonGH-116206)
41b15e9
diegorusso pushed a commit to diegorusso/cpython that referenced this pull request Apr 17, 2024
@markshannon @diegorusso
pythonGH-108362: Incremental Cycle GC (pythonGH-116206)
56ad354
abhinav-upadhyay added a commit to abhinav-upadhyay/cpython that referenced this pull request May 16, 2024
@abhinav-upadhyay
Move the comment to the right place where it used to be
fa21e29
In pythonGH-116206, the comment about moving reachable objects to next generation got moved from its original place to a place where there is no code below it. This PR puts it back where the actual movement of rechable objects happen to the next generation.
@abhinav-upadhyay abhinav-upadhyay mentioned this pull request May 16, 2024
Move the comment to the right place where it used to be #119085
Merged
@user202729 user202729 mentioned this pull request Jul 26, 2024
gc.DEBUG_STATS no longer print out anything #122298
Closed
@markshannon
markshannon deleted the incremental-gc-3 branch August 6, 2024 10:18
Yhg1s added a commit to Yhg1s/cpython that referenced this pull request Sep 29, 2024
@Yhg1s
Revert "pythonGH-108362: Incremental Cycle GC (pythonGH-116206)"
4be9e85
This reverts commit 1530932.
Yhg1s added a commit to Yhg1s/cpython that referenced this pull request Sep 29, 2024
@Yhg1s
Revert "pythonGH-108362: Incremental Cycle GC (pythonGH-116206)"
1388aff
This reverts commit 1530932.
Yhg1s added a commit to Yhg1s/cpython that referenced this pull request Sep 29, 2024
@Yhg1s
Revert "pythonGH-108362: Incremental Cycle GC (pythonGH-116206)"
d15eace
This reverts commit 1530932.
@Yhg1s Yhg1s mentioned this pull request Sep 29, 2024
[3.13] GH-124567: Revert the Incremental GC in 3.13 #124770
Merged
nascheme pushed a commit that referenced this pull request Aug 7, 2025
@abhinav-upadhyay
GH-119085: Move comment in Python/gc.c to correct place.
e3ad900
In GH-116206, the comment about moving reachable objects to next generation
got moved from its original place to a place where there is no code below
it. Put the comment back to where the actual movement of reachable objects
happens.
Agent-Hellboy pushed a commit to Agent-Hellboy/cpython that referenced this pull request Aug 19, 2025
@abhinav-upadhyay @Agent-Hellboy
pythonGH-119085: Move comment in Python/gc.c to correct place.
7bde5d6
In pythonGH-116206, the comment about moving reachable objects to next generation
got moved from its original place to a place where there is no code below
it. Put the comment back to where the actual movement of reachable objects
happens.
@pgdr pgdr mentioned this pull request Oct 20, 2025
Bring back elapsed time debugging in gc.c #140358
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@gvanrossum gvanrossum gvanrossum left review comments
@DinoV DinoV DinoV approved these changes
@pablogsal pablogsal Awaiting requested review from pablogsal pablogsal is a code owner
@ericsnowcurrently ericsnowcurrently Awaiting requested review from ericsnowcurrently ericsnowcurrently is a code owner

Uh oh!

There was an error while loading. Please reload this page.

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Uh oh!

There was an error while loading. Please reload this page.

5 participants

@markshannon @bedevere-bot @DinoV @ericsnowcurrently @gvanrossum
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
You are visitor 01402503 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.