GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — GH-90699: fix ref counting of static immortal strings by kumaraditya303 · Pull Request #94850 · python/cpython

Conversation

@kumaraditya303

@kumaraditya303 kumaraditya303 commented Jul 14, 2022 •
edited by bedevere-bot
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Contributor

Without this they can cause negative refcount in _Py_RefTotal.

  • Issue: Replace _Py_IDENTIFIER() with statically initialized objects. #90699

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
@kumaraditya303
fix recounting of immortal objects
cf62862
@bedevere-bot bedevere-bot added the awaiting review label Jul 14, 2022
@blurb-it
📜🤖 Added by blurb_it.
f530250
@kumaraditya303 kumaraditya303 mentioned this pull request Jul 14, 2022
Check for ref counting bugs in debug mode caused by immortal objects. #94851
Closed
@kumaraditya303 kumaraditya303 self-assigned this Jul 14, 2022
@kumaraditya303 kumaraditya303 added the 🔨 test-with-buildbots Test PR w/ buildbots; report in status section label Jul 14, 2022
@bedevere-bot

bedevere-bot commented Jul 14, 2022

Copy link
Copy Markdown

🤖 New build scheduled with the buildbot fleet by @kumaraditya303 for commit f530250 🤖

If you want to schedule another build, you need to add the ":hammer: test-with-buildbots" label again.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@bedevere-bot bedevere-bot removed the 🔨 test-with-buildbots Test PR w/ buildbots; report in status section label Jul 14, 2022
@kumaraditya303 kumaraditya303 added the needs backport to 3.11 only security fixes label Jul 14, 2022
@kumaraditya303
kumaraditya303 requested a review from ericsnowcurrently July 14, 2022 10:32
@corona10

corona10 commented Jul 14, 2022 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Member

Thanks for the work, Would you like to provide a reproducible way for detecting leaks?

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@kumaraditya303

kumaraditya303 commented Jul 14, 2022

Copy link
Copy Markdown
Contributor Author

Thanks for the work, Would you like to provide a reproducible way for detecting leaks?

The following script causes negative ref count:

for i in range(1000):
    a = repr(False)

Output:

@kumaraditya303 ➜ /workspaces/cpython (main ✗) $ ./python -X showrefcount main.py 
[-1001 refs, 0 blocks]
corona10 reacted with thumbs up emoji
All reactions
  • 👍 1 reaction

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@kumaraditya303
kumaraditya303 requested a review from corona10 July 14, 2022 12:59
@kumaraditya303

kumaraditya303 commented Jul 14, 2022

Copy link
Copy Markdown
Contributor Author

@corona10: The following script causes negative refcount with TextIOWrapper:

# main.py
from io import TextIOWrapper

a = TextIOWrapper(open('main.py','rb'))
for i in a:
    pass
a.close()

Output:

@kumaraditya303 ➜ /workspaces/cpython (main ✗) $ ./python -X showrefcount main.py 
[-2 refs, 0 blocks]
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

corona10
corona10 approved these changes Jul 14, 2022
View reviewed changes

@corona10 corona10 left a comment

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

I reproduced the issue with the attached scripts and issues are solved by @kumaraditya303 's patch :)

Thanks for the work!

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
@bedevere-bot bedevere-bot added awaiting merge and removed awaiting review labels Jul 14, 2022
corona10
corona10 requested changes Jul 14, 2022
View reviewed changes

@corona10 corona10 left a comment •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@kumaraditya303

Sorry Kumar, Looks like leaks exist through this patch.
PTAL

With patch


beginning 6 repetitions
123456
......
test_io leaked [2, 2, 2] references, sum=6
test_io failed (reference leak) in 3 min 23 sec

== Tests result: FAILURE ==

1 test failed:
    test_io

Total duration: 3 min 23 sec
Tests result: FAILURE

Without Patch


Raised RLIMIT_NOFILE: 256 -> 1024
0:00:00 load avg: 2.90 Run tests sequentially
0:00:00 load avg: 2.90 [1/1] test_io
beginning 6 repetitions
123456
......
test_io passed in 3 min 23 sec

== Tests result: SUCCESS ==

1 test OK.

Total duration: 3 min 23 sec
Tests result: SUCCESS

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
@bedevere-bot bedevere-bot removed the awaiting merge label Jul 14, 2022
@bedevere-bot

bedevere-bot commented Jul 14, 2022

Copy link
Copy Markdown

A Python core developer has requested some changes be made to your pull request before we can consider merging it. If you could please address their requests along with any other requests in other reviews from core developers that would be appreciated.

Once you have made the requested changes, please leave a comment on this pull request containing the phrase I have made the requested changes; please review again. I will then notify any core developers who have left a review that you're ready for them to take another look at this pull request.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@bedevere-bot bedevere-bot added the awaiting changes label Jul 14, 2022
@kumaraditya303

kumaraditya303 commented Jul 14, 2022

Copy link
Copy Markdown
Contributor Author

Looking, I wonder why only two bots failed if there are refleaks.

corona10 reacted with confused emoji
All reactions
  • 😕 1 reaction

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@kumaraditya303

kumaraditya303 commented Jul 14, 2022

Copy link
Copy Markdown
Contributor Author

It is also leaking at my local environment :(

Yeah, I am also able to reproduce, bisecting which one is causing leak

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@kumaraditya303

kumaraditya303 commented Jul 14, 2022

Copy link
Copy Markdown
Contributor Author

I am getting refleak on main branch too, test.test_io.CTextIOWrapperTest.test_reconfigure_locale is failing.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@kumaraditya303

kumaraditya303 commented Jul 15, 2022 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Contributor Author

Refleak was fixed by #94858

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@kumaraditya303
Merge branch 'main' of https://github.com/python/cpython into cached-…
0a76abd
…strs
@kumaraditya303
kumaraditya303 requested a review from corona10 July 18, 2022 15:16
@kumaraditya303 kumaraditya303 added the 🔨 test-with-buildbots Test PR w/ buildbots; report in status section label Jul 18, 2022
@bedevere-bot

bedevere-bot commented Jul 18, 2022

Copy link
Copy Markdown

🤖 New build scheduled with the buildbot fleet by @kumaraditya303 for commit 0a76abd 🤖

If you want to schedule another build, you need to add the ":hammer: test-with-buildbots" label again.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@bedevere-bot bedevere-bot removed the 🔨 test-with-buildbots Test PR w/ buildbots; report in status section label Jul 18, 2022
@kumaraditya303

kumaraditya303 commented Jul 19, 2022

Copy link
Copy Markdown
Contributor Author

Refleak buildbots are failing on main too, see #94979

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@kumaraditya303
Merge branch 'main' into cached-strs
f2f0327
corona10
corona10 approved these changes Jul 20, 2022
View reviewed changes

@corona10 corona10 left a comment

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM
Thank you, Kumar!

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
@bedevere-bot bedevere-bot added awaiting merge and removed awaiting changes labels Jul 20, 2022
@corona10
corona10 merged commit 1834133 into python:main Jul 20, 2022
@miss-islington

miss-islington commented Jul 20, 2022

Copy link
Copy Markdown
Contributor

Thanks @kumaraditya303 for the PR, and @corona10 for merging it 🌮🎉.. I'm working now to backport this PR to: 3.11.
🐍🍒⛏🤖

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@bedevere-bot bedevere-bot removed the awaiting merge label Jul 20, 2022
miss-islington pushed a commit to miss-islington/cpython that referenced this pull request Jul 20, 2022
@kumaraditya303 @miss-islington
pythonGH-90699: fix ref counting of static immortal strings (pythongh…
6d0cb0d
…-94850)

(cherry picked from commit 1834133)

Co-authored-by: Kumar Aditya <[email protected]>
@bedevere-bot

bedevere-bot commented Jul 20, 2022

Copy link
Copy Markdown

GH-95037 is a backport of this pull request to the 3.11 branch.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@bedevere-bot bedevere-bot removed the needs backport to 3.11 only security fixes label Jul 20, 2022
@kumaraditya303
kumaraditya303 deleted the cached-strs branch July 20, 2022 06:28
@kumaraditya303

kumaraditya303 commented Jul 20, 2022

Copy link
Copy Markdown
Contributor Author

Thanks for the review @corona10!

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

miss-islington added a commit that referenced this pull request Jul 20, 2022
@miss-islington @kumaraditya303
GH-90699: fix ref counting of static immortal strings (gh-94850)
84d58ad
(cherry picked from commit 1834133)

Co-authored-by: Kumar Aditya <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@corona10 corona10 corona10 approved these changes
@ericsnowcurrently ericsnowcurrently Awaiting requested review from ericsnowcurrently

Uh oh!

There was an error while loading. Please reload this page.

Assignees

@kumaraditya303 kumaraditya303

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Uh oh!

There was an error while loading. Please reload this page.

4 participants

@kumaraditya303 @bedevere-bot @corona10 @miss-islington
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
You are visitor 01402503 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.