GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — gh-118331: Handle errors in _PyObject_SetManagedDict by colesbury · Pull Request #118334 · python/cpython

Conversation

@colesbury

@colesbury colesbury commented Apr 26, 2024 •
edited by bedevere-app Bot
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Contributor

When detaching a dict, the copy_values call may fail due to out-of-memory errors. This can be triggered by test_no_memory in test_repl.

  • Issue: test_repl's test_no_memory: AssertionError: -6 not found in (1, 120) #118331

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
All reactions
@colesbury
pythongh-118331: Handle errors in _PyObject_SetManagedDict
e8451c3 
When detaching a dict, the copy_values call may fail due to
out-of-memory errors. This can be triggered by test_no_memory in
test_repl.
@colesbury colesbury added the skip news label Apr 26, 2024
@colesbury
colesbury requested review from DinoV and hugovk April 26, 2024 20:45
@colesbury
colesbury requested review from markshannon and methane as code owners April 26, 2024 20:45
@bedevere-app bedevere-app Bot mentioned this pull request Apr 26, 2024
test_repl's test_no_memory: AssertionError: -6 not found in (1, 120) #118331
Closed
@bedevere-app bedevere-app Bot added the awaiting core review label Apr 26, 2024
@hugovk

hugovk commented Apr 26, 2024

Copy link
Copy Markdown
Member

Running this on this branch:

make clean
git clean -fdx .
./configure --with-pydebug && make -j10
./python.exe Lib/test/test_repl.py

I still get the error:

.....F.
======================================================================
FAIL: test_no_memory (__main__.TestInteractiveInterpreter.test_no_memory)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/private/tmp/cpython/Lib/test/test_repl.py", line 86, in test_no_memory
    self.assertIn(p.returncode, (1, 120))
    ~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^
AssertionError: -6 not found in (1, 120)

----------------------------------------------------------------------
Ran 7 tests in 0.331s

FAILED (failures=1)
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
@colesbury

colesbury commented Apr 26, 2024

Copy link
Copy Markdown
Contributor Author

Hmmm... maybe there are multiple unhandled exceptions?

I haven't yet been able to reproduce additional failures with this PR.
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
@mpage

mpage commented Apr 26, 2024 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Contributor

My experience is the same as @colesbury:

  • I can repro the issue on gh-117225: Move colorize functionality to own internal module #118283 but not on main.
  • Merging this PR into gh-117225: Move colorize functionality to own internal module #118283 fixes the issue and I can no longer reproduce it.

I tested on both fedora and MacOS.
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
@colesbury colesbury mentioned this pull request Apr 26, 2024
Merge of #118283 and #118334 for CI #118336
Closed
@hugovk

hugovk commented Apr 26, 2024

Copy link
Copy Markdown
Member

It's really strange, I can repro on main with two different Macs: my work one (which is at work until Monday) and this one.

I can also repro with this branch and #118336.
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
@hugovk

hugovk commented Apr 27, 2024

Copy link
Copy Markdown
Member

gh-118331 fixes it for me, and also with that merged into this. Thanks!
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
DinoV
DinoV approved these changes Apr 29, 2024
View reviewed changes

@DinoV DinoV left a comment

Copy link
Copy Markdown
Contributor

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
All reactions
Comment thread Objects/dictobject.c

err = _PyDict_DetachFromObject(dict, obj);
if (err == 0) {
FT_ATOMIC_STORE_PTR(_PyObject_ManagedDictPointer(obj)->dict,

@DinoV DinoV Apr 29, 2024

Copy link
Copy Markdown
Contributor

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we're no longer assigning this before calling _PyDict_DetachFromObject we can bring back some of the assertions in _PyDict_DetachFromObject, specifically:

    assert(_PyObject_ManagedDictPointer(obj)->dict == mp);
    assert(_PyObject_InlineValuesConsistencyCheck(obj));

I think assigning after should be fine after all, because the object and the dict are locked, so we'd only see reads which are proceeding against the values which are being copied. Once the values are invalidated we'd go to the dict which is being detached, and then we'd finally start seeing the new dict.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
colesbury reacted with thumbs up emoji
All reactions
  • 👍 1 reaction
@bedevere-app bedevere-app Bot added awaiting merge and removed awaiting core review labels Apr 29, 2024
@colesbury
Add assertions
8cf397e
@colesbury
colesbury merged commit 79688b5 into python:main Apr 29, 2024
@colesbury
colesbury deleted the gh-118331-unraisable branch April 29, 2024 19:49
@bedevere-app bedevere-app Bot removed the awaiting merge label Apr 29, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@DinoV DinoV DinoV approved these changes
@hugovk hugovk Awaiting requested review from hugovk
@markshannon markshannon Awaiting requested review from markshannon markshannon is a code owner
@methane methane Awaiting requested review from methane methane is a code owner

Uh oh!

There was an error while loading. Please reload this page.

Assignees

No one assigned

Labels

skip news

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Uh oh!

There was an error while loading. Please reload this page.

4 participants

@colesbury @hugovk @mpage @DinoV
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
You are visitor 01402503 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.