GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — gh-129107: make bytearrayiter free-threading safe by tom-pytel · Pull Request #130096 · python/cpython

Conversation

@tom-pytel

@tom-pytel tom-pytel commented Feb 13, 2025 •
edited by bedevere-app Bot
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Contributor

This is the bytearray iterator made safe under free-threading. The behavior is copied from the list iterator, it is not synchronized but it won't crash.

  • Issue: bytearray is not free-thread safe #129107

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
@tom-pytel
pythongh-129107: make bytearrayiter free-threading safe
cac384d
@bedevere-app bedevere-app Bot added the awaiting review label Feb 13, 2025
@bedevere-app bedevere-app Bot mentioned this pull request Feb 13, 2025
bytearray is not free-thread safe #129107
Closed
@blurb-it
📜🤖 Added by blurb_it.
4626f8f
@tom-pytel

tom-pytel commented Feb 13, 2025

Copy link
Copy Markdown
Contributor Author

This PR is split off of #129108. Ping @kumaraditya303

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@tom-pytel

tom-pytel commented Feb 13, 2025 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Contributor Author

EDIT: Removed, unnecessary critical sections in bytearrayiter_next and bytearrayiter_reduce. Really no point having them in this PR, will add them back in bytearray PR after this one is merged.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

tom-pytel added 3 commits February 13, 2025 18:23
@tom-pytel
remove unnecessary critical sections
518d5c5
@tom-pytel
misc cleanup
8a1edd8
@tom-pytel
atomic PyByteArray_GET_SIZE
1069a64
@kumaraditya303
kumaraditya303 self-requested a review February 14, 2025 16:41
tom-pytel added 2 commits February 14, 2025 18:35
@tom-pytel
use support.Py_GIL_DISABLED
185e2f1
@tom-pytel
Merge branch 'main' into fix-issue-129107-iterator
cfa80bb
@tom-pytel

tom-pytel commented Feb 15, 2025

Copy link
Copy Markdown
Contributor Author

@kumaraditya303 I thought you would do the iterator first and backport it so removed an unnecessary critical section here if bytearray not safe itself. If you don't intend to backport this then I will just add it back here, but otherwise it doesn't make sense (though harmless) unless you backport bytearray as well (which I am not suggesting). For reference, the critical section is:

static PyObject *
bytearrayiter_next(PyObject *self)
{
    bytesiterobject *it = _bytesiterobject_CAST(self);
    int val;

    assert(it != NULL);
    Py_ssize_t index = FT_ATOMIC_LOAD_SSIZE_RELAXED(it->it_index);
    if (index < 0) {
        return NULL;
    }
    PyByteArrayObject *seq = it->it_seq;
    assert(PyByteArray_Check(seq));

    Py_BEGIN_CRITICAL_SECTION(seq);
    if (index < PyByteArray_GET_SIZE(seq)) {
        val = (unsigned char)PyByteArray_AS_STRING(seq)[index];
    }
    else {
        val = -1;
    }
    Py_END_CRITICAL_SECTION();

    if (val == -1) {
        FT_ATOMIC_STORE_SSIZE_RELAXED(it->it_index, -1);
#ifndef Py_GIL_DISABLED
        it->it_seq = NULL;
        Py_DECREF(seq);
#endif
        return NULL;
    }
    FT_ATOMIC_STORE_SSIZE_RELAXED(it->it_index, index + 1);
    return _PyLong_FromUnsignedChar((unsigned char)val);
}

Though can also just use bytearray_getitem here, but I don't like calling an unnecessary function where can just read a byte instead.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@kumaraditya303

kumaraditya303 commented Feb 15, 2025 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Contributor

I don't intend to backport the thread safety of bytearray as change is large and there isn't much point in backporting the iterator while bytearray isn't thread safe.

@colesbury Do you think we should backport the thread safety of bytearray?

Yes, it is not safe to read data without acquiring critical section of the seq, you need to add back the critical section, the change to use atomic makes sense.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

kumaraditya303
kumaraditya303 reviewed Feb 15, 2025
View reviewed changes
Comment thread Objects/bytearrayobject.c Resolved

Uh oh!

There was an error while loading. Please reload this page.

@kumaraditya303
kumaraditya303 requested a review from colesbury February 15, 2025 13:04
kumaraditya303
kumaraditya303 reviewed Feb 15, 2025
View reviewed changes
Comment thread Lib/test/test_bytes.py Resolved

Uh oh!

There was an error while loading. Please reload this page.

@tom-pytel
restore bytearrayiter_next critical section
6e15abd
@colesbury

colesbury commented Feb 15, 2025

Copy link
Copy Markdown
Contributor

I probably wouldn't backport the changes.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

kumaraditya303
kumaraditya303 reviewed Feb 16, 2025
View reviewed changes
Comment thread Objects/bytearrayobject.c Resolved

Uh oh!

There was an error while loading. Please reload this page.

kumaraditya303
kumaraditya303 reviewed Feb 18, 2025
View reviewed changes
Comment thread Misc/NEWS.d/next/Core_and_Builtins/2025-02-13-20-42-53.gh-issue-129107._olg-L.rst Resolved

Uh oh!

There was an error while loading. Please reload this page.

@tom-pytel @kumaraditya303
Update Misc/NEWS.d/next/Core_and_Builtins/2025-02-13-20-42-53.gh-issu…
82ee657
…e-129107._olg-L.rst

Co-authored-by: Kumar Aditya <[email protected]>
kumaraditya303
kumaraditya303 reviewed Feb 18, 2025
View reviewed changes
Comment thread Objects/bytearrayobject.c Resolved

Uh oh!

There was an error while loading. Please reload this page.

@tom-pytel
requested changes
db66626
kumaraditya303
kumaraditya303 reviewed Feb 18, 2025
View reviewed changes
Comment thread Objects/bytearrayobject.c Resolved

Uh oh!

There was an error while loading. Please reload this page.

kumaraditya303
kumaraditya303 reviewed Feb 18, 2025
View reviewed changes
Comment thread Objects/bytearrayobject.c Resolved

Uh oh!

There was an error while loading. Please reload this page.

@tom-pytel
requested changes
b41ca85
@kumaraditya303 kumaraditya303 mentioned this pull request Feb 18, 2025
list iterator and list reverse iterator are not thread safe #130263
Closed
kumaraditya303
kumaraditya303 reviewed Feb 18, 2025
View reviewed changes
Comment thread Lib/test/test_bytes.py Resolved

Uh oh!

There was an error while loading. Please reload this page.

@tom-pytel
test iter_reduce and iter_setstate
095e20e
kumaraditya303
kumaraditya303 approved these changes Feb 18, 2025
View reviewed changes

@kumaraditya303 kumaraditya303 left a comment

Copy link
Copy Markdown
Contributor

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM but I would like a review from @colesbury as well.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
@bedevere-app bedevere-app Bot added awaiting merge and removed awaiting review labels Feb 18, 2025
colesbury
colesbury approved these changes Feb 18, 2025
View reviewed changes
@kumaraditya303
kumaraditya303 merged commit 1b6bef8 into python:main Feb 19, 2025
@bedevere-app bedevere-app Bot removed the awaiting merge label Feb 19, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@colesbury colesbury colesbury approved these changes
@kumaraditya303 kumaraditya303 kumaraditya303 approved these changes

Uh oh!

There was an error while loading. Please reload this page.

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Uh oh!

There was an error while loading. Please reload this page.

3 participants

@tom-pytel @kumaraditya303 @colesbury
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
You are visitor 01402520 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.