GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — gh-110481: Implement biased reference counting by colesbury · Pull Request #110764 · python/cpython

Conversation

@colesbury

@colesbury colesbury commented Oct 12, 2023 •
edited by bedevere-app Bot
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Contributor

CPython's current reference counting implementation would not be thread-safe without the GIL. This implements biased reference counting in --disable-gil builds, which is thread-safe and has lower execution overhead compared to plain atomic reference counting. The design is described in the "Biased Reference Counting" section of PEP 703.

  • Issue: Implement biased reference counting in --disable-gil builds #110481

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
erlend-aasland, corona10, bumper-furkan-onder, furkanonder, laike9m, and danijar reacted with thumbs up emoji stonebig, FFY00, zhangkaizhao, jezdez, furkanonder, and danijar reacted with heart emoji itamaro, furkanonder, jerinphilip, ermakov-oleg, corona10, danijar, and jamestiotio reacted with rocket emoji
All reactions
  • 👍 6 reactions
  • ❤️ 6 reactions
  • 🚀 7 reactions
@colesbury
colesbury requested a review from vstinner October 12, 2023 10:06
@colesbury
colesbury requested review from markshannon and rhettinger as code owners October 12, 2023 10:06
@bedevere-app bedevere-app Bot added the awaiting review label Oct 12, 2023
@bedevere-app bedevere-app Bot mentioned this pull request Oct 12, 2023
Implement biased reference counting in --disable-gil builds #110481
Closed
@colesbury

colesbury commented Oct 12, 2023

Copy link
Copy Markdown
Contributor Author

Note for reviewer: the first commit in this PR refactors the internal-only _PyObject_HEAD_INIT macro to not have a trailing comma, which avoids the need for a number of #ifdef Py_NOGIL in the subsequent commit.
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
@rhettinger
rhettinger removed their request for review October 13, 2023 00:28
@corona10
corona10 self-requested a review October 17, 2023 15:36
@colesbury

colesbury commented Oct 17, 2023

Copy link
Copy Markdown
Contributor Author

@vstinner, will you have time to look at this PR this week?
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
@corona10

corona10 commented Oct 17, 2023

Copy link
Copy Markdown
Member

I read your PEP and related papers, so I can review for specific implementation of your works. I will leave review for this PR in this week :)
itamaro, colesbury, and danijar reacted with heart emoji
All reactions
  • ❤️ 3 reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
corona10
corona10 reviewed Oct 18, 2023
View reviewed changes
Comment thread Include/internal/pycore_object.h Outdated
{
if (op) {
#ifdef Py_NOGIL
op->ob_tid = 0;

@corona10 corona10 Oct 18, 2023

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we declare the constant variable for this? Something like UNKNOWN_TID?
I mean if someone already read the PEP, the person knows what the state that tid is zero stands for, but if not some kind of variable name will be helpful to read.

But I am not sure which will be the best naming

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
colesbury reacted with thumbs up emoji
All reactions
  • 👍 1 reaction

@colesbury colesbury Oct 19, 2023

Copy link
Copy Markdown
Contributor Author

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've named it _Py_UNOWNED_TID because these are objects that are not owned by any thread.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
All reactions
corona10
corona10 reviewed Oct 18, 2023
View reviewed changes
Comment thread Include/object.h Resolved

Uh oh!

There was an error while loading. Please reload this page.
corona10
corona10 reviewed Oct 18, 2023
View reviewed changes
Comment thread Include/object.h Outdated
// zero. The call will deallocate the object if the shared refcount is also
// zero. Otherwise, the thread gives up ownership and merges the reference
// count fields.
PyAPI_FUNC(void) _Py_MergeZeroRefcount(PyObject *);

@corona10 corona10 Oct 18, 2023 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When I read the code, the purpose of this function is to handle the object when the local ref count is zero.
So might be, we can improve function name to understand what's happening

Suggested change
PyAPI_FUNC(void) _Py_MergeZeroRefcount(PyObject *);
PyAPI_FUNC(void) _Py_MergeZeroRefLocalcount(PyObject *);

or

Suggested change
PyAPI_FUNC(void) _Py_MergeZeroRefcount(PyObject *);
PyAPI_FUNC(void) _Py_MergeZeroLocalRefcount(PyObject *);

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
colesbury reacted with thumbs up emoji
All reactions
  • 👍 1 reaction

@colesbury colesbury Oct 19, 2023

Copy link
Copy Markdown
Contributor Author

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've named it _Py_MergeZeroLocalRefcount

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
corona10 reacted with thumbs up emoji
All reactions
  • 👍 1 reaction
corona10
corona10 approved these changes Oct 18, 2023
View reviewed changes

@corona10 corona10 left a comment

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall looks good to me.
I just leave the styling things for other readers.
IMO, It will help to understand the mechanism between local refcount and shared ref count.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
All reactions
@bedevere-app bedevere-app Bot added awaiting merge and removed awaiting review labels Oct 18, 2023
corona10
corona10 reviewed Oct 19, 2023
View reviewed changes
Comment thread Objects/object.c
Py_ssize_t
_Py_ExplicitMergeRefcount(PyObject *op, Py_ssize_t extra)
{
Py_ssize_t refcnt;

@corona10 corona10 Oct 19, 2023 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Out of curiosity, Don't we have to add assert statement that op should not be an immortal object?

assert(!_Py_IsImmortal(op));

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
colesbury reacted with thumbs up emoji
All reactions
  • 👍 1 reaction

@colesbury colesbury Oct 19, 2023

Copy link
Copy Markdown
Contributor Author

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, that's a good assertion to add.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
All reactions
@corona10 corona10 mentioned this pull request Oct 19, 2023
gh-111062: Add nogil build test on GitHub Actions #111060
Closed
@colesbury
colesbury force-pushed the biased_reference_counting branch from 2144e0e to 9bc2b6b Compare October 19, 2023 16:43
@colesbury
colesbury requested review from brettcannon, ericsnowcurrently, ncoghlan, tiran and warsaw as code owners October 19, 2023 16:43
colesbury
colesbury commented Oct 19, 2023
View reviewed changes

@colesbury colesbury left a comment

Copy link
Copy Markdown
Contributor Author

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for reviewing this! I've updated the PR and rebased on the main branch.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
All reactions
Comment thread Include/internal/pycore_object.h Outdated
{
if (op) {
#ifdef Py_NOGIL
op->ob_tid = 0;

@colesbury colesbury Oct 19, 2023

Copy link
Copy Markdown
Contributor Author

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've named it _Py_UNOWNED_TID because these are objects that are not owned by any thread.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
All reactions
Comment thread Include/object.h Outdated
// zero. The call will deallocate the object if the shared refcount is also
// zero. Otherwise, the thread gives up ownership and merges the reference
// count fields.
PyAPI_FUNC(void) _Py_MergeZeroRefcount(PyObject *);

@colesbury colesbury Oct 19, 2023

Copy link
Copy Markdown
Contributor Author

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've named it _Py_MergeZeroLocalRefcount

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
corona10 reacted with thumbs up emoji
All reactions
  • 👍 1 reaction
Comment thread Objects/object.c
Py_ssize_t
_Py_ExplicitMergeRefcount(PyObject *op, Py_ssize_t extra)
{
Py_ssize_t refcnt;

@colesbury colesbury Oct 19, 2023

Copy link
Copy Markdown
Contributor Author

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, that's a good assertion to add.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
All reactions
Comment thread Include/object.h Resolved

Uh oh!

There was an error while loading. Please reload this page.
@brettcannon
brettcannon removed their request for review October 19, 2023 18:16
corona10
corona10 reviewed Oct 19, 2023
View reviewed changes
Comment thread Modules/_testcapi/heaptype_relative.c Resolved

Uh oh!

There was an error while loading. Please reload this page.
corona10
corona10 requested changes Oct 20, 2023
View reviewed changes

@corona10 corona10 left a comment

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@colesbury In my macOS, the PR is failed to build :) Pleace check

9 warnings generated.
gcc  -fno-strict-overflow -Wsign-compare -Wunreachable-code -DNDEBUG -g -O3 -Wall    -std=c11 -Wextra -Wno-unused-parameter -Wno-missing-field-initializers -Wstrict-prototypes -Werror=implicit-function-declaration -fvisibility=hidden  -I./Include/internal  -I. -I./Include     -c ./Modules/xxsubtype.c -o Modules/xxsubtype.o
In file included from ./Modules/_scproxy.c:9:
In file included from ./Include/Python.h:52:
./Include/object.h:276:22: error: implicit declaration of function '_Py_atomic_load_uint32_relaxed' is invalid in C99 [-Werror,-Wimplicit-function-declaration]
    uint32_t local = _Py_atomic_load_uint32_relaxed(&ob->ob_ref_local);
                     ^
./Include/object.h:280:25: error: implicit declaration of function '_Py_atomic_load_ssize_relaxed' is invalid in C99 [-Werror,-Wimplicit-function-declaration]
    Py_ssize_t shared = _Py_atomic_load_ssize_relaxed(&ob->ob_ref_shared);
                        ^
./Include/object.h:280:25: note: did you mean '_Py_atomic_load_uint32_relaxed'?
./Include/object.h:276:22: note: '_Py_atomic_load_uint32_relaxed' declared here
    uint32_t local = _Py_atomic_load_uint32_relaxed(&ob->ob_ref_local);
                     ^
./Include/object.h:343:9: error: implicit declaration of function '_Py_IsOnwedByCurrentThread' is invalid in C99 [-Werror,-Wimplicit-function-declaration]
    if (_Py_IsOnwedByCurrentThread(ob)) {
        ^

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
All reactions
@bedevere-app bedevere-app Bot added awaiting changes and removed awaiting merge labels Oct 20, 2023
colesbury added 2 commits October 31, 2023 00:34
@colesbury @corona10
Add missing guards
c6c6ec6
@colesbury @corona10
Add TODOs for missing Py_DECREF specializations
ac2c957 
These aren't necessary for correctness, just performance.
@corona10
corona10 force-pushed the biased_reference_counting branch from d0b6720 to ac2c957 Compare October 30, 2023 15:34
corona10
corona10 approved these changes Oct 30, 2023
View reviewed changes

@corona10 corona10 left a comment •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@colesbury cc @vstinner @ambv
LGTM Again

Let's not block the PEP-703 works for the things.
It will not affect the default implementation, and for the limited C API issue, I believe that we can solve it before we want to use free-threaded mode by default.

Let's solve it as a separate issue.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
All reactions
@corona10
corona10 enabled auto-merge (squash) October 30, 2023 15:38
@corona10
corona10 merged commit 6dfb8fe into python:main Oct 30, 2023
@bedevere-app bedevere-app Bot removed the awaiting merge label Oct 30, 2023
@bedevere-bot

bedevere-bot commented Oct 30, 2023

Copy link
Copy Markdown

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot s390x Debian 3.x has failed when building commit 6dfb8fe.

What do you need to do:

  1. Don't panic.
  2. Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
  3. Go to the page of the buildbot that failed (https://buildbot.python.org/all/#builders/49/builds/7039) and take a look at the build logs.
  4. Check if the failure is related to this commit (6dfb8fe) or if it is a false positive.
  5. If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/all/#builders/49/builds/7039

Summary of the results of the build (if available):

Click to see traceback logs 
remote: Enumerating objects: 96, done.        
remote: Counting objects:   1% (1/96)        
remote: Counting objects:   2% (2/96)        
remote: Counting objects:   3% (3/96)        
remote: Counting objects:   4% (4/96)        
remote: Counting objects:   5% (5/96)        
remote: Counting objects:   6% (6/96)        
remote: Counting objects:   7% (7/96)        
remote: Counting objects:   8% (8/96)        
remote: Counting objects:   9% (9/96)        
remote: Counting objects:  10% (10/96)        
remote: Counting objects:  11% (11/96)        
remote: Counting objects:  12% (12/96)        
remote: Counting objects:  13% (13/96)        
remote: Counting objects:  14% (14/96)        
remote: Counting objects:  15% (15/96)        
remote: Counting objects:  16% (16/96)        
remote: Counting objects:  17% (17/96)        
remote: Counting objects:  18% (18/96)        
remote: Counting objects:  19% (19/96)        
remote: Counting objects:  20% (20/96)        
remote: Counting objects:  21% (21/96)        
remote: Counting objects:  22% (22/96)        
remote: Counting objects:  23% (23/96)        
remote: Counting objects:  25% (24/96)        
remote: Counting objects:  26% (25/96)        
remote: Counting objects:  27% (26/96)        
remote: Counting objects:  28% (27/96)        
remote: Counting objects:  29% (28/96)        
remote: Counting objects:  30% (29/96)        
remote: Counting objects:  31% (30/96)        
remote: Counting objects:  32% (31/96)        
remote: Counting objects:  33% (32/96)        
remote: Counting objects:  34% (33/96)        
remote: Counting objects:  35% (34/96)        
remote: Counting objects:  36% (35/96)        
remote: Counting objects:  37% (36/96)        
remote: Counting objects:  38% (37/96)        
remote: Counting objects:  39% (38/96)        
remote: Counting objects:  40% (39/96)        
remote: Counting objects:  41% (40/96)        
remote: Counting objects:  42% (41/96)        
remote: Counting objects:  43% (42/96)        
remote: Counting objects:  44% (43/96)        
remote: Counting objects:  45% (44/96)        
remote: Counting objects:  46% (45/96)        
remote: Counting objects:  47% (46/96)        
remote: Counting objects:  48% (47/96)        
remote: Counting objects:  50% (48/96)        
remote: Counting objects:  51% (49/96)        
remote: Counting objects:  52% (50/96)        
remote: Counting objects:  53% (51/96)        
remote: Counting objects:  54% (52/96)        
remote: Counting objects:  55% (53/96)        
remote: Counting objects:  56% (54/96)        
remote: Counting objects:  57% (55/96)        
remote: Counting objects:  58% (56/96)        
remote: Counting objects:  59% (57/96)        
remote: Counting objects:  60% (58/96)        
remote: Counting objects:  61% (59/96)        
remote: Counting objects:  62% (60/96)        
remote: Counting objects:  63% (61/96)        
remote: Counting objects:  64% (62/96)        
remote: Counting objects:  65% (63/96)        
remote: Counting objects:  66% (64/96)        
remote: Counting objects:  67% (65/96)        
remote: Counting objects:  68% (66/96)        
remote: Counting objects:  69% (67/96)        
remote: Counting objects:  70% (68/96)        
remote: Counting objects:  71% (69/96)        
remote: Counting objects:  72% (70/96)        
remote: Counting objects:  73% (71/96)        
remote: Counting objects:  75% (72/96)        
remote: Counting objects:  76% (73/96)        
remote: Counting objects:  77% (74/96)        
remote: Counting objects:  78% (75/96)        
remote: Counting objects:  79% (76/96)        
remote: Counting objects:  80% (77/96)        
remote: Counting objects:  81% (78/96)        
remote: Counting objects:  82% (79/96)        
remote: Counting objects:  83% (80/96)        
remote: Counting objects:  84% (81/96)        
remote: Counting objects:  85% (82/96)        
remote: Counting objects:  86% (83/96)        
remote: Counting objects:  87% (84/96)        
remote: Counting objects:  88% (85/96)        
remote: Counting objects:  89% (86/96)        
remote: Counting objects:  90% (87/96)        
remote: Counting objects:  91% (88/96)        
remote: Counting objects:  92% (89/96)        
remote: Counting objects:  93% (90/96)        
remote: Counting objects:  94% (91/96)        
remote: Counting objects:  95% (92/96)        
remote: Counting objects:  96% (93/96)        
remote: Counting objects:  97% (94/96)        
remote: Counting objects:  98% (95/96)        
remote: Counting objects: 100% (96/96)        
remote: Counting objects: 100% (96/96), done.        
remote: Compressing objects:   2% (1/42)        
remote: Compressing objects:   4% (2/42)        
remote: Compressing objects:   7% (3/42)        
remote: Compressing objects:   9% (4/42)        
remote: Compressing objects:  11% (5/42)        
remote: Compressing objects:  14% (6/42)        
remote: Compressing objects:  16% (7/42)        
remote: Compressing objects:  19% (8/42)        
remote: Compressing objects:  21% (9/42)        
remote: Compressing objects:  23% (10/42)        
remote: Compressing objects:  26% (11/42)        
remote: Compressing objects:  28% (12/42)        
remote: Compressing objects:  30% (13/42)        
remote: Compressing objects:  33% (14/42)        
remote: Compressing objects:  35% (15/42)        
remote: Compressing objects:  38% (16/42)        
remote: Compressing objects:  40% (17/42)        
remote: Compressing objects:  42% (18/42)        
remote: Compressing objects:  45% (19/42)        
remote: Compressing objects:  47% (20/42)        
remote: Compressing objects:  50% (21/42)        
remote: Compressing objects:  52% (22/42)        
remote: Compressing objects:  54% (23/42)        
remote: Compressing objects:  57% (24/42)        
remote: Compressing objects:  59% (25/42)        
remote: Compressing objects:  61% (26/42)        
remote: Compressing objects:  64% (27/42)        
remote: Compressing objects:  66% (28/42)        
remote: Compressing objects:  69% (29/42)        
remote: Compressing objects:  71% (30/42)        
remote: Compressing objects:  73% (31/42)        
remote: Compressing objects:  76% (32/42)        
remote: Compressing objects:  78% (33/42)        
remote: Compressing objects:  80% (34/42)        
remote: Compressing objects:  83% (35/42)        
remote: Compressing objects:  85% (36/42)        
remote: Compressing objects:  88% (37/42)        
remote: Compressing objects:  90% (38/42)        
remote: Compressing objects:  92% (39/42)        
remote: Compressing objects:  95% (40/42)        
remote: Compressing objects:  97% (41/42)        
remote: Compressing objects: 100% (42/42)        
remote: Compressing objects: 100% (42/42), done.        
remote: Total 49 (delta 47), reused 8 (delta 7), pack-reused 0        
From https://github.com/python/cpython
 * branch                  main       -> FETCH_HEAD
Note: switching to '6dfb8fe0236718e9afc8136ff2b58dcfbc182022'.

You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by switching back to a branch.

If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -c with the switch command. Example:

  git switch -c <new-branch-name>

Or undo this operation with:

  git switch -

Turn off this advice by setting config variable advice.detachedHead to false

HEAD is now at 6dfb8fe023 gh-110481: Implement biased reference counting (gh-110764)
Switched to and reset branch 'main'

In file included from ./Include/Python.h:52,
                 from Parser/token.c:3:
./Include/object.h: In function ‘_Py_ThreadId’:
./Include/object.h:258:5: error: #error "define _Py_ThreadId for this platform"
  258 |   # error "define _Py_ThreadId for this platform"
      |     ^~~~~
In file included from ./Include/Python.h:52,
                 from Parser/pegen.h:4,
                 from Parser/parser.c:2:
./Include/object.h: In function ‘_Py_ThreadId’:
./Include/object.h:258:5: error: #error "define _Py_ThreadId for this platform"
  258 |   # error "define _Py_ThreadId for this platform"
      |     ^~~~~
In file included from ./Include/Python.h:52,
                 from ./Programs/python.c:3:
./Include/object.h: In function ‘_Py_ThreadId’:
./Include/object.h:258:5: error: #error "define _Py_ThreadId for this platform"
  258 |   # error "define _Py_ThreadId for this platform"
      |     ^~~~~
make: *** [Makefile:2757: Parser/token.o] Error 1
make: *** Waiting for unfinished jobs....
In file included from ./Include/Python.h:52,
                 from Parser/pegen.c:1:
./Include/object.h: In function ‘_Py_ThreadId’:
./Include/object.h:258:5: error: #error "define _Py_ThreadId for this platform"
  258 |   # error "define _Py_ThreadId for this platform"
      |     ^~~~~
make: *** [Makefile:1530: Programs/python.o] Error 1
In file included from ./Include/Python.h:52,
                 from Parser/pegen_errors.c:1:
./Include/object.h: In function ‘_Py_ThreadId’:
./Include/object.h:258:5: error: #error "define _Py_ThreadId for this platform"
  258 |   # error "define _Py_ThreadId for this platform"
      |     ^~~~~
In file included from ./Include/Python.h:52,
                 from Parser/action_helpers.c:1:
./Include/object.h: In function ‘_Py_ThreadId’:
./Include/object.h:258:5: error: #error "define _Py_ThreadId for this platform"
  258 |   # error "define _Py_ThreadId for this platform"
      |     ^~~~~
make: *** [Makefile:2757: Parser/pegen_errors.o] Error 1
make: *** [Makefile:2757: Parser/pegen.o] Error 1
make: *** [Makefile:2757: Parser/action_helpers.o] Error 1
make: *** [Makefile:2757: Parser/parser.o] Error 1

find: ‘build’: No such file or directory
find: ‘build’: No such file or directory
find: ‘build’: No such file or directory
find: ‘build’: No such file or directory
make: [Makefile:2849: clean-retain-profile] Error 1 (ignored)
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
@bedevere-bot

bedevere-bot commented Oct 30, 2023

Copy link
Copy Markdown

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot s390x RHEL7 3.x has failed when building commit 6dfb8fe.

What do you need to do:

  1. Don't panic.
  2. Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
  3. Go to the page of the buildbot that failed (https://buildbot.python.org/all/#builders/179/builds/5821) and take a look at the build logs.
  4. Check if the failure is related to this commit (6dfb8fe) or if it is a false positive.
  5. If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/all/#builders/179/builds/5821

Summary of the results of the build (if available):

Click to see traceback logs 
remote: Enumerating objects: 96, done.�[K
remote: Counting objects:   1% (1/96)�[K
remote: Counting objects:   2% (2/96)�[K
remote: Counting objects:   3% (3/96)�[K
remote: Counting objects:   4% (4/96)�[K
remote: Counting objects:   5% (5/96)�[K
remote: Counting objects:   6% (6/96)�[K
remote: Counting objects:   7% (7/96)�[K
remote: Counting objects:   8% (8/96)�[K
remote: Counting objects:   9% (9/96)�[K
remote: Counting objects:  10% (10/96)�[K
remote: Counting objects:  11% (11/96)�[K
remote: Counting objects:  12% (12/96)�[K
remote: Counting objects:  13% (13/96)�[K
remote: Counting objects:  14% (14/96)�[K
remote: Counting objects:  15% (15/96)�[K
remote: Counting objects:  16% (16/96)�[K
remote: Counting objects:  17% (17/96)�[K
remote: Counting objects:  18% (18/96)�[K
remote: Counting objects:  19% (19/96)�[K
remote: Counting objects:  20% (20/96)�[K
remote: Counting objects:  21% (21/96)�[K
remote: Counting objects:  22% (22/96)�[K
remote: Counting objects:  23% (23/96)�[K
remote: Counting objects:  25% (24/96)�[K
remote: Counting objects:  26% (25/96)�[K
remote: Counting objects:  27% (26/96)�[K
remote: Counting objects:  28% (27/96)�[K
remote: Counting objects:  29% (28/96)�[K
remote: Counting objects:  30% (29/96)�[K
remote: Counting objects:  31% (30/96)�[K
remote: Counting objects:  32% (31/96)�[K
remote: Counting objects:  33% (32/96)�[K
remote: Counting objects:  34% (33/96)�[K
remote: Counting objects:  35% (34/96)�[K
remote: Counting objects:  36% (35/96)�[K
remote: Counting objects:  37% (36/96)�[K
remote: Counting objects:  38% (37/96)�[K
remote: Counting objects:  39% (38/96)�[K
remote: Counting objects:  40% (39/96)�[K
remote: Counting objects:  41% (40/96)�[K
remote: Counting objects:  42% (41/96)�[K
remote: Counting objects:  43% (42/96)�[K
remote: Counting objects:  44% (43/96)�[K
remote: Counting objects:  45% (44/96)�[K
remote: Counting objects:  46% (45/96)�[K
remote: Counting objects:  47% (46/96)�[K
remote: Counting objects:  48% (47/96)�[K
remote: Counting objects:  50% (48/96)�[K
remote: Counting objects:  51% (49/96)�[K
remote: Counting objects:  52% (50/96)�[K
remote: Counting objects:  53% (51/96)�[K
remote: Counting objects:  54% (52/96)�[K
remote: Counting objects:  55% (53/96)�[K
remote: Counting objects:  56% (54/96)�[K
remote: Counting objects:  57% (55/96)�[K
remote: Counting objects:  58% (56/96)�[K
remote: Counting objects:  59% (57/96)�[K
remote: Counting objects:  60% (58/96)�[K
remote: Counting objects:  61% (59/96)�[K
remote: Counting objects:  62% (60/96)�[K
remote: Counting objects:  63% (61/96)�[K
remote: Counting objects:  64% (62/96)�[K
remote: Counting objects:  65% (63/96)�[K
remote: Counting objects:  66% (64/96)�[K
remote: Counting objects:  67% (65/96)�[K
remote: Counting objects:  68% (66/96)�[K
remote: Counting objects:  69% (67/96)�[K
remote: Counting objects:  70% (68/96)�[K
remote: Counting objects:  71% (69/96)�[K
remote: Counting objects:  72% (70/96)�[K
remote: Counting objects:  73% (71/96)�[K
remote: Counting objects:  75% (72/96)�[K
remote: Counting objects:  76% (73/96)�[K
remote: Counting objects:  77% (74/96)�[K
remote: Counting objects:  78% (75/96)�[K
remote: Counting objects:  79% (76/96)�[K
remote: Counting objects:  80% (77/96)�[K
remote: Counting objects:  81% (78/96)�[K
remote: Counting objects:  82% (79/96)�[K
remote: Counting objects:  83% (80/96)�[K
remote: Counting objects:  84% (81/96)�[K
remote: Counting objects:  85% (82/96)�[K
remote: Counting objects:  86% (83/96)�[K
remote: Counting objects:  87% (84/96)�[K
remote: Counting objects:  88% (85/96)�[K
remote: Counting objects:  89% (86/96)�[K
remote: Counting objects:  90% (87/96)�[K
remote: Counting objects:  91% (88/96)�[K
remote: Counting objects:  92% (89/96)�[K
remote: Counting objects:  93% (90/96)�[K
remote: Counting objects:  94% (91/96)�[K
remote: Counting objects:  95% (92/96)�[K
remote: Counting objects:  96% (93/96)�[K
remote: Counting objects:  97% (94/96)�[K
remote: Counting objects:  98% (95/96)�[K
remote: Counting objects: 100% (96/96)�[K
remote: Counting objects: 100% (96/96), done.�[K
remote: Compressing objects:   2% (1/42)�[K
remote: Compressing objects:   4% (2/42)�[K
remote: Compressing objects:   7% (3/42)�[K
remote: Compressing objects:   9% (4/42)�[K
remote: Compressing objects:  11% (5/42)�[K
remote: Compressing objects:  14% (6/42)�[K
remote: Compressing objects:  16% (7/42)�[K
remote: Compressing objects:  19% (8/42)�[K
remote: Compressing objects:  21% (9/42)�[K
remote: Compressing objects:  23% (10/42)�[K
remote: Compressing objects:  26% (11/42)�[K
remote: Compressing objects:  28% (12/42)�[K
remote: Compressing objects:  30% (13/42)�[K
remote: Compressing objects:  33% (14/42)�[K
remote: Compressing objects:  35% (15/42)�[K
remote: Compressing objects:  38% (16/42)�[K
remote: Compressing objects:  40% (17/42)�[K
remote: Compressing objects:  42% (18/42)�[K
remote: Compressing objects:  45% (19/42)�[K
remote: Compressing objects:  47% (20/42)�[K
remote: Compressing objects:  50% (21/42)�[K
remote: Compressing objects:  52% (22/42)�[K
remote: Compressing objects:  54% (23/42)�[K
remote: Compressing objects:  57% (24/42)�[K
remote: Compressing objects:  59% (25/42)�[K
remote: Compressing objects:  61% (26/42)�[K
remote: Compressing objects:  64% (27/42)�[K
remote: Compressing objects:  66% (28/42)�[K
remote: Compressing objects:  69% (29/42)�[K
remote: Compressing objects:  71% (30/42)�[K
remote: Compressing objects:  73% (31/42)�[K
remote: Compressing objects:  76% (32/42)�[K
remote: Compressing objects:  78% (33/42)�[K
remote: Compressing objects:  80% (34/42)�[K
remote: Compressing objects:  83% (35/42)�[K
remote: Compressing objects:  85% (36/42)�[K
remote: Compressing objects:  88% (37/42)�[K
remote: Compressing objects:  90% (38/42)�[K
remote: Compressing objects:  92% (39/42)�[K
remote: Compressing objects:  95% (40/42)�[K
remote: Compressing objects:  97% (41/42)�[K
remote: Compressing objects: 100% (42/42)�[K
remote: Compressing objects: 100% (42/42), done.�[K
remote: Total 49 (delta 47), reused 8 (delta 7), pack-reused 0�[K
From https://github.com/python/cpython
 * branch            main       -> FETCH_HEAD
Note: checking out '6dfb8fe0236718e9afc8136ff2b58dcfbc182022'.

You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by performing another checkout.

If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -b with the checkout command again. Example:

  git checkout -b new_branch_name

HEAD is now at 6dfb8fe... gh-110481: Implement biased reference counting (gh-110764)
Switched to and reset branch 'main'

In file included from ./Include/Python.h:52:0,
                 from Parser/token.c:3:
./Include/object.h: In function ‘_Py_ThreadId’:
./Include/object.h:258:5: error: #error "define _Py_ThreadId for this platform"
   # error "define _Py_ThreadId for this platform"
     ^
In file included from ./Include/Python.h:52:0,
                 from ./Programs/python.c:3:
./Include/object.h: In function ‘_Py_ThreadId’:
./Include/object.h:258:5: error: #error "define _Py_ThreadId for this platform"
   # error "define _Py_ThreadId for this platform"
     ^
In file included from ./Include/Python.h:52:0,
                 from Parser/pegen.c:1:
./Include/object.h: In function ‘_Py_ThreadId’:
./Include/object.h:258:5: error: #error "define _Py_ThreadId for this platform"
   # error "define _Py_ThreadId for this platform"
     ^
In file included from ./Include/Python.h:52:0,
                 from Parser/pegen.h:4,
                 from Parser/parser.c:2:
./Include/object.h: In function ‘_Py_ThreadId’:
./Include/object.h:258:5: error: #error "define _Py_ThreadId for this platform"
   # error "define _Py_ThreadId for this platform"
     ^
make: *** [Parser/token.o] Error 1
make: *** Waiting for unfinished jobs....
In file included from ./Include/Python.h:52:0,
                 from Parser/action_helpers.c:1:
./Include/object.h: In function ‘_Py_ThreadId’:
./Include/object.h:258:5: error: #error "define _Py_ThreadId for this platform"
   # error "define _Py_ThreadId for this platform"
     ^
In file included from ./Include/Python.h:52:0,
                 from Parser/pegen_errors.c:1:
./Include/object.h: In function ‘_Py_ThreadId’:
./Include/object.h:258:5: error: #error "define _Py_ThreadId for this platform"
   # error "define _Py_ThreadId for this platform"
     ^
make: *** [Programs/python.o] Error 1
make: *** [Parser/pegen.o] Error 1
make: *** [Parser/pegen_errors.o] Error 1
make: *** [Parser/action_helpers.o] Error 1
make: *** [Parser/parser.o] Error 1

find: ‘build’: No such file or directory
find: ‘build’: No such file or directory
find: ‘build’: No such file or directory
find: ‘build’: No such file or directory
make: [clean-retain-profile] Error 1 (ignored)
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
@bedevere-bot

bedevere-bot commented Oct 30, 2023

Copy link
Copy Markdown

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot s390x SLES 3.x has failed when building commit 6dfb8fe.

What do you need to do:

  1. Don't panic.
  2. Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
  3. Go to the page of the buildbot that failed (https://buildbot.python.org/all/#builders/540/builds/7003) and take a look at the build logs.
  4. Check if the failure is related to this commit (6dfb8fe) or if it is a false positive.
  5. If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/all/#builders/540/builds/7003

Summary of the results of the build (if available):

Click to see traceback logs 
remote: Enumerating objects: 96, done.        
remote: Counting objects:   1% (1/96)        
remote: Counting objects:   2% (2/96)        
remote: Counting objects:   3% (3/96)        
remote: Counting objects:   4% (4/96)        
remote: Counting objects:   5% (5/96)        
remote: Counting objects:   6% (6/96)        
remote: Counting objects:   7% (7/96)        
remote: Counting objects:   8% (8/96)        
remote: Counting objects:   9% (9/96)        
remote: Counting objects:  10% (10/96)        
remote: Counting objects:  11% (11/96)        
remote: Counting objects:  12% (12/96)        
remote: Counting objects:  13% (13/96)        
remote: Counting objects:  14% (14/96)        
remote: Counting objects:  15% (15/96)        
remote: Counting objects:  16% (16/96)        
remote: Counting objects:  17% (17/96)        
remote: Counting objects:  18% (18/96)        
remote: Counting objects:  19% (19/96)        
remote: Counting objects:  20% (20/96)        
remote: Counting objects:  21% (21/96)        
remote: Counting objects:  22% (22/96)        
remote: Counting objects:  23% (23/96)        
remote: Counting objects:  25% (24/96)        
remote: Counting objects:  26% (25/96)        
remote: Counting objects:  27% (26/96)        
remote: Counting objects:  28% (27/96)        
remote: Counting objects:  29% (28/96)        
remote: Counting objects:  30% (29/96)        
remote: Counting objects:  31% (30/96)        
remote: Counting objects:  32% (31/96)        
remote: Counting objects:  33% (32/96)        
remote: Counting objects:  34% (33/96)        
remote: Counting objects:  35% (34/96)        
remote: Counting objects:  36% (35/96)        
remote: Counting objects:  37% (36/96)        
remote: Counting objects:  38% (37/96)        
remote: Counting objects:  39% (38/96)        
remote: Counting objects:  40% (39/96)        
remote: Counting objects:  41% (40/96)        
remote: Counting objects:  42% (41/96)        
remote: Counting objects:  43% (42/96)        
remote: Counting objects:  44% (43/96)        
remote: Counting objects:  45% (44/96)        
remote: Counting objects:  46% (45/96)        
remote: Counting objects:  47% (46/96)        
remote: Counting objects:  48% (47/96)        
remote: Counting objects:  50% (48/96)        
remote: Counting objects:  51% (49/96)        
remote: Counting objects:  52% (50/96)        
remote: Counting objects:  53% (51/96)        
remote: Counting objects:  54% (52/96)        
remote: Counting objects:  55% (53/96)        
remote: Counting objects:  56% (54/96)        
remote: Counting objects:  57% (55/96)        
remote: Counting objects:  58% (56/96)        
remote: Counting objects:  59% (57/96)        
remote: Counting objects:  60% (58/96)        
remote: Counting objects:  61% (59/96)        
remote: Counting objects:  62% (60/96)        
remote: Counting objects:  63% (61/96)        
remote: Counting objects:  64% (62/96)        
remote: Counting objects:  65% (63/96)        
remote: Counting objects:  66% (64/96)        
remote: Counting objects:  67% (65/96)        
remote: Counting objects:  68% (66/96)        
remote: Counting objects:  69% (67/96)        
remote: Counting objects:  70% (68/96)        
remote: Counting objects:  71% (69/96)        
remote: Counting objects:  72% (70/96)        
remote: Counting objects:  73% (71/96)        
remote: Counting objects:  75% (72/96)        
remote: Counting objects:  76% (73/96)        
remote: Counting objects:  77% (74/96)        
remote: Counting objects:  78% (75/96)        
remote: Counting objects:  79% (76/96)        
remote: Counting objects:  80% (77/96)        
remote: Counting objects:  81% (78/96)        
remote: Counting objects:  82% (79/96)        
remote: Counting objects:  83% (80/96)        
remote: Counting objects:  84% (81/96)        
remote: Counting objects:  85% (82/96)        
remote: Counting objects:  86% (83/96)        
remote: Counting objects:  87% (84/96)        
remote: Counting objects:  88% (85/96)        
remote: Counting objects:  89% (86/96)        
remote: Counting objects:  90% (87/96)        
remote: Counting objects:  91% (88/96)        
remote: Counting objects:  92% (89/96)        
remote: Counting objects:  93% (90/96)        
remote: Counting objects:  94% (91/96)        
remote: Counting objects:  95% (92/96)        
remote: Counting objects:  96% (93/96)        
remote: Counting objects:  97% (94/96)        
remote: Counting objects:  98% (95/96)        
remote: Counting objects: 100% (96/96)        
remote: Counting objects: 100% (96/96), done.        
remote: Compressing objects:   2% (1/42)        
remote: Compressing objects:   4% (2/42)        
remote: Compressing objects:   7% (3/42)        
remote: Compressing objects:   9% (4/42)        
remote: Compressing objects:  11% (5/42)        
remote: Compressing objects:  14% (6/42)        
remote: Compressing objects:  16% (7/42)        
remote: Compressing objects:  19% (8/42)        
remote: Compressing objects:  21% (9/42)        
remote: Compressing objects:  23% (10/42)        
remote: Compressing objects:  26% (11/42)        
remote: Compressing objects:  28% (12/42)        
remote: Compressing objects:  30% (13/42)        
remote: Compressing objects:  33% (14/42)        
remote: Compressing objects:  35% (15/42)        
remote: Compressing objects:  38% (16/42)        
remote: Compressing objects:  40% (17/42)        
remote: Compressing objects:  42% (18/42)        
remote: Compressing objects:  45% (19/42)        
remote: Compressing objects:  47% (20/42)        
remote: Compressing objects:  50% (21/42)        
remote: Compressing objects:  52% (22/42)        
remote: Compressing objects:  54% (23/42)        
remote: Compressing objects:  57% (24/42)        
remote: Compressing objects:  59% (25/42)        
remote: Compressing objects:  61% (26/42)        
remote: Compressing objects:  64% (27/42)        
remote: Compressing objects:  66% (28/42)        
remote: Compressing objects:  69% (29/42)        
remote: Compressing objects:  71% (30/42)        
remote: Compressing objects:  73% (31/42)        
remote: Compressing objects:  76% (32/42)        
remote: Compressing objects:  78% (33/42)        
remote: Compressing objects:  80% (34/42)        
remote: Compressing objects:  83% (35/42)        
remote: Compressing objects:  85% (36/42)        
remote: Compressing objects:  88% (37/42)        
remote: Compressing objects:  90% (38/42)        
remote: Compressing objects:  92% (39/42)        
remote: Compressing objects:  95% (40/42)        
remote: Compressing objects:  97% (41/42)        
remote: Compressing objects: 100% (42/42)        
remote: Compressing objects: 100% (42/42), done.        
remote: Total 49 (delta 47), reused 8 (delta 7), pack-reused 0        
From https://github.com/python/cpython
 * branch                  main       -> FETCH_HEAD
Note: switching to '6dfb8fe0236718e9afc8136ff2b58dcfbc182022'.

You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by switching back to a branch.

If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -c with the switch command. Example:

  git switch -c <new-branch-name>

Or undo this operation with:

  git switch -

Turn off this advice by setting config variable advice.detachedHead to false

HEAD is now at 6dfb8fe023 gh-110481: Implement biased reference counting (gh-110764)
Switched to and reset branch 'main'

In file included from ./Include/Python.h:52:0,
                 from Parser/pegen.c:1:
./Include/object.h: In function ‘_Py_ThreadId’:
./Include/object.h:258:5: error: #error "define _Py_ThreadId for this platform"
   # error "define _Py_ThreadId for this platform"
     ^~~~~
In file included from ./Include/Python.h:52:0,
                 from ./Programs/python.c:3:
./Include/object.h: In function ‘_Py_ThreadId’:
./Include/object.h:258:5: error: #error "define _Py_ThreadId for this platform"
   # error "define _Py_ThreadId for this platform"
     ^~~~~
In file included from ./Include/Python.h:52:0,
                 from Parser/token.c:3:
./Include/object.h: In function ‘_Py_ThreadId’:
./Include/object.h:258:5: error: #error "define _Py_ThreadId for this platform"
   # error "define _Py_ThreadId for this platform"
     ^~~~~
In file included from ./Include/Python.h:52:0,
                 from Parser/action_helpers.c:1:
./Include/object.h: In function ‘_Py_ThreadId’:
./Include/object.h:258:5: error: #error "define _Py_ThreadId for this platform"
   # error "define _Py_ThreadId for this platform"
     ^~~~~
make: *** [Makefile:1527: Programs/python.o] Error 1
make: *** Waiting for unfinished jobs....
make: *** [Makefile:2754: Parser/token.o] Error 1
In file included from ./Include/Python.h:52:0,
                 from Parser/pegen.h:4,
                 from Parser/parser.c:2:
./Include/object.h: In function ‘_Py_ThreadId’:
./Include/object.h:258:5: error: #error "define _Py_ThreadId for this platform"
   # error "define _Py_ThreadId for this platform"
     ^~~~~
make: *** [Makefile:2754: Parser/pegen.o] Error 1
In file included from ./Include/Python.h:52:0,
                 from Parser/pegen_errors.c:1:
./Include/object.h: In function ‘_Py_ThreadId’:
./Include/object.h:258:5: error: #error "define _Py_ThreadId for this platform"
   # error "define _Py_ThreadId for this platform"
     ^~~~~
make: *** [Makefile:2754: Parser/action_helpers.o] Error 1
make: *** [Makefile:2754: Parser/pegen_errors.o] Error 1
make: *** [Makefile:2754: Parser/parser.o] Error 1

find: ‘build’: No such file or directory
find: ‘build’: No such file or directory
find: ‘build’: No such file or directory
find: ‘build’: No such file or directory
make: [Makefile:2846: clean-retain-profile] Error 1 (ignored)
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
@bedevere-bot

bedevere-bot commented Oct 30, 2023

Copy link
Copy Markdown

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot s390x RHEL7 LTO + PGO 3.x has failed when building commit 6dfb8fe.

What do you need to do:

  1. Don't panic.
  2. Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
  3. Go to the page of the buildbot that failed (https://buildbot.python.org/all/#builders/244/builds/5777) and take a look at the build logs.
  4. Check if the failure is related to this commit (6dfb8fe) or if it is a false positive.
  5. If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/all/#builders/244/builds/5777

Summary of the results of the build (if available):

Click to see traceback logs 
remote: Enumerating objects: 96, done.�[K
remote: Counting objects:   1% (1/96)�[K
remote: Counting objects:   2% (2/96)�[K
remote: Counting objects:   3% (3/96)�[K
remote: Counting objects:   4% (4/96)�[K
remote: Counting objects:   5% (5/96)�[K
remote: Counting objects:   6% (6/96)�[K
remote: Counting objects:   7% (7/96)�[K
remote: Counting objects:   8% (8/96)�[K
remote: Counting objects:   9% (9/96)�[K
remote: Counting objects:  10% (10/96)�[K
remote: Counting objects:  11% (11/96)�[K
remote: Counting objects:  12% (12/96)�[K
remote: Counting objects:  13% (13/96)�[K
remote: Counting objects:  14% (14/96)�[K
remote: Counting objects:  15% (15/96)�[K
remote: Counting objects:  16% (16/96)�[K
remote: Counting objects:  17% (17/96)�[K
remote: Counting objects:  18% (18/96)�[K
remote: Counting objects:  19% (19/96)�[K
remote: Counting objects:  20% (20/96)�[K
remote: Counting objects:  21% (21/96)�[K
remote: Counting objects:  22% (22/96)�[K
remote: Counting objects:  23% (23/96)�[K
remote: Counting objects:  25% (24/96)�[K
remote: Counting objects:  26% (25/96)�[K
remote: Counting objects:  27% (26/96)�[K
remote: Counting objects:  28% (27/96)�[K
remote: Counting objects:  29% (28/96)�[K
remote: Counting objects:  30% (29/96)�[K
remote: Counting objects:  31% (30/96)�[K
remote: Counting objects:  32% (31/96)�[K
remote: Counting objects:  33% (32/96)�[K
remote: Counting objects:  34% (33/96)�[K
remote: Counting objects:  35% (34/96)�[K
remote: Counting objects:  36% (35/96)�[K
remote: Counting objects:  37% (36/96)�[K
remote: Counting objects:  38% (37/96)�[K
remote: Counting objects:  39% (38/96)�[K
remote: Counting objects:  40% (39/96)�[K
remote: Counting objects:  41% (40/96)�[K
remote: Counting objects:  42% (41/96)�[K
remote: Counting objects:  43% (42/96)�[K
remote: Counting objects:  44% (43/96)�[K
remote: Counting objects:  45% (44/96)�[K
remote: Counting objects:  46% (45/96)�[K
remote: Counting objects:  47% (46/96)�[K
remote: Counting objects:  48% (47/96)�[K
remote: Counting objects:  50% (48/96)�[K
remote: Counting objects:  51% (49/96)�[K
remote: Counting objects:  52% (50/96)�[K
remote: Counting objects:  53% (51/96)�[K
remote: Counting objects:  54% (52/96)�[K
remote: Counting objects:  55% (53/96)�[K
remote: Counting objects:  56% (54/96)�[K
remote: Counting objects:  57% (55/96)�[K
remote: Counting objects:  58% (56/96)�[K
remote: Counting objects:  59% (57/96)�[K
remote: Counting objects:  60% (58/96)�[K
remote: Counting objects:  61% (59/96)�[K
remote: Counting objects:  62% (60/96)�[K
remote: Counting objects:  63% (61/96)�[K
remote: Counting objects:  64% (62/96)�[K
remote: Counting objects:  65% (63/96)�[K
remote: Counting objects:  66% (64/96)�[K
remote: Counting objects:  67% (65/96)�[K
remote: Counting objects:  68% (66/96)�[K
remote: Counting objects:  69% (67/96)�[K
remote: Counting objects:  70% (68/96)�[K
remote: Counting objects:  71% (69/96)�[K
remote: Counting objects:  72% (70/96)�[K
remote: Counting objects:  73% (71/96)�[K
remote: Counting objects:  75% (72/96)�[K
remote: Counting objects:  76% (73/96)�[K
remote: Counting objects:  77% (74/96)�[K
remote: Counting objects:  78% (75/96)�[K
remote: Counting objects:  79% (76/96)�[K
remote: Counting objects:  80% (77/96)�[K
remote: Counting objects:  81% (78/96)�[K
remote: Counting objects:  82% (79/96)�[K
remote: Counting objects:  83% (80/96)�[K
remote: Counting objects:  84% (81/96)�[K
remote: Counting objects:  85% (82/96)�[K
remote: Counting objects:  86% (83/96)�[K
remote: Counting objects:  87% (84/96)�[K
remote: Counting objects:  88% (85/96)�[K
remote: Counting objects:  89% (86/96)�[K
remote: Counting objects:  90% (87/96)�[K
remote: Counting objects:  91% (88/96)�[K
remote: Counting objects:  92% (89/96)�[K
remote: Counting objects:  93% (90/96)�[K
remote: Counting objects:  94% (91/96)�[K
remote: Counting objects:  95% (92/96)�[K
remote: Counting objects:  96% (93/96)�[K
remote: Counting objects:  97% (94/96)�[K
remote: Counting objects:  98% (95/96)�[K
remote: Counting objects: 100% (96/96)�[K
remote: Counting objects: 100% (96/96), done.�[K
remote: Compressing objects:   2% (1/42)�[K
remote: Compressing objects:   4% (2/42)�[K
remote: Compressing objects:   7% (3/42)�[K
remote: Compressing objects:   9% (4/42)�[K
remote: Compressing objects:  11% (5/42)�[K
remote: Compressing objects:  14% (6/42)�[K
remote: Compressing objects:  16% (7/42)�[K
remote: Compressing objects:  19% (8/42)�[K
remote: Compressing objects:  21% (9/42)�[K
remote: Compressing objects:  23% (10/42)�[K
remote: Compressing objects:  26% (11/42)�[K
remote: Compressing objects:  28% (12/42)�[K
remote: Compressing objects:  30% (13/42)�[K
remote: Compressing objects:  33% (14/42)�[K
remote: Compressing objects:  35% (15/42)�[K
remote: Compressing objects:  38% (16/42)�[K
remote: Compressing objects:  40% (17/42)�[K
remote: Compressing objects:  42% (18/42)�[K
remote: Compressing objects:  45% (19/42)�[K
remote: Compressing objects:  47% (20/42)�[K
remote: Compressing objects:  50% (21/42)�[K
remote: Compressing objects:  52% (22/42)�[K
remote: Compressing objects:  54% (23/42)�[K
remote: Compressing objects:  57% (24/42)�[K
remote: Compressing objects:  59% (25/42)�[K
remote: Compressing objects:  61% (26/42)�[K
remote: Compressing objects:  64% (27/42)�[K
remote: Compressing objects:  66% (28/42)�[K
remote: Compressing objects:  69% (29/42)�[K
remote: Compressing objects:  71% (30/42)�[K
remote: Compressing objects:  73% (31/42)�[K
remote: Compressing objects:  76% (32/42)�[K
remote: Compressing objects:  78% (33/42)�[K
remote: Compressing objects:  80% (34/42)�[K
remote: Compressing objects:  83% (35/42)�[K
remote: Compressing objects:  85% (36/42)�[K
remote: Compressing objects:  88% (37/42)�[K
remote: Compressing objects:  90% (38/42)�[K
remote: Compressing objects:  92% (39/42)�[K
remote: Compressing objects:  95% (40/42)�[K
remote: Compressing objects:  97% (41/42)�[K
remote: Compressing objects: 100% (42/42)�[K
remote: Compressing objects: 100% (42/42), done.�[K
remote: Total 49 (delta 47), reused 8 (delta 7), pack-reused 0�[K
From https://github.com/python/cpython
 * branch            main       -> FETCH_HEAD
Note: checking out '6dfb8fe0236718e9afc8136ff2b58dcfbc182022'.

You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by performing another checkout.

If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -b with the checkout command again. Example:

  git checkout -b new_branch_name

HEAD is now at 6dfb8fe... gh-110481: Implement biased reference counting (gh-110764)
Switched to and reset branch 'main'

find: ‘build’: No such file or directory
find: ‘build’: No such file or directory
find: ‘build’: No such file or directory
find: ‘build’: No such file or directory
make[2]: [clean-retain-profile] Error 1 (ignored)
In file included from ./Include/Python.h:52:0,
                 from ./Programs/python.c:3:
./Include/object.h: In function ‘_Py_ThreadId’:
./Include/object.h:258:5: error: #error "define _Py_ThreadId for this platform"
   # error "define _Py_ThreadId for this platform"
     ^
In file included from ./Include/Python.h:52:0,
                 from Parser/pegen.c:1:
./Include/object.h: In function ‘_Py_ThreadId’:
./Include/object.h:258:5: error: #error "define _Py_ThreadId for this platform"
   # error "define _Py_ThreadId for this platform"
     ^
In file included from ./Include/Python.h:52:0,
                 from Parser/token.c:3:
./Include/object.h: In function ‘_Py_ThreadId’:
./Include/object.h:258:5: error: #error "define _Py_ThreadId for this platform"
   # error "define _Py_ThreadId for this platform"
     ^
In file included from ./Include/Python.h:52:0,
                 from Parser/pegen_errors.c:1:
./Include/object.h: In function ‘_Py_ThreadId’:
./Include/object.h:258:5: error: #error "define _Py_ThreadId for this platform"
   # error "define _Py_ThreadId for this platform"
     ^
make[2]: *** [Programs/python.o] Error 1
make[2]: *** Waiting for unfinished jobs....
make[2]: *** [Parser/pegen_errors.o] Error 1
make[2]: *** [Parser/token.o] Error 1
In file included from ./Include/Python.h:52:0,
                 from Parser/pegen.h:4,
                 from Parser/parser.c:2:
./Include/object.h: In function ‘_Py_ThreadId’:
./Include/object.h:258:5: error: #error "define _Py_ThreadId for this platform"
   # error "define _Py_ThreadId for this platform"
     ^
In file included from ./Include/Python.h:52:0,
                 from Parser/action_helpers.c:1:
./Include/object.h: In function ‘_Py_ThreadId’:
./Include/object.h:258:5: error: #error "define _Py_ThreadId for this platform"
   # error "define _Py_ThreadId for this platform"
     ^
make[2]: *** [Parser/pegen.o] Error 1
make[2]: *** [Parser/action_helpers.o] Error 1
make[2]: *** [Parser/parser.o] Error 1
make[1]: *** [profile-gen-stamp] Error 2
make: *** [profile-run-stamp] Error 2

find: ‘build’: No such file or directory
find: ‘build’: No such file or directory
find: ‘build’: No such file or directory
find: ‘build’: No such file or directory
make: [clean-retain-profile] Error 1 (ignored)
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
@corona10

corona10 commented Oct 30, 2023

Copy link
Copy Markdown
Member

For the build bot issue, please follow up: #111503
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
@ericsnowcurrently ericsnowcurrently mentioned this pull request Oct 30, 2023
gh-109515: Allow a large number of frozen modules on Windows #109516
Merged
@vstinner

vstinner commented Oct 30, 2023

Copy link
Copy Markdown
Member

For the limited C API, I wrote a first change: PR #111505.
colesbury reacted with thumbs up emoji
All reactions
  • 👍 1 reaction

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
@colesbury colesbury mentioned this pull request Oct 30, 2023
--disable-gil builds are not compatible with the limited API #111506
Closed
@vstinner vstinner mentioned this pull request Oct 30, 2023
gh-110481: Fix _Py_ThreadId for non-free-threaded mode #111503
Merged
@vstinner

vstinner commented Oct 30, 2023

Copy link
Copy Markdown
Member

Congrats @colesbury, that's a massive change :-)
h-vetinari, erlend-aasland, and rachtsingh reacted with hooray emoji
All reactions
  • 🎉 3 reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
dimaqq
dimaqq reviewed Oct 31, 2023
View reviewed changes
Comment thread Include/object.h
Comment on lines +251 to +257
#elif defined(__arm__)
__asm__ ("mrc p15, 0, %0, c13, c0, 3\nbic %0, %0, #3" : "=r" (tid));
#elif defined(__aarch64__) && defined(__APPLE__)
__asm__ ("mrs %0, tpidrro_el0" : "=r" (tid));
#elif defined(__aarch64__)
__asm__ ("mrs %0, tpidr_el0" : "=r" (tid));
#else

@dimaqq dimaqq Oct 31, 2023

Copy link
Copy Markdown
Contributor

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Caveat: I'm not an expert.
In case of x86/x86-64 above, the tid values has to be copied to a regular register for comparison.
In case of ARM32, the comparison could be made directly, were it not for the 2 lsb.
In case of ARM64, the comparison could be made directly.

Leaving aside the less common case of stamping the biased reference count block, and focusing on the more common case of comparing the current tid with object owner tid:

Does the compiler optimise this move away?
Does it perhaps not matter, as move is fast compared to loading the object's value from ram or cache?
Or would a custom comparison macro make sense here?

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
All reactions

@colesbury colesbury Oct 31, 2023

Copy link
Copy Markdown
Contributor Author

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is backwards: on x86/x86-64 you can do a comparison without copying tid to a register; you can't on ARM.

On x86, many instructions, such as cmp, can operate on memory addresses, including addresses with fs: or gs: prefixes. It doesn't matter much here because because the object's owner tid also requires a memory load. So either one or the other needs to be moved to a temporary registers and it's more convenient to move the current tid to a temporary register because it may be re-used across multiple reference count operations.

On ARM you can't do a comparison with a system or coprocessor register. You need to move it to a gpr first.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
dimaqq, erlend-aasland, and corona10 reacted with thumbs up emoji
All reactions
  • 👍 3 reactions
@mdboom

mdboom commented Oct 31, 2023 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Contributor

I ran this PR on the Faster CPython team's benchmarking infrastructure, and its effect is basically immeasurable, 1.00x slower with a reliability of 95%: https://github.com/faster-cpython/benchmarking-public/blob/main/results/bm-20231030-3.13.0a1%2B-6dfb8fe/bm-20231030-linux-x86_64-python-6dfb8fe0236718e9afc8-3.13.0a1%2B-6dfb8fe-vs-base.md

EDIT: As @ericsnowcurrently pointed out, this is with a default build, so is not terribly interesting. I'll rerun with --disable-gil and report back.
stonebig reacted with heart emoji
All reactions
  • ❤️ 1 reaction

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
@vstinner

vstinner commented Oct 31, 2023

Copy link
Copy Markdown
Member

EDIT: As @ericsnowcurrently pointed out, this is with a default build, so is not terribly interesting.

This PR only impacts Python if Python is configured with --disable-gil.
corona10 reacted with thumbs up emoji stonebig reacted with heart emoji
All reactions
  • 👍 1 reaction
  • ❤️ 1 reaction

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
@colesbury
colesbury deleted the biased_reference_counting branch October 31, 2023 18:58
@mdboom

mdboom commented Oct 31, 2023

Copy link
Copy Markdown
Contributor

And the correct benchmarking results are in. With --disable-gil (and some hackery to get pyperf to build psutil from source), it's 1.06x slower at the 99th percentile on the HPT metric. Details here: https://github.com/faster-cpython/benchmarking-public/blob/main/results/bm-20231031-3.13.0a1%2B-bbb758f/bm-20231031-linux-x86_64-mdboom-measure_biased_ref_c-3.13.0a1%2B-bbb758f-vs-base.md
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
@mdboom

mdboom commented Oct 31, 2023

Copy link
Copy Markdown
Contributor

@colesbury: An interesting thing the benchmarking revealed is the coverage benchmark significantly regressed, but that may be due to failing to install a C extension and falling back to a pure Python codepath.
ericsnowcurrently, colesbury, and corona10 reacted with thumbs up emoji
All reactions
  • 👍 3 reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
@colesbury

colesbury commented Nov 2, 2023

Copy link
Copy Markdown
Contributor Author

@mdboom, Yeah the coverage C extension fails to install due to pypa/packaging#727. That's fixed in the packaging main branch, but still needs a number of steps:

  1. A packaging release
  2. Vendor the new packaging release in pip
  3. A pip release
  4. New bundled copy of pip in cpython's ensurepip.
vstinner reacted with thumbs up emoji
All reactions
  • 👍 1 reaction

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
FullteaR pushed a commit to FullteaR/cpython that referenced this pull request Nov 3, 2023
@colesbury @FullteaR
pythongh-110481: Implement biased reference counting (pythongh-110764)
6a480d8
@corona10 corona10 mentioned this pull request Nov 9, 2023
--disable-gil builds install incompatible stable ABI extensions #111870
Closed
@ericsnowcurrently ericsnowcurrently mentioned this pull request Feb 1, 2024
gh-110481: Implement inter-thread queue for biased reference counting #114824
Merged
aisk pushed a commit to aisk/cpython that referenced this pull request Feb 11, 2024
@colesbury @aisk
pythongh-110481: Implement biased reference counting (pythongh-110764)
f1c2b78
Glyphack pushed a commit to Glyphack/cpython that referenced this pull request Sep 2, 2024
@colesbury @Glyphack
pythongh-110481: Implement biased reference counting (pythongh-110764)
2295beb
@encukou encukou mentioned this pull request Dec 16, 2025
gh-140550: Update xxlimited with 3.15 limited API & PEP 697 #142827
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@vstinner vstinner vstinner requested changes
@markshannon markshannon markshannon left review comments
@erlend-aasland erlend-aasland erlend-aasland left review comments
@corona10 corona10 corona10 approved these changes
@ericsnowcurrently ericsnowcurrently Awaiting requested review from ericsnowcurrently ericsnowcurrently is a code owner
@ncoghlan ncoghlan Awaiting requested review from ncoghlan ncoghlan is a code owner
@warsaw warsaw Awaiting requested review from warsaw warsaw is a code owner
@tiran tiran Awaiting requested review from tiran
+1 more reviewer
@dimaqq dimaqq dimaqq left review comments
Reviewers whose approvals may not affect merge requirements

Uh oh!

There was an error while loading. Please reload this page.

Assignees

No one assigned

Labels

topic-free-threading

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Uh oh!

There was an error while loading. Please reload this page.

8 participants

@colesbury @corona10 @markshannon @bedevere-bot @vstinner @mdboom @dimaqq @erlend-aasland
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
You are visitor 01402503 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.