GeezerVison

The World Wide Web, restored to 800 pixels of glory.

GeezerVison is showing github.com — gh-104372: Cleanup _posixsubprocess `make_inheritable` for async signal safety by gpshead · Pull Request #104518 · python/cpython

Conversation

gpshead commented May 16, 2023 •

edited by bedevere-bot

Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link

Copy Markdown

Member

We move all of the Python C API calls into the parent process up front instead of doing PyLong_AsLong and PyErr_Occurred and PyTuple_GET* from the post-fork/vfork child process.

Much of this was long overdue. We shouldn't have been using PyTuple and PyLong APIs within all of the lower level functions anyways.

Issue: subprocess can block all Python threads when using vfork() until the child process exec() succeeds or fails. #104372

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions


          pythongh-104372: Cleanup make_inheritable for async signal safety.

adae5af

We move all of the Python C API calls into the parent process up front
instead of doing PyLong_AsLong and PyErr_Occurred and PyTuple_GET from
the post-fork/vfork child process.

gpshead self-assigned this

May 16, 2023

bedevere-bot added the awaiting core review label

May 16, 2023

bedevere-bot mentioned this pull request

May 16, 2023

subprocess can block all Python threads when using vfork() until the child process exec() succeeds or fails. #104372

Closed

gpshead commented May 16, 2023

Copy link

Copy Markdown

Member Author

This is a code cleanup that #104515 mentions as desirable so that a nicer PR can be made for the larger issue.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

gpshead added 🔨 test-with-buildbots

Test PR w/ buildbots; report in status section

🔨 test-with-refleak-buildbots Test PR w/ refleak buildbots; report in status section labels

May 16, 2023

bedevere-bot commented May 16, 2023

Copy link

Copy Markdown

🤖 New build scheduled with the buildbot fleet by @gpshead for commit adae5af 🤖

If you want to schedule another build, you need to add the 🔨 test-with-buildbots label again.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

bedevere-bot commented May 16, 2023

Copy link

Copy Markdown

🤖 New build scheduled with the buildbot fleet by @gpshead for commit adae5af 🤖

If you want to schedule another build, you need to add the 🔨 test-with-refleak-buildbots label again.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

bedevere-bot removed 🔨 test-with-refleak-buildbots

Test PR w/ refleak buildbots; report in status section

🔨 test-with-buildbots Test PR w/ buildbots; report in status section labels

May 16, 2023

Eclips4 commented May 16, 2023 •

edited

Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link

Copy Markdown

Member

Fails in test_gzip, test_httpservers, test_nntplib, test_tarfile, test_xmlrpc related to isolation of io module.
(see #104510 for more details)

gpshead reacted with hooray emoji

All reactions

🎉 1 reaction

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.


          Merge branch 'main' into subprocess-child_exec-cleanup

bfc9d9e

gpshead added the 🔨 test-with-buildbots Test PR w/ buildbots; report in status section label

May 16, 2023

bedevere-bot commented May 16, 2023

Copy link

Copy Markdown

🤖 New build scheduled with the buildbot fleet by @gpshead for commit bfc9d9e 🤖

If you want to schedule another build, you need to add the 🔨 test-with-buildbots label again.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

bedevere-bot removed the 🔨 test-with-buildbots Test PR w/ buildbots; report in status section label

May 16, 2023

gpshead added 🔨 test-with-buildbots

Test PR w/ buildbots; report in status section

🔨 test-with-refleak-buildbots Test PR w/ refleak buildbots; report in status section labels

May 16, 2023

bedevere-bot commented May 16, 2023

Copy link

Copy Markdown

🤖 New build scheduled with the buildbot fleet by @gpshead for commit bfc9d9e 🤖

If you want to schedule another build, you need to add the 🔨 test-with-refleak-buildbots label again.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

bedevere-bot removed the 🔨 test-with-refleak-buildbots Test PR w/ refleak buildbots; report in status section label

May 16, 2023

bedevere-bot commented May 16, 2023

Copy link

Copy Markdown

🤖 New build scheduled with the buildbot fleet by @gpshead for commit bfc9d9e 🤖

If you want to schedule another build, you need to add the 🔨 test-with-buildbots label again.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

bedevere-bot removed the 🔨 test-with-buildbots Test PR w/ buildbots; report in status section label

May 16, 2023


          Merge branch 'main' into subprocess-child_exec-cleanup

e073dde

gpshead commented May 17, 2023

Copy link

Copy Markdown

Member Author

!buildbot root

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

bedevere-bot commented May 17, 2023

Copy link

Copy Markdown

🤖 New build scheduled with the buildbot fleet by @gpshead for commit e073dde 🤖

The command will test the builders whose names match following regular expression: root

The builders matched are:

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

gpshead commented May 17, 2023

Copy link

Copy Markdown

Member Author

!buildbot Debian root

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

bedevere-bot commented May 17, 2023

Copy link

Copy Markdown

🤖 New build scheduled with the buildbot fleet by @gpshead for commit e073dde 🤖

The command will test the builders whose names match following regular expression: Debian root

The builders matched are:

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

gpshead commented May 17, 2023

Copy link

Copy Markdown

Member Author

!buildbot AMD64 Debian root

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

bedevere-bot commented May 17, 2023

Copy link

Copy Markdown

🤖 New build scheduled with the buildbot fleet by @gpshead for commit e073dde 🤖

The command will test the builders whose names match following regular expression: AMD64 Debian root

The builders matched are:

AMD64 Debian root PR

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.


          📜🤖 Added by blurb_it.

cd3108c

gpshead commented May 17, 2023

Copy link

Copy Markdown

Member Author

!buildbot AMD64 Debian root

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

bedevere-bot commented May 17, 2023

Copy link

Copy Markdown

🤖 New build scheduled with the buildbot fleet by @gpshead for commit cd3108c 🤖

The command will test the builders whose names match following regular expression: AMD64 Debian root

The builders matched are:

AMD64 Debian root PR

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

gpshead merged commit c649df6 into python:main

May 17, 2023

bedevere-bot removed the awaiting core review label

May 17, 2023

gpshead added the needs backport to 3.11 only security fixes label

May 17, 2023

miss-islington commented May 17, 2023

Copy link

Copy Markdown

Contributor

Thanks @gpshead for the PR 🌮🎉.. I'm working now to backport this PR to: 3.11.
🐍🍒⛏🤖 I'm not a witch! I'm not a witch!

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

miss-islington commented May 17, 2023

Copy link

Copy Markdown

Contributor

Sorry @gpshead, I had trouble checking out the 3.11 backport branch.
Please retry by removing and re-adding the "needs backport to 3.11" label.
Alternatively, you can backport using cherry_picker on the command line.
cherry_picker c649df63e0d052044a4660101d5769ff46ae9234 3.11

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

bedevere-bot commented May 17, 2023

Copy link

Copy Markdown

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot s390x Fedora 3.x has failed when building commit c649df6.

What do you need to do:

Don't panic.
Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
Go to the page of the buildbot that failed (https://buildbot.python.org/all/#builders/223/builds/3795) and take a look at the build logs.
Check if the failure is related to this commit (c649df6) or if it is a false positive.
If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/all/#builders/223/builds/3795

Failed tests:

test_tools

Failed subtests:

test_freeze_simple_script - test.test_tools.test_freeze.TestFreeze.test_freeze_simple_script

Summary of the results of the build (if available):

== Tests result: FAILURE then FAILURE ==

444 tests OK.

10 slowest tests:

test_concurrent_futures: 2 min 46 sec
test_multiprocessing_spawn: 2 min 14 sec
test_math: 2 min 12 sec
test_tools: 2 min
test_multiprocessing_forkserver: 1 min 25 sec
test_multiprocessing_fork: 1 min 16 sec
test_tokenize: 1 min 13 sec
test_signal: 1 min 3 sec
test_gdb: 1 min 2 sec
test_socket: 52.4 sec

1 test failed:
test_tools

20 tests skipped:
test.test_asyncio.test_windows_events
test.test_asyncio.test_windows_utils test_devpoll test_ioctl
test_kqueue test_launcher test_msilib test_nis test_peg_generator
test_perf_profiler test_readline test_startfile test_tix
test_tkinter test_ttk test_winconsoleio test_winreg test_winsound
test_wmi test_zipfile64

1 re-run test:
test_tools

Total duration: 7 min 49 sec

Click to see traceback logs

Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/tmp/test_python_3ag2_1jl/tmpu9ksrovt/cpython/Lib/sysconfig.py", line 740, in get_config_var
    return get_config_vars().get(name)
           ^^^^^^^^^^^^^^^^^
  File "/tmp/test_python_3ag2_1jl/tmpu9ksrovt/cpython/Lib/sysconfig.py", line 723, in get_config_vars
    _init_config_vars()
  File "/tmp/test_python_3ag2_1jl/tmpu9ksrovt/cpython/Lib/sysconfig.py", line 670, in _init_config_vars
    _init_posix(_CONFIG_VARS)
  File "/tmp/test_python_3ag2_1jl/tmpu9ksrovt/cpython/Lib/sysconfig.py", line 536, in _init_posix
    _temp = __import__(name, globals(), locals(), ['build_time_vars'], 0)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ModuleNotFoundError: No module named '_sysconfigdata_d_linux_s390x-linux-gnu'


Traceback (most recent call last):
  File "/home/dje/cpython-buildarea/3.x.edelsohn-fedora-z/build/Lib/test/test_tools/test_freeze.py", line 27, in test_freeze_simple_script
    outdir, scriptfile, python = helper.prepare(script, outdir)
                                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/dje/cpython-buildarea/3.x.edelsohn-fedora-z/build/Tools/freeze/test/freeze.py", line 184, in prepare
    _run_quiet([MAKE, '-C', builddir, 'install'])
  File "/home/dje/cpython-buildarea/3.x.edelsohn-fedora-z/build/Tools/freeze/test/freeze.py", line 25, in _run_quiet
    return subprocess.run(
           ^^^^^^^^^^^^^^^
  File "/home/dje/cpython-buildarea/3.x.edelsohn-fedora-z/build/Lib/subprocess.py", line 571, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['/usr/bin/make', '-C', '/tmp/test_python_3ag2_1jl/tmpu9ksrovt/python-build', 'install']' returned non-zero exit status 2.


Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/tmp/tmpgtasm7e9/cpython/Lib/sysconfig.py", line 740, in get_config_var
    return get_config_vars().get(name)
           ^^^^^^^^^^^^^^^^^
  File "/tmp/tmpgtasm7e9/cpython/Lib/sysconfig.py", line 723, in get_config_vars
    _init_config_vars()
  File "/tmp/tmpgtasm7e9/cpython/Lib/sysconfig.py", line 670, in _init_config_vars
    _init_posix(_CONFIG_VARS)
  File "/tmp/tmpgtasm7e9/cpython/Lib/sysconfig.py", line 536, in _init_posix
    _temp = __import__(name, globals(), locals(), ['build_time_vars'], 0)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ModuleNotFoundError: No module named '_sysconfigdata_d_linux_s390x-linux-gnu'


Traceback (most recent call last):
  File "/home/dje/cpython-buildarea/3.x.edelsohn-fedora-z/build/Lib/test/test_tools/test_freeze.py", line 27, in test_freeze_simple_script
    outdir, scriptfile, python = helper.prepare(script, outdir)
                                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/dje/cpython-buildarea/3.x.edelsohn-fedora-z/build/Tools/freeze/test/freeze.py", line 184, in prepare
    _run_quiet([MAKE, '-C', builddir, 'install'])
  File "/home/dje/cpython-buildarea/3.x.edelsohn-fedora-z/build/Tools/freeze/test/freeze.py", line 25, in _run_quiet
    return subprocess.run(
           ^^^^^^^^^^^^^^^
  File "/home/dje/cpython-buildarea/3.x.edelsohn-fedora-z/build/Lib/subprocess.py", line 571, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['/usr/bin/make', '-C', '/tmp/tmpgtasm7e9/python-build', 'install']' returned non-zero exit status 2.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

bedevere-bot commented May 17, 2023

Copy link

Copy Markdown

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot s390x Fedora Clang 3.x has failed when building commit c649df6.

What do you need to do:

Don't panic.
Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
Go to the page of the buildbot that failed (https://buildbot.python.org/all/#builders/3/builds/3921) and take a look at the build logs.
Check if the failure is related to this commit (c649df6) or if it is a false positive.
If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/all/#builders/3/builds/3921

Failed tests:

test_tools

Failed subtests:

test_freeze_simple_script - test.test_tools.test_freeze.TestFreeze.test_freeze_simple_script
test_pathnames - test.test_tarfile.GzipWriteTest.test_pathnames

Summary of the results of the build (if available):

== Tests result: FAILURE then FAILURE ==

444 tests OK.

10 slowest tests:

test_tools: 2 min 32 sec
test_concurrent_futures: 2 min 15 sec
test_multiprocessing_spawn: 2 min 12 sec
test_multiprocessing_forkserver: 1 min 26 sec
test_math: 1 min 18 sec
test_signal: 1 min 12 sec
test_multiprocessing_fork: 1 min 10 sec
test_capi: 49.6 sec
test_socket: 47.3 sec
test_gdb: 43.2 sec

1 test failed:
test_tools

2 re-run tests:
test_tarfile test_tools

Total duration: 8 min 26 sec

Click to see traceback logs

Traceback (most recent call last):
  File "/home/dje/cpython-buildarea/3.x.edelsohn-fedora-z.clang/build/Lib/test/test_tarfile.py", line 1469, in test_pathnames
    self._test_pathname(os.path.join("..", "foo"))
  File "/home/dje/cpython-buildarea/3.x.edelsohn-fedora-z.clang/build/Lib/test/test_tarfile.py", line 1409, in _test_pathname
    os_helper.create_empty_file(foo)
  File "/home/dje/cpython-buildarea/3.x.edelsohn-fedora-z.clang/build/Lib/test/support/os_helper.py", line 538, in create_empty_file
    fd = os.open(filename, os.O_WRONLY | os.O_CREAT | os.O_TRUNC)
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
OSError: [Errno 28] No space left on device: '/home/dje/cpython-buildarea/3.x.edelsohn-fedora-z.clang/build/build/test_python_1713233æ/test_python_worker_1734541æ/@test_1734541_tmpæ-tardir/foo'


Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/tmp/test_python_n6t5sdkl/tmpxtrewzr7/cpython/Lib/sysconfig.py", line 740, in get_config_var
    return get_config_vars().get(name)
           ^^^^^^^^^^^^^^^^^
  File "/tmp/test_python_n6t5sdkl/tmpxtrewzr7/cpython/Lib/sysconfig.py", line 723, in get_config_vars
    _init_config_vars()
  File "/tmp/test_python_n6t5sdkl/tmpxtrewzr7/cpython/Lib/sysconfig.py", line 670, in _init_config_vars
    _init_posix(_CONFIG_VARS)
  File "/tmp/test_python_n6t5sdkl/tmpxtrewzr7/cpython/Lib/sysconfig.py", line 536, in _init_posix
    _temp = __import__(name, globals(), locals(), ['build_time_vars'], 0)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ModuleNotFoundError: No module named '_sysconfigdata_d_linux_s390x-linux-gnu'


Traceback (most recent call last):
  File "/home/dje/cpython-buildarea/3.x.edelsohn-fedora-z.clang/build/Lib/test/test_tools/test_freeze.py", line 27, in test_freeze_simple_script
    outdir, scriptfile, python = helper.prepare(script, outdir)
                                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/dje/cpython-buildarea/3.x.edelsohn-fedora-z.clang/build/Tools/freeze/test/freeze.py", line 184, in prepare
    _run_quiet([MAKE, '-C', builddir, 'install'])
  File "/home/dje/cpython-buildarea/3.x.edelsohn-fedora-z.clang/build/Tools/freeze/test/freeze.py", line 25, in _run_quiet
    return subprocess.run(
           ^^^^^^^^^^^^^^^
  File "/home/dje/cpython-buildarea/3.x.edelsohn-fedora-z.clang/build/Lib/subprocess.py", line 571, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['/usr/bin/make', '-C', '/tmp/tmp_be2fs30/python-build', 'install']' returned non-zero exit status 2.


Traceback (most recent call last):
  File "/home/dje/cpython-buildarea/3.x.edelsohn-fedora-z.clang/build/Lib/test/test_tools/test_freeze.py", line 27, in test_freeze_simple_script
    outdir, scriptfile, python = helper.prepare(script, outdir)
                                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/dje/cpython-buildarea/3.x.edelsohn-fedora-z.clang/build/Tools/freeze/test/freeze.py", line 184, in prepare
    _run_quiet([MAKE, '-C', builddir, 'install'])
  File "/home/dje/cpython-buildarea/3.x.edelsohn-fedora-z.clang/build/Tools/freeze/test/freeze.py", line 25, in _run_quiet
    return subprocess.run(
           ^^^^^^^^^^^^^^^
  File "/home/dje/cpython-buildarea/3.x.edelsohn-fedora-z.clang/build/Lib/subprocess.py", line 571, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['/usr/bin/make', '-C', '/tmp/test_python_n6t5sdkl/tmpxtrewzr7/python-build', 'install']' returned non-zero exit status 2.


Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/tmp/tmp_be2fs30/cpython/Lib/sysconfig.py", line 740, in get_config_var
    return get_config_vars().get(name)
           ^^^^^^^^^^^^^^^^^
  File "/tmp/tmp_be2fs30/cpython/Lib/sysconfig.py", line 723, in get_config_vars
    _init_config_vars()
  File "/tmp/tmp_be2fs30/cpython/Lib/sysconfig.py", line 670, in _init_config_vars
    _init_posix(_CONFIG_VARS)
  File "/tmp/tmp_be2fs30/cpython/Lib/sysconfig.py", line 536, in _init_posix
    _temp = __import__(name, globals(), locals(), ['build_time_vars'], 0)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ModuleNotFoundError: No module named '_sysconfigdata_d_linux_s390x-linux-gnu'

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

bedevere-bot commented May 17, 2023

Copy link

Copy Markdown

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot AMD64 Ubuntu Shared 3.x has failed when building commit c649df6.

What do you need to do:

Don't panic.
Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
Go to the page of the buildbot that failed (https://buildbot.python.org/all/#builders/506/builds/4791) and take a look at the build logs.
Check if the failure is related to this commit (c649df6) or if it is a false positive.
If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/all/#builders/506/builds/4791

Failed tests:

test__xxsubinterpreters

Summary of the results of the build (if available):

Click to see traceback logs

remote: Enumerating objects: 9, done.        
remote: Counting objects:  11% (1/9)        
remote: Counting objects:  22% (2/9)        
remote: Counting objects:  33% (3/9)        
remote: Counting objects:  44% (4/9)        
remote: Counting objects:  55% (5/9)        
remote: Counting objects:  66% (6/9)        
remote: Counting objects:  77% (7/9)        
remote: Counting objects:  88% (8/9)        
remote: Counting objects: 100% (9/9)        
remote: Counting objects: 100% (9/9), done.        
remote: Compressing objects:  11% (1/9)        
remote: Compressing objects:  22% (2/9)        
remote: Compressing objects:  33% (3/9)        
remote: Compressing objects:  44% (4/9)        
remote: Compressing objects:  55% (5/9)        
remote: Compressing objects:  66% (6/9)        
remote: Compressing objects:  77% (7/9)        
remote: Compressing objects:  88% (8/9)        
remote: Compressing objects: 100% (9/9)        
remote: Compressing objects: 100% (9/9), done.        
remote: Total 9 (delta 0), reused 1 (delta 0), pack-reused 0        
From https://github.com/python/cpython
 * branch                  main       -> FETCH_HEAD
Note: switching to 'c649df63e0d052044a4660101d5769ff46ae9234'.

You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by switching back to a branch.

If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -c with the switch command. Example:

  git switch -c <new-branch-name>

Or undo this operation with:

  git switch -

Turn off this advice by setting config variable advice.detachedHead to false

HEAD is now at c649df63e0 gh-104372: Cleanup _posixsubprocess `make_inheritable` for async signal safety and no GIL requirement (#104518)
Switched to and reset branch 'main'

configure: WARNING: pkg-config is missing. Some dependencies may not be detected correctly.

Fatal Python error: drop_gil: PyMUTEX_LOCK(gil->switch_mutex) failed
Python runtime state: initialized

Thread 0x00007fa574af5b80 (most recent call first):
  File "/srv/buildbot/buildarea/3.x.bolen-ubuntu/build/Lib/test/test__xxsubinterpreters.py", line 76 in clean_up_interpreters
  File "/srv/buildbot/buildarea/3.x.bolen-ubuntu/build/Lib/test/test__xxsubinterpreters.py", line 88 in tearDown
  File "/srv/buildbot/buildarea/3.x.bolen-ubuntu/build/Lib/unittest/case.py", line 594 in _callTearDown
  File "/srv/buildbot/buildarea/3.x.bolen-ubuntu/build/Lib/unittest/case.py", line 637 in run
  File "/srv/buildbot/buildarea/3.x.bolen-ubuntu/build/Lib/unittest/case.py", line 690 in __call__
  File "/srv/buildbot/buildarea/3.x.bolen-ubuntu/build/Lib/unittest/suite.py", line 122 in run
  File "/srv/buildbot/buildarea/3.x.bolen-ubuntu/build/Lib/unittest/suite.py", line 84 in __call__
  File "/srv/buildbot/buildarea/3.x.bolen-ubuntu/build/Lib/unittest/suite.py", line 122 in run
  File "/srv/buildbot/buildarea/3.x.bolen-ubuntu/build/Lib/unittest/suite.py", line 84 in __call__
  File "/srv/buildbot/buildarea/3.x.bolen-ubuntu/build/Lib/unittest/suite.py", line 122 in run
  File "/srv/buildbot/buildarea/3.x.bolen-ubuntu/build/Lib/unittest/suite.py", line 84 in __call__
  File "/srv/buildbot/buildarea/3.x.bolen-ubuntu/build/Lib/unittest/runner.py", line 240 in run
  File "/srv/buildbot/buildarea/3.x.bolen-ubuntu/build/Lib/test/support/__init__.py", line 1115 in _run_suite
  File "/srv/buildbot/buildarea/3.x.bolen-ubuntu/build/Lib/test/support/__init__.py", line 1241 in run_unittest
  File "/srv/buildbot/buildarea/3.x.bolen-ubuntu/build/Lib/test/libregrtest/runtest.py", line 294 in _test_module
  File "/srv/buildbot/buildarea/3.x.bolen-ubuntu/build/Lib/test/libregrtest/runtest.py", line 330 in _runtest_inner2
  File "/srv/buildbot/buildarea/3.x.bolen-ubuntu/build/Lib/test/libregrtest/runtest.py", line 373 in _runtest_inner
  File "/srv/buildbot/buildarea/3.x.bolen-ubuntu/build/Lib/test/libregrtest/runtest.py", line 248 in _runtest
  File "/srv/buildbot/buildarea/3.x.bolen-ubuntu/build/Lib/test/libregrtest/runtest.py", line 278 in runtest
  File "/srv/buildbot/buildarea/3.x.bolen-ubuntu/build/Lib/test/libregrtest/main.py", line 365 in rerun_failed_tests
  File "/srv/buildbot/buildarea/3.x.bolen-ubuntu/build/Lib/test/libregrtest/main.py", line 783 in _main
  File "/srv/buildbot/buildarea/3.x.bolen-ubuntu/build/Lib/test/libregrtest/main.py", line 738 in main
  File "/srv/buildbot/buildarea/3.x.bolen-ubuntu/build/Lib/test/libregrtest/main.py", line 802 in main
  File "/srv/buildbot/buildarea/3.x.bolen-ubuntu/build/Lib/test/__main__.py", line 2 in <module>
  File "/srv/buildbot/buildarea/3.x.bolen-ubuntu/build/Lib/runpy.py", line 88 in _run_code
  File "/srv/buildbot/buildarea/3.x.bolen-ubuntu/build/Lib/runpy.py", line 198 in _run_module_as_main
Fatal Python error: Aborted

Thread 0x00007fa574af5b80 (most recent call first):
  File "/srv/buildbot/buildarea/3.x.bolen-ubuntu/build/Lib/test/test__xxsubinterpreters.py", line 76 in clean_up_interpreters
  File "/srv/buildbot/buildarea/3.x.bolen-ubuntu/build/Lib/test/test__xxsubinterpreters.py", line 88 in tearDown
  File "/srv/buildbot/buildarea/3.x.bolen-ubuntu/build/Lib/unittest/case.py", line 594 in _callTearDown
  File "/srv/buildbot/buildarea/3.x.bolen-ubuntu/build/Lib/unittest/case.py", line 637 in run
  File "/srv/buildbot/buildarea/3.x.bolen-ubuntu/build/Lib/unittest/case.py", line 690 in __call__
  File "/srv/buildbot/buildarea/3.x.bolen-ubuntu/build/Lib/unittest/suite.py", line 122 in run
  File "/srv/buildbot/buildarea/3.x.bolen-ubuntu/build/Lib/unittest/suite.py", line 84 in __call__
  File "/srv/buildbot/buildarea/3.x.bolen-ubuntu/build/Lib/unittest/suite.py", line 122 in run
  File "/srv/buildbot/buildarea/3.x.bolen-ubuntu/build/Lib/unittest/suite.py", line 84 in __call__
  File "/srv/buildbot/buildarea/3.x.bolen-ubuntu/build/Lib/unittest/suite.py", line 122 in run
  File "/srv/buildbot/buildarea/3.x.bolen-ubuntu/build/Lib/unittest/suite.py", line 84 in __call__
  File "/srv/buildbot/buildarea/3.x.bolen-ubuntu/build/Lib/unittest/runner.py", line 240 in run
  File "/srv/buildbot/buildarea/3.x.bolen-ubuntu/build/Lib/test/support/__init__.py", line 1115 in _run_suite
  File "/srv/buildbot/buildarea/3.x.bolen-ubuntu/build/Lib/test/support/__init__.py", line 1241 in run_unittest
  File "/srv/buildbot/buildarea/3.x.bolen-ubuntu/build/Lib/test/libregrtest/runtest.py", line 294 in _test_module
  File "/srv/buildbot/buildarea/3.x.bolen-ubuntu/build/Lib/test/libregrtest/runtest.py", line 330 in _runtest_inner2
  File "/srv/buildbot/buildarea/3.x.bolen-ubuntu/build/Lib/test/libregrtest/runtest.py", line 373 in _runtest_inner
  File "/srv/buildbot/buildarea/3.x.bolen-ubuntu/build/Lib/test/libregrtest/runtest.py", line 248 in _runtest
  File "/srv/buildbot/buildarea/3.x.bolen-ubuntu/build/Lib/test/libregrtest/runtest.py", line 278 in runtest
  File "/srv/buildbot/buildarea/3.x.bolen-ubuntu/build/Lib/test/libregrtest/main.py", line 365 in rerun_failed_tests
  File "/srv/buildbot/buildarea/3.x.bolen-ubuntu/build/Lib/test/libregrtest/main.py", line 783 in _main
  File "/srv/buildbot/buildarea/3.x.bolen-ubuntu/build/Lib/test/libregrtest/main.py", line 738 in main
  File "/srv/buildbot/buildarea/3.x.bolen-ubuntu/build/Lib/test/libregrtest/main.py", line 802 in main
  File "/srv/buildbot/buildarea/3.x.bolen-ubuntu/build/Lib/test/__main__.py", line 2 in <module>
  File "/srv/buildbot/buildarea/3.x.bolen-ubuntu/build/Lib/runpy.py", line 88 in _run_code
  File "/srv/buildbot/buildarea/3.x.bolen-ubuntu/build/Lib/runpy.py", line 198 in _run_module_as_main

Extension modules: _testcapi, _xxsubinterpreters (total: 2)
Aborted (core dumped)
make: *** [Makefile:1977: buildbottest] Error 134

Cannot open file '/srv/buildbot/buildarea/3.x.bolen-ubuntu/build/test-results.xml' for upload

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

JelleZijlstra pushed a commit to JelleZijlstra/cpython that referenced this pull request

May 18, 2023


          pythongh-104372: Cleanup _posixsubprocess make_inheritable for asyn…

4b2bf16

…c signal safety and no GIL requirement (python#104518)

Move all of the Python C API calls into the parent process up front
instead of doing PyLong_AsLong and PyErr_Occurred and PyTuple_GET from
the post-fork/vfork child process.

Much of this was long overdue. We shouldn't have been using PyTuple and
PyLong APIs within all of these low level functions anyways.

gpshead deleted the subprocess-child_exec-cleanup branch

May 18, 2023 03:35

gpshead mentioned this pull request

May 20, 2023

gh-104372: Use non-Raw malloc for c_fds_to_keep in _posixsubprocess #104697

Merged

gpshead removed the needs backport to 3.11 only security fixes label

May 20, 2023

gpshead added a commit to gpshead/cpython that referenced this pull request

May 23, 2023


          [3.11] pythongh-104372: Cleanup _posixsubprocess make_inheritable f…

2c04e99

…or async signal safety and no GIL requirement (pythonGH-104518)

Move all of the Python C API calls into the parent process up front
instead of doing PyLong_AsLong and PyErr_Occurred and PyTuple_GET from
the post-fork/vfork child process.

Much of this was long overdue. We shouldn't have been using PyTuple and
PyLong APIs within all of these low level functions anyways..
(cherry picked from commit c649df6)

Co-authored-by: Gregory P. Smith <[email protected]>

gpshead added a commit to gpshead/cpython that referenced this pull request

May 23, 2023


          [3.11] pythongh-104372: Cleanup _posixsubprocess make_inheritable f…

80ee5fc

…or async signal safety and no GIL requirement (pythonGH-104518)

Move all of the Python C API calls into the parent process up front
instead of doing PyLong_AsLong and PyErr_Occurred and PyTuple_GET from
the post-fork/vfork child process.

Much of this was long overdue. We shouldn't have been using PyTuple and
PyLong APIs within all of these low level functions anyways..
(cherry picked from commit c649df6)

Co-authored-by: Gregory P. Smith <[email protected]>

gpshead added a commit to gpshead/cpython that referenced this pull request

May 23, 2023


          [3.11] pythongh-104372: Cleanup _posixsubprocess make_inheritable f…

ff570e5

…or async signal safety and no GIL requirement (pythonGH-104518)

Move all of the Python C API calls into the parent process up front
instead of doing PyLong_AsLong and PyErr_Occurred and PyTuple_GET from
the post-fork/vfork child process.

Much of this was long overdue. We shouldn't have been using PyTuple and
PyLong APIs within all of these low level functions anyways..
(cherry picked from commit c649df6)

Co-authored-by: Gregory P. Smith <[email protected]>

gpshead added a commit to gpshead/cpython that referenced this pull request

May 23, 2023


          pythongh-104372: Cleanup _posixsubprocess make_inheritable for asyn…

de47455

…c signal safety and no GIL requirement (python#104518)

Move all of the Python C API calls into the parent process up front
instead of doing PyLong_AsLong and PyErr_Occurred and PyTuple_GET from
the post-fork/vfork child process.

Much of this was long overdue. We shouldn't have been using PyTuple and
PyLong APIs within all of these low level functions anyways.

(cherry picked from commit c649df6)

gpshead added a commit to gpshead/cpython that referenced this pull request

May 23, 2023


          [3.11] pythongh-104372: Cleanup _posixsubprocess make_inheritable f…

0991dae

…or async signal safety and no GIL requirement (pythonGH-104518)

Move all of the Python C API calls into the parent process up front
instead of doing PyLong_AsLong and PyErr_Occurred and PyTuple_GET from
the post-fork/vfork child process.

Much of this was long overdue. We shouldn't have been using PyTuple and
PyLong APIs within all of these low level functions anyways..
(cherry picked from commit c649df6)

Co-authored-by: Gregory P. Smith <[email protected]>

This was referenced May 23, 2023

[3.11] gh-104372: Cleanup _posixsubprocess make_inheritable for async signal safety gh-104518 #104785

Merged

gh-104372: Drop the GIL around the vfork() call. #104782

Merged

gpshead added a commit that referenced this pull request

May 24, 2023


          [3.11] gh-104372: Cleanup _posixsubprocess make_inheritable for async…

6d00ae3

… signal safety gh-104518 (#104785)

Move all of the Python C API calls into the parent process up front
instead of doing PyLong_AsLong and PyErr_Occurred and PyTuple_GET from
the post-fork/vfork child process.

Much of this was long overdue. We shouldn't have been using PyTuple and
PyLong APIs within all of these low level functions anyways.

This is a backport of c649df6 for #104518 and the tiny adjustment in d1732fe #104697.

Backporting this allows backporting of the real bug fix that requires it.

Co-authored-by: Gregory P. Smith [Google] <[email protected]>

Reviewers

No reviews

Uh oh!

There was an error while loading. Please reload this page.

Assignees

gpshead

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Uh oh!

There was an error while loading. Please reload this page.

4 participants

Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.

You are visitor 01402520 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.

GeezerVison

Conversation

gpshead commented May 16, 2023 • edited by bedevere-bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

gpshead commented May 16, 2023

Uh oh!

bedevere-bot commented May 16, 2023

Uh oh!

bedevere-bot commented May 16, 2023

Uh oh!

Eclips4 commented May 16, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

bedevere-bot commented May 16, 2023

Uh oh!

bedevere-bot commented May 16, 2023

Uh oh!

bedevere-bot commented May 16, 2023

Uh oh!

gpshead commented May 17, 2023

Uh oh!

bedevere-bot commented May 17, 2023

Uh oh!

gpshead commented May 17, 2023

Uh oh!

bedevere-bot commented May 17, 2023

Uh oh!

gpshead commented May 17, 2023

Uh oh!

bedevere-bot commented May 17, 2023

Uh oh!

gpshead commented May 17, 2023

Uh oh!

bedevere-bot commented May 17, 2023

Uh oh!

miss-islington commented May 17, 2023

Uh oh!

miss-islington commented May 17, 2023

Uh oh!

bedevere-bot commented May 17, 2023

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Uh oh!

bedevere-bot commented May 17, 2023

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Uh oh!

bedevere-bot commented May 17, 2023

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

gpshead commented May 16, 2023 •

edited by bedevere-bot

Loading

Uh oh!

There was an error while loading. Please reload this page.

Eclips4 commented May 16, 2023 •

edited

Loading

Uh oh!

There was an error while loading. Please reload this page.