GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — test_concurrent_futures.test_shutdown: AssertionError: b'' != b'apple' · Issue #112542 · python/cpython

test_concurrent_futures.test_shutdown: AssertionError: b'' != b'apple' #112542

New issue
Copy link
New issue
Copy link
Closed
Closed
test_concurrent_futures.test_shutdown: AssertionError: b'' != b'apple'#112542
Copy link
Labels
3.13only security fixesonly security fixesOS-freebsdtestsTests in the Lib/test dirTests in the Lib/test dirtype-bugAn unexpected behavior, bug, or errorAn unexpected behavior, bug, or error

Description

@furkanonder
furkanonder
opened on Nov 30, 2023
Issue body actions

Bug report

Bug description:

Bug report

Bug description:

Configuration:

./configure --with-pydebug

Test Output: (full-test-output.txt)

freebsd@generic:~/cpython % ./python -m test -j2
== CPython 3.13.0a2+ (heads/main:37589d76bb, Apr 29 2023, 11:49:22) [Clang 14.0.5 (https://github.com/llvm/llvm-project.git llvmorg-14.0.5-0-gc12386
== FreeBSD-13.2-RELEASE-arm64-aarch64-64bit-ELF little-endian
== Python build: debug
== cwd: /home/freebsd/cpython/build/test_python_worker_32780æ
== CPU count: 4
== encodings: locale=UTF-8 FS=utf-8
== resources: all test resources are disabled, use -u option to unskip tests

Using random seed: 3819139510
0:00:00 load avg: 2.45 Run 469 tests in parallel using 2 worker processes
0:00:02 load avg: 2.42 [  1/469] test.test_asyncio.test_buffered_proto passed
0:00:04 load avg: 2.42 [  2/469] test.test_asyncio.test_context passed
0:00:11 load avg: 2.38 [  3/469] test.test_asyncio.test_eager_task_factory passed
0:00:15 load avg: 2.35 [  4/469] test.test_asyncio.test_base_events passed
0:00:39 load avg: 2.23 [  5/469] test.test_asyncio.test_futures passed
0:00:41 load avg: 2.21 [  6/469] test.test_asyncio.test_futures2 passed -- running (1): test.test_asyncio.test_events (30.4 sec)
0:00:49 load avg: 2.20 [  7/469] test.test_asyncio.test_locks passed -- running (1): test.test_asyncio.test_events (38.4 sec)
0:00:53 load avg: 2.26 [  8/469] test.test_asyncio.test_pep492 passed -- running (1): test.test_asyncio.test_events (41.7 sec)
0:00:59 load avg: 2.24 [  9/469] test.test_asyncio.test_proactor_events passed -- running (1): test.test_asyncio.test_events (47.8 sec)
0:01:01 load avg: 2.22 [ 10/469] test.test_asyncio.test_protocols passed -- running (1): test.test_asyncio.test_events (50.1 sec)
0:01:06 load avg: 2.20 [ 11/469] test.test_asyncio.test_queues passed -- running (1): test.test_asyncio.test_events (54.8 sec)
0:01:09 load avg: 2.19 [ 12/469] test.test_asyncio.test_runners passed -- running (1): test.test_asyncio.test_events (58.3 sec)
0:01:15 load avg: 2.17 [ 13/469] test.test_asyncio.test_selector_events passed -- running (1): test.test_asyncio.test_events (1 min 4 sec)
0:01:18 load avg: 2.17 [ 14/469] test.test_asyncio.test_events passed (1 min 6 sec)
0:01:21 load avg: 2.08 [ 15/469] test.test_asyncio.test_server passed
/home/freebsd/cpython/Lib/asyncio/events.py:84: ResourceWarning: unclosed <socket.socket fd=8, family=1, type=1, proto=0, laddr=./test_python_jpgh9dpl.sock>
  self._context.run(self._callback, *self._args)
ResourceWarning: Enable tracemalloc to get the object allocation traceback
Executing <Task finished name='Task-44' coro=<UnixServerCleanupTests.test_unix_server_cleanup_replaced() done, defined at /home/freebsd/cpython/Lib/test/test_asyncio/test_server.py:227> result=None created at /home/freebsd/cpython/Lib/asyncio/runners.py:100> took 0.104 seconds
0:01:33 load avg: 1.91 [ 16/469] test.test_asyncio.test_sock_lowlevel passed
0:02:03 load avg: 1.62 running (2): test.test_asyncio.test_sendfile (47.9 sec), test.test_asyncio.test_ssl (30.0 sec)

0:02:33 load avg: 1.58 running (2): test.test_asyncio.test_sendfile (1 min 17 sec), test.test_asyncio.test_ssl (1 min)
0:02:37 load avg: 1.61 [ 17/469] test.test_asyncio.test_sendfile passed (1 min 22 sec) -- running (1): test.test_asyncio.test_ssl (1 min 4 sec)
0:02:46 load avg: 2.04 [ 18/469] test.test_asyncio.test_sslproto passed -- running (1): test.test_asyncio.test_ssl (1 min 12 sec)
0:02:50 load avg: 2.04 [ 19/469] test.test_asyncio.test_streams passed -- running (1): test.test_asyncio.test_ssl (1 min 17 sec)
0:03:16 load avg: 2.02 [ 20/469] test.test_asyncio.test_ssl passed (1 min 42 sec)
0:03:20 load avg: 1.94 [ 21/469] test.test_asyncio.test_subprocess passed
0:03:27 load avg: 1.95 [ 22/469] test.test_asyncio.test_taskgroups passed
0:03:30 load avg: 1.95 [ 23/469] test.test_asyncio.test_threads passed
0:03:34 load avg: 1.87 [ 24/469] test.test_asyncio.test_timeouts passed
0:03:37 load avg: 1.96 [ 25/469] test.test_asyncio.test_transports passed
0:03:46 load avg: 1.97 [ 26/469] test.test_asyncio.test_unix_events passed
0:03:50 load avg: 1.97 [ 27/469] test.test_asyncio.test_waitfor passed -- running (1): test.test_asyncio.test_tasks (30.5 sec)
0:03:52 load avg: 1.97 [ 28/469] test.test_asyncio.test_windows_events skipped -- running (1): test.test_asyncio.test_tasks (32.8 sec)
test.test_asyncio.test_windows_events skipped -- Windows only
0:03:54 load avg: 1.97 [ 29/469] test.test_asyncio.test_windows_utils skipped -- running (1): test.test_asyncio.test_tasks (34.7 sec)
test.test_asyncio.test_windows_utils skipped -- Windows only
0:04:23 load avg: 1.75 [ 30/469] test.test_concurrent_futures.test_as_completed passed -- running (1): test.test_asyncio.test_tasks (1 min 3 sec)
0:04:40 load avg: 1.84 [ 31/469] test.test_asyncio.test_tasks passed (1 min 20 sec)
0:04:45 load avg: 1.77 [ 32/469] test.test_concurrent_futures.test_future passed
0:04:54 load avg: 1.80 [ 33/469] test.test_concurrent_futures.test_init passed -- running (1): test.test_concurrent_futures.test_deadlock (30.8 sec)
0:05:24 load avg: 2.12 running (2): test.test_concurrent_futures.test_process_pool (30.0 sec), test.test_concurrent_futures.test_deadlock (1 min)
0:05:27 load avg: 2.12 [ 34/469] test.test_concurrent_futures.test_deadlock passed (1 min 4 sec) -- running (1): test.test_concurrent_futures.test_process_pool (33.8 sec)
0:05:57 load avg: 2.95 running (2): test.test_concurrent_futures.test_process_pool (1 min 3 sec), test.test_concurrent_futures.test_shutdown (30.0 sec)
0:06:09 load avg: 3.83 [ 35/469] test.test_concurrent_futures.test_process_pool passed (1 min 15 sec) -- running (1): test.test_concurrent_futures.test_shutdown (41.3 sec)
0:06:13 load avg: 3.68 [ 36/469] test.test_concurrent_futures.test_thread_pool passed -- running (1): test.test_concurrent_futures.test_shutdown (45.2 sec)
0:06:30 load avg: 3.30 [ 37/469/1] test.test_concurrent_futures.test_shutdown failed (1 failure) (1 min 2 sec)
test test.test_concurrent_futures.test_shutdown failed -- Traceback (most recent call last):
  File "/home/freebsd/cpython/Lib/test/test_concurrent_futures/test_shutdown.py", line 50, in test_interpreter_shutdown
    self.assertEqual(out.strip(), b"apple")
AssertionError: b'' != b'apple'

...
..
.

== Tests result: FAILURE ==

24 tests skipped:
    test.test_asyncio.test_windows_events
    test.test_asyncio.test_windows_utils test_ctypes test_dbm_gnu
    test_devpoll test_epoll test_idle test_ioctl test_launcher
    test_msvcrt test_perf_profiler test_perfmaps test_readline
    test_sqlite3 test_stable_abi_ctypes test_startfile test_tcl
    test_tkinter test_ttk test_ttk_textonly test_turtle
    test_winconsoleio test_winreg test_wmi

8 tests skipped (resource denied):
    test_curses test_peg_generator test_smtpnet test_socketserver
    test_urllib2net test_urllibnet test_winsound test_zipfile64

4 tests failed:
    test.test_concurrent_futures.test_shutdown
    test.test_gdb.test_backtrace test.test_gdb.test_misc
    test.test_gdb.test_pretty_print

433 tests OK.

Total duration: 57 min 8 sec
Total tests: run=39,180 failures=7 skipped=1,476
Total test files: run=461/469 failed=4 skipped=24 resource_denied=8
Result: FAILURE

Environment:

freebsd@generic:~/cpython % uname -a
FreeBSD generic 13.2-RELEASE FreeBSD 13.2-RELEASE releng/13.2-n254617-525ecfdad597 GENERIC arm64
freebsd@generic:~/cpython % 

CPython versions tested on:

CPython main branch

Operating systems tested on:

Other

Activity

  1. furkanonder
    added
    type-bugAn unexpected behavior, bug, or errorAn unexpected behavior, bug, or error
    on Nov 30, 2023
  2. furkanonder commented on Nov 30, 2023

    @furkanonder
    furkanonder
    on Nov 30, 2023
    ContributorAuthor
    More actions

    CC: @vstinner

  3. furkanonder
    added
    testsTests in the Lib/test dirTests in the Lib/test dir
    OS-freebsd
    on Nov 30, 2023
  4. vstinner commented on Nov 30, 2023

    @vstinner
    vstinner
    on Nov 30, 2023
    Member
    More actions

    Sadly, the failure is not new:

    • test_concurrent_futures.test_shutdown: test_interpreter_shutdown() fails randomly (race condition) #109047
    • test_concurrent_futures: test_cancel_futures_wait_false() and test_interpreter_shutdown() failed on GHA Windows x64 #89099

    My analysis: #109047 (comment)

    Do you want to work on a fix?

    I fixed the new RuntimeError variant in #109810

  5. furkanonder
    added this to Multiprocessing issueson Dec 2, 2023
  6. furkanonder commented on Dec 5, 2023

    @furkanonder
    furkanonder
    on Dec 5, 2023
    ContributorAuthor
    More actions

    Sadly, the failure is not new:

    * [test_concurrent_futures.test_shutdown: test_interpreter_shutdown() fails randomly (race condition) #109047](https://github.com/python/cpython/issues/109047)
    
    * [test_concurrent_futures: test_cancel_futures_wait_false() and test_interpreter_shutdown() failed on GHA Windows x64 #89099](https://github.com/python/cpython/issues/89099)
    

    My analysis: #109047 (comment)

    Do you want to work on a fix?

    I fixed the new RuntimeError variant in #109810

    Do you want to work on a fix?

    Sure, I would like to work on a fix.

  7. furkanonder
    moved this to In Progress in Multiprocessing issueson Dec 8, 2023
  8. LeamHall commented on Jan 10, 2024

    @LeamHall
    LeamHall
    on Jan 10, 2024
    More actions

    This just started happening today on Devuan Linux (Linux josiah 6.1.0-17-amd64 #1 SMP PREEMPT_DYNAMIC Debian 6.1.69-1 (2023-12-30) x86_64 GNU/Linux). It did not happen yesterday, from yesterday's pull from main. It did not happen on today's build on Fedora 37.

  9. vstinner commented on Jan 22, 2024

    @vstinner
    vstinner
    on Jan 22, 2024
    Member
    More actions

    The test now fails on the Windows 11 Refleaks 3.12 buildbot: https://buildbot.python.org/all/#/builders/1103.

  10. sobolevn commented on Jan 31, 2024

    @sobolevn
    sobolevn
    on Jan 31, 2024
    Member
    More actions

    It happened again on GitHub: https://github.com/python/cpython/actions/runs/7723607693/job/21054076387

  11. vstinner commented on Jan 31, 2024

    @vstinner
    vstinner
    on Jan 31, 2024
    Member
    More actions

    It happened again on GitHub: https://github.com/python/cpython/actions/runs/7723607693/job/21054076387

    That's a Windows x64 build. Apparently, the issue is specific to Windows.

  12. vstinner commented on Jan 31, 2024

    @vstinner
    vstinner
    on Jan 31, 2024
    Member
    More actions

    Apparently, the issue is specific to Windows.

    Sorry, I wrote too quickly. The race condition was seen on FreeBSD and Linux as well, but it's really hard to reproduce.

  13. colesbury commented on Mar 12, 2024

    @colesbury
    colesbury
    on Mar 12, 2024
    Contributor
    More actions

    This happens reliably in the free-threaded build with PYTHON_GIL=0. The cause is that we are trying to start a thread at interpreter shutdown, which sometimes fails depending on if starting the thread is faster than shutting down. The "can't create new thread at interpreter shutdown" error is hidden due to #109810, which catches the exception. Nobody looks at the result, so no exception is printed.

    The order of operations is:

    1. t.submit(sleep_and_print, 1.0, "apple") starts _ExecutorManagerThread and then begins interpreter shutdown
    2. _ExecutorManagerThread calls add_call_item_to_queue which starts a new thread.

    If (2) happens quickly enough, then we get b"apple" and the tests pass. If the interpreter shutdown sets interp->finalizing first, then the Queue thread fails to start and nothing gets printed.

    This is closely related to:

    • Checking for sys.is_finalizing before thread.start() still yields "can't create new thread at interpreter shutdown" #114570
    • 3.12 regression: "can't create new thread at interpreter shutdown" from non-daemon threads or atexit handlers #113964
    • test_concurrent_futures.test_shutdown: test_interpreter_shutdown() fails randomly (race condition) #109047
    Reacted by Furkan Onder
  14. colesbury commented on Mar 12, 2024

    @colesbury
    colesbury
    on Mar 12, 2024
    Contributor
    More actions

    #116677 should address this as well as a number of other related issues.

    Reacted by Furkan Onder
    Reacted by Furkan Onder
  15. furkanonder
    added
    3.13only security fixesonly security fixes
    on Aug 21, 2024
  16. furkanonder commented on Sep 1, 2024

    @furkanonder
    furkanonder
    on Sep 1, 2024
    ContributorAuthor
    More actions

    #116677 should address this as well as a number of other related issues.
    I've been running the tests for a while, and there haven't been any failed cases. The issue appears to be solved.
    Thank you for fixing it! @colesbury

  17. furkanonder
    closed this as completedon Sep 1, 2024
  18. github-project-automation
    moved this from In Progress to Done in Multiprocessing issueson Sep 1, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    3.13only security fixesonly security fixesOS-freebsdtestsTests in the Lib/test dirTests in the Lib/test dirtype-bugAn unexpected behavior, bug, or errorAn unexpected behavior, bug, or error

    Projects

    • Multiprocessing issues
      Status
      Done
      Show more project fields

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions

    You are visitor 01402503 — GeezerVison does not store the pages you look up.
    Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.