GeezerVison

The World Wide Web, restored to 800 pixels of glory.

GeezerVison is showing github.com — [subinterpreters] crash when importing _sre in subinterpreters in parallel (Python 3.9 regression) · Issue #90228 · python/cpython

[subinterpreters] crash when importing _sre in subinterpreters in parallel (Python 3.9 regression) #90228

New issue

Copy link

New issue

Copy link

Closed

[subinterpreters] crash when importing _sre in subinterpreters in parallel (Python 3.9 regression)#90228

Copy link

Labels

3.10 (EOL)end of lifeend of life3.11only security fixesonly security fixes3.9 (EOL)end of lifeend of lifeinterpreter-core(Objects, Python, Grammar, and Parser dirs)(Objects, Python, Grammar, and Parser dirs)topic-regextopic-subinterpreterstype-crashA hard crash of the interpreter, possibly with a core dumpA hard crash of the interpreter, possibly with a core dump

Description

graysky

mannequin

opened

on Dec 14, 2021

Issue body actions

BPO	46070
Nosy	@vstinner, @encukou, @ericsnowcurrently, @ndjensen, @hroncok, @uckelman, @corona10, @miss-islington, @shihai1991, @erlend-aasland, @bsteffensmeier, @M-Reimer, @prahal
PRs	bpo-46070: Fix asyncio initialisation guard #30423 [3.10] bpo-46070: Fix asyncio initialisation guard (GH-30423) #30453 [3.9] bpo-46070: Fix asyncio initialisation guard (GH-30423) #30454 bpo-46070: Revert "bpo-36854: Move _PyRuntimeState.gc to PyInterpreterState (GH-17287)" #30564 [3.10] bpo-46070: Revert "bpo-36854: Move _PyRuntimeState.gc to PyInterpreterState (GH-17287)" #30565 [3.9] bpo-46070: Revert "bpo-36854: Move _PyRuntimeState.gc to PyInterpreterState (GH-17287)" #30566 bpo-46070: _PyGC_Fini() untracks objects #30577 [3.10] bpo-46070: _PyGC_Fini() untracks objects (GH-30577) #30578 [3.9] bpo-46070: _PyGC_Fini() untracks objects (GH-30577) #30579 [3.9] bpo-46070: _PyGC_Fini() untracks objects (GH-30577) #30580
Files	win_py399_crash_reproducer.py: Reproducer that intermittently crashes on windows bug.py pyobject_ob_interp.patch sqlite3_crash.py: Reproducer that crashes 90% of the time bug.py_asyncio_cpustressed-crash.log: asyncio bug.py crash while running stress -c `nproc --all`

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = None
created_at = <Date 2021-12-14.10:57:50.149>
labels = ['interpreter-core', 'type-bug', '3.9', '3.10', '3.11', 'expert-subinterpreters']
title = '[subinterpreters] crash when importing _sre in subinterpreters in parallel (Python 3.9 regression)'
updated_at = <Date 2022-03-24.21:16:20.771>
user = 'https://bugs.python.org/graysky'

bugs.python.org fields:

activity = <Date 2022-03-24.21:16:20.771>
actor = 'prahal'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = ['Interpreter Core', 'Subinterpreters']
creation = <Date 2021-12-14.10:57:50.149>
creator = 'graysky'
dependencies = []
files = ['50494', '50497', '50560', '50699', '50700']
hgrepos = []
issue_num = 46070
keywords = ['patch']
message_count = 54.0
messages = ['408520', '408539', '408540', '408541', '408558', '408568', '408633', '408662', '408664', '408665', '409245', '409255', '409461', '409573', '409686', '409772', '409778', '409780', '409795', '409798', '409802', '409962', '409965', '409966', '409970', '409980', '409983', '409984', '409991', '410010', '410014', '410442', '410444', '410446', '410447', '410493', '410497', '410498', '410500', '410505', '410507', '410509', '410510', '410513', '410517', '410518', '410520', '415954', '415955', '415963', '415964', '415970', '415975', '415977']
nosy_count = 15.0
nosy_names = ['vstinner', 'petr.viktorin', 'eric.snow', 'ndjensen', 'hroncok', 'uckelman', 'corona10', 'miss-islington', 'shihai1991', 'erlendaasland', 'graysky', 'bsteffensmeier', 'M-Reimer', 'jokot3', 'prahal']
pr_nums = ['30423', '30453', '30454', '30564', '30565', '30566', '30577', '30578', '30579', '30580']
priority = 'normal'
resolution = None
stage = 'patch review'
status = 'open'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue46070'
versions = ['Python 3.9', 'Python 3.10', 'Python 3.11']

Activity

graysky commented on Dec 14, 2021
grayskymannequin
on Dec 14, 2021
MannequinAuthor
More actions
Seems as though cpython is broken when working with subinterpreters. The problematic change could be (d0d2965) affecting import.c.[1] Reverting this commit and rebuilding python fixes the issues on my system with some scripts that import sqlite, for example, the Kodi plugin YouTube[2] and the IMDB Trailers plugin. Others have reported similar breakage with other python code[3].

Example output when the bug manifests:

ERROR <general>: Traceback (most recent call last): ERROR <general>: File "/usr/lib/python3.10/sqlite3/dbapi2.py", line 83, in register_adapters_and_converters ERROR <general>: ERROR <general>: register_converter("timestamp", convert_timestamp) ERROR <general>: ERROR <general>: KeyError ERROR <general>: : ERROR <general>: 'timepart_full' ERROR <general>: ERROR <general>: Exception ignored deletion of interned string failed ERROR <general>: :

References:

d0d2965

Plugin incompatible with python 3.10.1 anxdpanic/plugin.video.youtube#255

https://bbs.archlinux.org/viewtopic.php?id=272121
graysky
added
3.10 (EOL)end of lifeend of life

interpreter-core(Objects, Python, Grammar, and Parser dirs)(Objects, Python, Grammar, and Parser dirs)
on Dec 14, 2021
ericsnowcurrently commented on Dec 14, 2021
ericsnowcurrently
on Dec 14, 2021
Member
More actions
(related: bpo-44059)

Presumably the problem relates to global state used in different interpreters leading to an inconsistent state in the crashing extension (or its dependencies).

@graysky, do you know if this was a problem before Python 3.8?
ericsnowcurrently
added
topic-subinterpreters

3.9 (EOL)end of lifeend of life

3.11only security fixesonly security fixes

type-bugAn unexpected behavior, bug, or errorAn unexpected behavior, bug, or error
on Dec 14, 2021
encukou commented on Dec 14, 2021
encukou
on Dec 14, 2021
Member
More actions
Interned strings were broken in #64257, see bpo-46006. Maybe that's also the issue here?
58 remaining items
ezio-melotti
added this to Subinterpreterson Apr 15, 2022
ezio-melotti
moved this to Todo in Subinterpreterson Apr 15, 2022
AlexWaygood
added
topic-regex
on Apr 19, 2022
vstinner commented on Apr 28, 2022
vstinner
on Apr 28, 2022
Member
More actions
So far, only released Python v3.11.0a4 include my "_PyGC_Fini() untracks objects" fix. The 3.9 and 3.10 branch include my fix, but there is no release including it yet.

Scheduled releases:

Python 3.9.13: Monday, 2022-05-16: https://peps.python.org/pep-0596/

Python 3.10.5: Monday, 2022-06-06: https://peps.python.org/pep-0619/

I managed to reproduce the bug.py crash with main branch up to commit 12c0012 .

Oh really? This version include my fix. I cannot reproduce it on the current main branch (commit ea2f5bc) nor at commit 12c0012. Which commands did you type to build Python and reproduce the crash? What is your OS and C compiler?
Reacted by Peter Fern
vstinner commented on Apr 28, 2022
vstinner
on Apr 28, 2022
Member
More actions
The original issue is fixed. Please open a new issue for sqlite3_crash.py.
vstinner commented on Apr 28, 2022
vstinner
on Apr 28, 2022
Member
More actions
I created #92036 for sqlite3_crash.py bug.
erlend-aasland commented on Apr 28, 2022
erlend-aasland
on Apr 28, 2022
Contributor
More actions
Should this be closed then?
vstinner commented on Apr 28, 2022
vstinner
on Apr 28, 2022
Member
More actions
I kept this issue open for this reason:

It would be nice to add some tests.

Reacted by Erlend E. Aasland and Eric Snow
vstinner commented on Apr 28, 2022
vstinner
on Apr 28, 2022
Member
More actions
My notes about this bug: https://twitter.com/VictorStinner/status/1481783100191428611
Reacted by Erlend E. Aasland and Jose Riha
heitbaum
added a commit that references this issue on May 17, 2022
Python3: update to 3.9.13
5c655c7
serhiy-storchaka
added
type-crashA hard crash of the interpreter, possibly with a core dumpA hard crash of the interpreter, possibly with a core dump
and removed
type-bugAn unexpected behavior, bug, or errorAn unexpected behavior, bug, or error
on Jul 3, 2026
serhiy-storchaka commented on Jul 3, 2026
serhiy-storchaka
on Jul 3, 2026
Member
More actions
_sre was converted to multi-phase init with per-interpreter state, which removed the root cause. Importing _sre in 160 subinterpreters across 8 parallel threads runs cleanly on a debug build of main. Closing as fixed.
serhiy-storchaka
closed this as completed on Jul 3, 2026
github-project-automation
moved this from Todo to Done in Subinterpreterson Jul 3, 2026

Metadata

Assignees

No one assigned

Labels

Projects

Subinterpreters
Status
Done
Show more project fields

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions

You are visitor 01402503 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.

GeezerVison

[subinterpreters] crash when importing _sre in subinterpreters in parallel (Python 3.9 regression) #90228

Description

Activity

graysky commented on Dec 14, 2021

ericsnowcurrently commented on Dec 14, 2021

encukou commented on Dec 14, 2021

58 remaining items

vstinner commented on Apr 28, 2022

vstinner commented on Apr 28, 2022

vstinner commented on Apr 28, 2022

erlend-aasland commented on Apr 28, 2022

vstinner commented on Apr 28, 2022

vstinner commented on Apr 28, 2022

serhiy-storchaka commented on Jul 3, 2026

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions