GeezerVison is showing github.com — ImportError: _ssl.so: undefined symbol: _PyModule_Add · Issue #108525 · python/cpython
ImportError: _ssl.so: undefined symbol: _PyModule_Add #108525DescriptionBug report
Environment
$ uname
Linux
$ python --version{,}
Python 3.11.4 (tags/v3.11.4:d2340ef257, Aug 26 2023, 22:42:28) [GCC 12.2.0]Description
Importing the ssl shared library from a Python 3.11.5 build from a Python 3.11.4 binary produces an ImportError due to an undefined symbol. Isn't CPython's ABI supposed to be fully compatible between patch releases?
CPython’s Application Binary Interface (ABI) is forward- and backwards-compatible across a minor release.
Example
FROM debian
RUN apt update\
&& apt install --yes build-essential git libssl-dev zlib1g-dev
RUN git clone --branch v3.11.4 https://github.com/python/cpython 3.11.4\
&& cd 3.11.4 && ./configure && make
RUN git clone --branch v3.11.5 https://github.com/python/cpython 3.11.5\
&& cd 3.11.5 && ./configure && make
RUN mv 3.11.5/build/lib.linux-x86_64-3.11/_ssl.cpython-311-x86_64-linux-gnu.so\
3.11.4/build/lib.linux-x86_64-3.11/_ssl.cpython-311-x86_64-linux-gnu.so
RUN 3.11.4/python -m sslError
Traceback (most recent call last):
File "<frozen runpy>", line 198, in _run_module_as_main
File "<frozen runpy>", line 88, in _run_code
File "//3.11.2/Lib/ssl.py", line 100, in <module>
import _ssl # if we can't import it, let the error propagate
^^^^^^^^^^^
ImportError: //3.11.2/build/lib.linux-x86_64-3.11/_ssl.cpython-311-x86_64-linux-gnu.so: undefined symbol: _PyModule_Add
Observations
It looks like fced79f is the breaking commit, as per git bisect results.
Linked PRs
Reacted by Willi Sontopski, Satadru Pramanik, DO, MPH, MEng, Sandro Huber, Mathieu Tarral and Alex Povel Activityhansegucker commented on Aug 27, 2023 I also observed the issue on my system. Reacted by Ben Felder, Helio Machado, Zvolský, McCoy R. Becker, Shubham Singh, Dmitry Chelnokov, Chaitanya and Solly CorvinM commented on Aug 27, 2023 Also able to reproduce between 3.11 branch and 3.11.4, looks like #106863 broke the ABI Reacted by Helio Machado and Ben Felder sunmy2019 commented on Aug 28, 2023 serhiy-storchaka commented on Aug 28, 2023 I don't think this warranty extends to the private API.
Names prefixed by an underscore, such as _Py_InternalState, are private API that can change without notice even in patch releases.
If you only use the public C API in your extension, you can expect forward- and backwards-compatibinility. added a commit that references this issue on Aug 28, 2023This issue will close once commit 0b3c4df is merged into the 'main' branch.0b3c4df Templarrr commented on Aug 31, 2023 I see the same error locally in poetry after my Ubuntu upgraded 3.11.? -> 3.11.5. Any advice how this can be mitigated/fixed? It doesn't look like I can downgrade local python back to 3.11.4 - deadsnakes ppa don't list 3.11.4 anymore Reacted by Corvin McPherson, Helio Machado, McCoy R. Becker, Rushil Srivastava, Carlos and Alexander Dorozhinskiy LostInDarkMath commented on Aug 31, 2023 @Templarrr I had the same issue and fixed it by updating poetry to the latest version. Reacted by Corvin McPherson, Ismail Zouaoui, Ignat Vakorin, McCoy R. Becker, Mazen Mardini, Ser Shu, Rémy Chabin, Weidenthal Sandra and Pedro Pereira erlend-aasland commented on Aug 31, 2023 Templarrr commented on Aug 31, 2023 Reacted by Ser Shu Reacted by Willi Sontopski, Ismail Zouaoui and McCoy R. Becker encukou commented on Aug 31, 2023 The reproducer looks contrived. In what situation would pieces of CPython get mixed up like this? Is Ubuntu's _ssl module not built together with the interpreter? CorvinM commented on Aug 31, 2023
The reproducer looks contrived. In what situation would pieces of CPython get mixed up like this? Is Ubuntu's _ssl module not built together with the interpreter?
A faulty install or packaging script could easily cause this (but wouldn't be noticed in most cases, since they are usually ABI compatible) because the file paths are named identically (I.e., specific to 3.11, not 3.11.5). CorvinM commented on Aug 31, 2023 In this case it looks like a problem with poetry packaging (or its pythonpath?) 0x2b3bfa0 commented on Aug 31, 2023 The original issue was also related to Python + Poetry, by the way. Reacted by Corvin McPherson 0x2b3bfa0 commented on Sep 1, 2023
A faulty install or packaging script could easily cause this [...] because the file paths are named identically (i.e., specific to 3.11, not 3.11.5)
This. ☝🏼 💯 Reacted by Corvin McPherson zvolsky commented on Sep 14, 2023
@LostInDarkMath: I had the same issue and fixed it by updating poetry to the latest version.
My poetry --version was 1.6.1.
poetry self update exactly as other poetry commands has failed with this issue.
I decided to "update" via curl -sSL https://install.python-poetry.org | python3 -, i.e. reinstall,
the poetry --version is now again 1.6.1.
And Poetry works again.
(Debian Testing) Reacted by Michael Odintsov, Corvin McPherson, sbruens, Emmanuel Arias, AL, Lucas Parzianello, Maria Belen, Ser Shu, Paddy Alton, Magnus Pierrau and 10 more Reacted by Maria Belen, Dmitry Chelnokov, Yoav Cohen and Владислав Reacted by Karim Moradi and Владислав dusty-phillips commented on Oct 6, 2023 I encountered this issue in a python3.11-slim Dockerfile even after updating poetry.
In my case, the issue was that the Dockerfile in question was installing the python3-dev apt package, which was providing 3.11.2 headers. I was able to remove this dependency without any repercussions.
Was devilishly hard to track down so thought I'd share in case it helps someone else. Reacted by Helio Machado and Rushil Srivastava cove9988 commented on Nov 14, 2023 Yes, the old poetry 1.6.1 is not working well with python 3.11
if you curl -sSL https://install.python-poetry.org | python3 - upgrade to 1.7.1, It works well. Reacted by Paddy Alton, Helio Machado, Andrei Shabanski, Alexandre Karpov and luked99 vstinner commented on Dec 2, 2023
Importing the ssl shared library from a Python 3.11.5 build from a Python 3.11.4 binary produces an ImportError due to an undefined symbol.
Usually, a Python binary loads its "own" Python stdlib. I don't think that we want to support using a newer/older Python with newer/older stdlib. You should use the exact same version for the Python binary and its stdlib.
What can be done is to enhance error reporting. For example, but storing the version in stdlib C extensions and raise an error if there is version mismatch.
Today, you get an ImportError because of a missing symbol. But tomorrow, you might get a crash because of subtle different behavior :-( vstinner commented on Mar 18, 2024 I close the issue. I don't think that we want to support this use case.
MetadataMetadataDevelopmentNo branches or pull requestsIssue actions
|