GeezerVison is showing github.com — pyexpat changes in 3.10.14 broke Supervisor · Issue #117173 · python/cpython
pyexpat changes in 3.10.14 broke Supervisor #117173DescriptionBug report
Bug description:
Link for supervisor issue Supervisor/supervisor#1636
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/bin/supervisord", line 33, in <module>
sys.exit(load_entry_point('supervisor==4.2.4', 'console_scripts', 'supervisord')())
File "/usr/bin/supervisord", line 25, in importlib_load_entry_point
return next(matches).load()
File "/usr/lib/python3.10/importlib/metadata/__init__.py", line 171, in load
module = import_module(match.group('module'))
File "/usr/lib/python3.10/importlib/__init__.py", line 126, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
File "<frozen importlib._bootstrap_external>", line 883, in exec_module
File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
File "/usr/lib/python3.10/site-packages/supervisor/supervisord.py", line 38, in <module>
from supervisor.medusa import asyncore_25 as asyncore
File "/usr/lib/python3.10/site-packages/supervisor/medusa/asyncore_25.py", line 58, in <module>
from supervisor.compat import as_string, as_bytes
File "/usr/lib/python3.10/site-packages/supervisor/compat.py", line 69, in <module>
import xmlrpclib
ModuleNotFoundError: No module named 'xmlrpclib'
Traceback (most recent call last):
File "/usr/lib/python3.10/site-packages/supervisor/compat.py", line 67, in <module>
import xmlrpc.client as xmlrpclib
File "/usr/lib/python3.10/xmlrpc/client.py", line 138, in <module>
from xml.parsers import expat
File "/usr/lib/python3.10/xml/parsers/expat.py", line 4, in <module>
from pyexpat import *
ImportError: Error relocating /usr/lib/python3.10/lib-dynload/pyexpat.cpython-310-x86_64-linux-gnu.so: XML_SetReparseDeferralEnabled: symbol not found
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/bin/supervisord", line 33, in <module>
sys.exit(load_entry_point('supervisor==4.2.4', 'console_scripts', 'supervisord')())
File "/usr/bin/supervisord", line 25, in importlib_load_entry_point
return next(matches).load()
File "/usr/lib/python3.10/importlib/metadata/__init__.py", line 171, in load
module = import_module(match.group('module'))
File "/usr/lib/python3.10/importlib/__init__.py", line 126, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
File "<frozen importlib._bootstrap_external>", line 883, in exec_module
File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
File "/usr/lib/python3.10/site-packages/supervisor/supervisord.py", line 38, in <module>
from supervisor.medusa import asyncore_25 as asyncore
File "/usr/lib/python3.10/site-packages/supervisor/medusa/asyncore_25.py", line 58, in <module>
from supervisor.compat import as_string, as_bytes
File "/usr/lib/python3.10/site-packages/supervisor/compat.py", line 69, in <module>
import xmlrpclib
ModuleNotFoundError: No module named 'xmlrpclib'CPython versions tested on:
3.10
Operating systems tested on:
Linux ActivityEclips4 commented on Mar 23, 2024 Hello!
From e2caef7:
The :mod:`xmlrpclib` module has been renamed to :mod:`xmlrpc.client` in Python 3.0.
So, there's no problem, just use a xmlrpc.client instead of xmlrpclib.
It was renamed long time ago, but name of supervisor issue says "Fatal error after update python from 3.10.13 to 3.10.14", I'm sure that this cannot be true, import xmlrpclib cannot work on any of 3.10 versions, probably you're missing something. unlike777 commented on Mar 23, 2024 Hello!
Thanks for the prompt feedback, but I don’t agree with you
The supervisor definitely works on 3.10.13 - verified.
And it stops working on 3.10.14.
It turns out that the minor version influences the supervisor.
If you look at what was done in 3.10.14, this is exactly a fix for an xml vulnerability in Python. https://github.com/python/cpython/commits/3.10/
If you look at the supervisor code, they solved the problem you describe
https://github.com/Supervisor/supervisor/blob/main/supervisor/compat.py#L66
try: # pragma: no cover
import xmlrpc.client as xmlrpclib
except ImportError: # pragma: no cover
import xmlrpclibAlso, the supervisor hasn’t changed anything for a long time, and the breakdown occurred precisely after the release of Python 3.10.14
The problem is on your side Eclips4 commented on Mar 23, 2024 A closer look at the traceback makes it clear that the problem is probably related to version of the pyexpat library bundled in your system. I'm not expert in this kind of issues, so I'm re-open the issue. AlexWaygood commented on Mar 23, 2024 hugovk commented on Mar 23, 2024 Highlighting the most relevant part of the traceback:
Traceback (most recent call last):
File "/usr/lib/python3.10/site-packages/supervisor/compat.py", line 67, in <module>
import xmlrpc.client as xmlrpclib
File "/usr/lib/python3.10/xmlrpc/client.py", line 138, in <module>
from xml.parsers import expat
File "/usr/lib/python3.10/xml/parsers/expat.py", line 4, in <module>
from pyexpat import *
ImportError: Error relocating /usr/lib/python3.10/lib-dynload/pyexpat.cpython-310-x86_64-linux-gnu.so: XML_SetReparseDeferralEnabled: symbol not foundDiff between the releases: v3.10.13...v3.10.14
Had three expat changes:
Reacted by Alex Waygood and Kirill Podoprigora hugovk commented on Mar 23, 2024 And the related issues:
This last one seems most relevant, to expose XML_SetReparseDeferralEnabled, as the traceback says XML_SetReparseDeferralEnabled: symbol not found.
We don't provide Linux installers (and don't provide any installers for 3.10 because it's in the security-only/source-code only phase).
Could it be a problem with a distro-provided Python installation?
(See also jupyterhub/repo2docker-action#113 for a similar report.) changed the title [-]Release 3.10.14 broke Supervisor[/-] [+]pyexpat changes in 3.10.14 broke Supervisor[/+] on Mar 23, 2024hartwork commented on Mar 23, 2024 From a quick look my understanding is that @unlike777 is using Python binaries compiled on a host that did have symbol XML_SetReparseDeferralEnabled present in libexpat — through a backport or version >=2.6.0 — but the target system has installed binaries that lack these symbols. If that's what's going on, the fix is to get these two sides back in sync. Reacted by Serhiy Storchaka hartwork commented on Mar 23, 2024 @unlike777 which distro is this on, which release of the distro, and how did you install CPython and libexpat and which version are they now? unlike777 commented on Mar 23, 2024 @hartwork
Thank you very much for being included!
I use alpine:3, 3.16, 3.16.3, latest
Install the supervisor:
apk add supervisor
[supervisor 1/3] RUN apk add supervisor
2.126 (1/10) Installing mpdecimal (2.5.1-r1)
2.437 (2/10) Installing python3 (3.10.14-r0)
3.440 (3/10) Installing py3-appdirs (1.4.4-r3)
3.483 (4/10) Installing py3-more-itertools (8.13.0-r0)
3.532 (5/10) Installing py3-ordered-set (4.0.2-r3)
3.575 (6/10) Installing py3-parsing (2.4.7-r3)
3.629 (7/10) Installing py3-six (1.16.0-r1)
3.672 (8/10) Installing py3-packaging (21.3-r0)
3.720 (9/10) Installing py3-setuptools (59.4.0-r0)
3.818 (10/10) Installing supervisor (4.2.4-r0) hartwork commented on Mar 23, 2024 @unlike777 thanks. I cannot reproduce the issue with Alpine 3.16. This will need a robust minimal reproducer. What's a way to reliably reproduce the issue? hartwork commented on Mar 23, 2024 Reacted by unlike777, Hugo van Kemenade, Kirill Podoprigora and Shaun Walbridge AlexWaygood commented on Mar 23, 2024 Reacted by Kirill Podoprigora
MetadataMetadataDevelopmentNo branches or pull requestsIssue actions
|