GeezerVison is showing github.com — "dyld: Library not loaded: /lib/libpython3.13.dylib" in 3.13.0a4 MacOS non-framework build with custom prefix · Issue #115737 · python/cpython
"dyld: Library not loaded: /lib/libpython3.13.dylib" in 3.13.0a4 MacOS non-framework build with custom prefix #115737DescriptionBug report
Bug description:
The above error was discovered in Pyenv CI check (logs are on the link).
See pyenv/pyenv#2903 (comment) for diagnostics and probable cause.
CPython versions tested on:
3.13
Operating systems tested on:
macOS
Linked PRs
Reacted by Edgar Ramírez Mondragón Activityronaldoussoren commented on Feb 20, 2024 How can we reproduce this? In particular, what is the command-line for configure and any additional customization of the build process? ned-deily commented on Feb 20, 2024 Reacted by Erlend E. Aasland and Edgar Ramírez Mondragón ronaldoussoren commented on Feb 20, 2024 I'd still like to know how to reproduce this without having to hunt through the pyenv GitHub. The linked issue seems to use -rpath somehow, which is not something we explicitly support. Reacted by Erlend E. Aasland and Ned Deily freakboy3742 commented on Feb 20, 2024 @ned-deily As with @ronaldoussoren, I'm trying to work out what the reproduction instructions are... I'll dig in and see if I can reverse engineer the problem. Reacted by Erlend E. Aasland native-api commented on Feb 21, 2024 ContributorAuthor More actions Attached a Bash trace log from the Pyenv invocation which contains configure/make calls and ennvar assignments. That's what I usually use to diagnose Pyenv issues.
out.txt
If you need a more distilled view, I'll be able to compose it when I return from work in about 10 hours.
Setting rpaths is required in MacOS to find dylibs outside of the default search path -- in Homebrew and under the custom installation prefix. freakboy3742 commented on Feb 21, 2024 @native-api Thanks for that detail - based on that, I've been able work out the reproduction case is:
./configure --prefix=$(pwd)/root --enable-shared --libdir=$(pwd)/root/lib && make -j4 all install && $(pwd)/root/bin/python3.13 -m test.pythoninfo
freakboy3742 commented on Feb 21, 2024 FWIW: it looks like the problem slipped in because while I did test non-framework builds, and non-framework builds in a non-default location, I didn't check a shared (i.e. ,--enable-shared) non-framework, non-default location build. Reacted by Ned Deily and Erlend E. Aasland ned-deily commented on Feb 21, 2024
shared (i.e. ,--enable-shared) non-framework, non-default location build
Sorry, I should have thought to suggest testing that one as well originally. I generally avoid --enable-shared builds on macOS as they often lead to problems like this but they are used. native-api commented on Feb 21, 2024 ContributorAuthor More actions We have enabled --enable-shared by default by user requests. The rationale was it is required for products that link against libpython and it's the setting in official CPython Docker images. ned-deily commented on Feb 21, 2024 To be sure, there's nothing inherently bad about using --enable-shared on macOS, especially when running from an installed Python. You just need to be vigilant during development and testing when running from a Python build directory to make sure the interpreter is not using the old installed shared library rather than the newly built one.
I've commented on the PR on how we can simplify the proposed fix.
That said, I stumbled across another potential problem here with --enable-shared. In the reproduction case above, there is:
--prefix=$(pwd)/root --enable-shared --libdir=$(pwd)/root/lib
Now it turns out that specifying that value for libdir is a no-op since the default value of libdir is $prefix/lib so there's no need to use libdir in this case. However, if you were to use a different value for libdir, then the macOS install_name, i.e. the path to the shared library that is baked into the python executable by the linker, is hardwired to be $prefix/lib so attempting to execute python will immediately segfault. It would be a simple change here to fix the install_name for the enable-shared case to use libdir, which is where the shared library is installed, and, while that does prevent the segfault, the installed Python is still not very usable as the standard library extension modules are also installed into libdir but the default sys.path doesn't take that into account so imports of them fail. This isn't a new issue it seems and has probably been around for almost forever but, as far as I recall, no one has every complained about it so it is probably best to leave well enough alone and not attempt to change this behavior here. Reacted by Erlend E. Aasland added a commit that references this issue on Feb 21, 2024gh-115737: Correct libpython install name for macOS shared library bu… ned-deily commented on Feb 21, 2024 Thanks for the report and the followup info. The simplified fix is merged into main for release in 3.13.0a5. Reacted by Erlend E. Aasland and native-api
MetadataMetadataDevelopmentNo branches or pull requestsIssue actions
|