Description

intel-hm

opened on Mar 18, 2025

Issue body actions

Bug report

Bug description:

from Python-3.13.2.tar.xz

./configure --prefix=$RUN/python/3.13.2
make -j 16
make -j 4 install

dynamic libraries are installed to $RUN/3.13.2/lib64/python3.13/lib-dynload
but internally sys.path uses $RUN/3.13.2/lib/python3.13/lib-dynload

setting a link resolves the

errors that show up:

> python3
**Could not find platform dependent libraries <exec_prefix>**
Python 3.13.2 (main, Mar 18 2025, 10:25:05) [GCC 7.5.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
**warning: can't use pyrepl: No module named '_opcode'**

pip3 download pip wheel setuptools 
...
    from _posixsubprocess import fork_exec as _fork_exec
**ModuleNotFoundError: No module named '_posixsubprocess'**

CPython versions tested on:

3.13

Operating systems tested on:

Linux

Reacted by James Thomas Moon

Activity

1. intel-hm

   added

   type-bugAn unexpected behavior, bug, or errorAn unexpected behavior, bug, or error

   on Mar 18, 2025
2. picnixz commented on Mar 18, 2025

   

   picnixz

   on Mar 18, 2025

   Member

   More actions

   As an openSUSE user, I had this problem as well. I think I did a combination of --exec-prefix=... and altinstall. I can't remember exactly the command and I won't be back until Sunday. I can check at that time.
3. picnixz

   added

   topic-installation

   OS-linux

   on Mar 18, 2025
4. picnixz

   changed the title [-]installation form source on "openSUSE Leap 15.5" and "SUSE Linux Enterprise Server 15 SP6" - incorrect location of python3.13/lib-dynload[/-] [+]Incorrect installation of `lib-dynload` for custom builds on openSUSE[/+] on Mar 18, 2025
5. tfpf commented on Mar 22, 2025

   

   tfpf

   on Mar 22, 2025

   More actions

   This was previously reported and discussed in #108819. There are workarounds suggested in the comments on that issue, but no comments from any maintainers.
6. picnixz commented on Mar 22, 2025

   

   picnixz

   on Mar 22, 2025

   Member

   More actions

   I think I actually also specified the libdir explicitly so that it doesn't use lib64 but lib instead. I'll see if I can write better installation instructions for openSUSE
7. intel-hm commented on Mar 22, 2025

   

   intel-hm

   on Mar 22, 2025

   Author

   More actions

   I think that's an error in the configure script. And it's not only OPENSUSE - it's also SLES
8. picnixz commented on Mar 22, 2025

   

   picnixz

   on Mar 22, 2025

   Member

   More actions

   Yes I think so as well but I'll need to confirm it properly. I'll also tag those responsible for openSUSE contact
9. picnixz commented on Apr 28, 2025

   

   picnixz

   on Apr 28, 2025

   Member

   More actions

   @mcepl From a configure PoV, what's the best? detect that we're on openSUSE or SLES or do something else?
10. picnixz

    added

    OS-unsupported

    and removed

    OS-linux

    on Apr 28, 2025
11. github-project-automation

    added this to Unsupported platformson Apr 28, 2025
12. picnixz

    added

    OS-linux

    and removed

    OS-unsupported

    on Apr 28, 2025
13. picnixz

    removed this from Unsupported platformson Apr 28, 2025
14. intel-hm commented on Apr 28, 2025

    

    intel-hm

    on Apr 28, 2025

    Author

    More actions

    could someone figure out what causes this difference?
    dynamic libraries are installed to $RUN/3.13.2/lib64/python3.13/lib-dynload
    but internally sys.path uses $RUN/3.13.2/lib/python3.13/lib-dynload

    how are those 2 paths defined in configure? what is different on SUSE that causes the difference?
15. tfpf commented on Apr 28, 2025

    

    tfpf

    on Apr 28, 2025

    More actions

    could someone figure out what causes this difference? dynamic libraries are installed to $RUN/3.13.2/lib64/python3.13/lib-dynload but internally sys.path uses $RUN/3.13.2/lib/python3.13/lib-dynload

    how are those 2 paths defined in configure? what is different on SUSE that causes the difference?

    I believe it’s because the configure script doesn’t handle the possibility that CONFIG_SITE may be set. I’m pasting a part of my analysis from #108819.

    Since CONFIG_SITE is set to /usr/share/site/x86_64-pc-linux-gnu, said file is sourced (this has the effect of setting libdir to '${exec_prefix}/lib64'), and the makefile generated contains, among other things:

    LIBDIR=		${exec_prefix}/lib64
    PLATLIBDIR=	lib
    

    which I think is the reason lib-dynload went in the wrong location:

    BINLIBDEST=	$(LIBDIR)/python$(VERSION)
    DESTSHARED=	$(BINLIBDEST)/lib-dynload
    

    while the remaining files, which depend on PLATLIBDIR (which is hard-coded in configure.ac) went in the expected location.

    Reacted by Filipe Laíns

    Reacted by Filipe Laíns
16. FFY00 commented on Apr 29, 2025

    

    FFY00

    on Apr 29, 2025

    Member

    More actions

    I opened GH-133163, which might fix this issue. @intel-hm, if you could test it, it'd be appreciated.
17. intel-hm commented on Apr 29, 2025

    

    intel-hm

    on Apr 29, 2025

    Author

    More actions

    if you give me the right git commands to download the correct files (my git knowledge is barely existant....)
18. FFY00 commented on Apr 29, 2025

    

    FFY00

    on Apr 29, 2025

    Member

    More actions

    if you give me the right git commands to download the correct files (my git knowledge is barely existant....)

    In that case, I'd recommend you use the Github CLI, which simplifies this quite significantly 😊

    Using it, it should be as straightforward as:

    $ gh pr checkout 133163

    Otherwise, you'll have to manually add a remote for my fork, and then checkout the correct branch.

    $ git remote add ffy00 https://github.com/FFY00/cpython.git
    $ git fetch ffy00
    $ git checkout gh-108819
19. intel-hm commented on Apr 29, 2025

    

    intel-hm

    on Apr 29, 2025

    Author

    More actions

    wouldn't I also need a git clone? Github CLI is not really an option

    $ git clone https://github.com/FFY00/cpython.git
    $ git remote add ffy00 https://github.com/FFY00/cpython.git
    $ git fetch ffy00
    $ git checkout gh-108819

    Reacted by Filipe Laíns
20. FFY00 commented on Apr 29, 2025

    

    FFY00

    on Apr 29, 2025

    Member

    More actions

    You can either clone the main repo and add my fork as a remote, or clone my fork directly, then you can checkout my gh-108819 branch.
21. mcepl commented on Apr 29, 2025

    

    mcepl

    on Apr 29, 2025

    Contributor

    More actions

    I obviously don’t see anything like this with our system packages:

    Python 3.13.3 (main, Apr 11 2025, 19:56:42) [GCC]
    Type 'copyright', 'credits' or 'license' for more information
    IPython 8.31.0 -- An enhanced Interactive Python. Type '?' for help.
    In [1]: import sys
    
    In [2]: sys.path
    Out[2]: 
    ['/usr/bin',
     '/usr/lib64/python313.zip',
     '/usr/lib64/python3.13',
     '/usr/lib64/python3.13/lib-dynload',
     '',
     '/home/matej/.local/lib/python3.13/site-packages',
     '/usr/lib/python3.13/site-packages/setuptools/_vendor',
     '/home/matej/archiv/knihovna/repos/osc',
     '/home/matej/archiv/2024/SUSE/projekty/maintenance-toolkit',
     '/home/matej/archiv/knihovna/repos/epy/src',
     '/usr/lib64/python3.13/site-packages',
     '/usr/lib64/python3.13/site-packages/PIL',
     '/usr/lib64/python3.13/_import_failed',
     '/usr/lib/python3.13/site-packages']
    
    In [3]: 
    

    However, we have plenty of patches and configuration. Perhaps looking at https://build.opensuse.org/package/show/devel:languages:python:Factory/python313 would show something interesting. However, to be honest, I don’t care that much about user-build builds; I think we are doing pretty good job in packaging Python ourselves and we support those.
22. FFY00 commented on Apr 29, 2025

    

    FFY00

    on Apr 29, 2025

    Member

    More actions

    That'd be because you set --with-platlibdir to match the system libdir, so no mismatch would happen there.
23. intel-hm commented on Apr 29, 2025

    

    intel-hm

    on Apr 29, 2025

    Author

    More actions

    looks good:

    git clone https://github.com/python/cpython.git
    cd cpython/
    git remote add ffy00 https://github.com/FFY00/cpython.git
    git fetch ffy00
    git checkout gh-108819
    
    ./configure --prefix=/lfs/lfs19/python/3.12.7
    make
    make install
    
    ~/cpython> find /lfs/lfs19/python/3.12.7/ -name lib-dynload
    /lfs/lfs19/python/3.12.7/lib/python3.14/lib-dynload
    
    ~/cpython> /lfs/lfs19/python/3.12.7/bin/python3
    Python 3.14.0a7+ (heads/gh-108819:a2b43c9d137, Apr 29 2025, 12:15:56) [GCC 7.5.0] on linux
    Type "help", "copyright", "credits" or "license" for more information.
    >>> from _posixsubprocess import fork_exec as _fork_exec
    >>>
    
    

    Reacted by Filipe Laíns
24. picnixz commented on Jun 14, 2025

    

    picnixz

    on Jun 14, 2025

    Member

    More actions

    FTR, I usually solve the installation issue by supplying --with-platlibdir=lib64 since the default one falls back to lib instead of lib64.

Issue actions