Description

relativisticelectron

opened on May 23, 2022

Issue body actions

Bug report

The following script

import ctypes
from ctypes import  util

print(f'ctypes version {ctypes.__version__}')


print('Test foofoofoo')
print(f"returns: {util.find_library('foofoofoo')}\n")

print('Test secp256k1')
print(f"returns: {util.find_library('secp256k1')}\n")

print('Test libsecp256k1')
print(f"returns: {util.find_library('libsecp256k1')}\n")

gives the output:

ctypes version 1.1.0
Test foofoofoo
returns: None

Test secp256k1
returns: libsecp256k1.so.0

Test libsecp256k1
Traceback (most recent call last):
  File "/home/<user>/programs/testctypes.py", line 14, in <module>
    print(f"returns: {util.find_library('libsecp256k1')}\n")
  File "/home/<user>/programs/anaconda3/envs/python310/lib/python3.10/ctypes/util.py", line 351, in find_library
    _get_soname(_findLib_gcc(name)) or _get_soname(_findLib_ld(name))
  File "/home/<user>/programs/anaconda3/envs/python310/lib/python3.10/ctypes/util.py", line 148, in _findLib_gcc
    if not _is_elf(file):
  File "/home/<user>/programs/anaconda3/envs/python310/lib/python3.10/ctypes/util.py", line 100, in _is_elf
    with open(filename, 'br') as thefile:
FileNotFoundError: [Errno 2] No such file or directory: b'liblibsecp256k1.a'

find_library should not fail, but return None if nothing was found. In Ubuntu 20.04 this script runs without any exception.

Your environment

• ctypes version 1.1.0
• Operating system and architecture: Kubuntu 22.04 LTS x86_64, linux 5.15.0-27-generic
• Bug can be reproduced in python versions: python 3.10 (via anaconda and via ubuntu deb repositories), python 3.8 (via anaconda)
• The installed libsecp256k1 packages are
  

contains the files

/.
/usr
/usr/lib
/usr/lib/x86_64-linux-gnu
/usr/lib/x86_64-linux-gnu/libsecp256k1.so.0
/usr/lib/x86_64-linux-gnu/libsecp256k1.so.0.0.0
/usr/share
/usr/share/doc
/usr/share/doc/libsecp256k1-0
/usr/share/doc/libsecp256k1-0/changelog.Debian.gz
/usr/share/doc/libsecp256k1-0/copyright

And contains the files

/.
/usr
/usr/include
/usr/include/secp256k1.h
/usr/include/secp256k1_ecdh.h
/usr/include/secp256k1_extrakeys.h
/usr/include/secp256k1_preallocated.h
/usr/include/secp256k1_recovery.h
/usr/include/secp256k1_schnorrsig.h
/usr/lib
/usr/lib/x86_64-linux-gnu
/usr/lib/x86_64-linux-gnu/libsecp256k1.a
/usr/lib/x86_64-linux-gnu/libsecp256k1.so
/usr/lib/x86_64-linux-gnu/pkgconfig
/usr/lib/x86_64-linux-gnu/pkgconfig/libsecp256k1.pc
/usr/share
/usr/share/doc
/usr/share/doc/libsecp256k1-dev
/usr/share/doc/libsecp256k1-dev/changelog.Debian.gz
/usr/share/doc/libsecp256k1-dev/copyright

And in /usr/lib/x86_64-linux-gnu/ are the following relevant files containing secp256k1:

This bug was noticed in cryptoadvance/specter-desktop#1710

Activity

1. relativisticelectron

   added

   type-bugAn unexpected behavior, bug, or errorAn unexpected behavior, bug, or error

   on May 23, 2022
2. erlend-aasland

   added

   topic-ctypes

   on May 23, 2022
3. relativisticelectron commented on Jun 1, 2022

   

   relativisticelectron

   on Jun 1, 2022

   Author

   More actions

   When I do a print(trace) just before
   

   cpython/Lib/ctypes/util.py

   Line 139 in 8241a69

   res = re.findall(expr, trace)

   
   I get

   b'/usr/lib/gcc/x86_64-linux-gnu/11/../../../x86_64-linux-gnu/Scrt1.o\n/usr/lib/gcc/x86_64-linux-gnu/11/../../../x86_64-linux-gnu/crti.o\n/usr/lib/gcc/x86_64-linux-gnu/11/crtbeginS.o\n/usr/bin/ld: cannot find -llibsecp256k1: No such file or directory\n/usr/bin/ld: note to link with /usr/lib/gcc/x86_64-linux-gnu/11/../../../x86_64-linux-gnu/libsecp256k1.a use -l:libsecp256k1.a or rename it to liblibsecp256k1.a\n/usr/lib/gcc/x86_64-linux-gnu/11/libgcc.a\n/usr/lib/gcc/x86_64-linux-gnu/11/libgcc_s.so\n/usr/lib/gcc/x86_64-linux-gnu/11/../../../x86_64-linux-gnu/libgcc_s.so.1\n/usr/lib/gcc/x86_64-linux-gnu/11/libgcc.a\ncollect2: error: ld returned 1 exit status\n'
   
   

   here with replaced \n

   
   b'/usr/lib/gcc/x86_64-linux-gnu/11/../../../x86_64-linux-gnu/Scrt1.o
   /usr/lib/gcc/x86_64-linux-gnu/11/../../../x86_64-linux-gnu/crti.o
   /usr/lib/gcc/x86_64-linux-gnu/11/crtbeginS.o
   /usr/bin/ld: cannot find -llibsecp256k1: No such file or directory
   /usr/bin/ld: note to link with /usr/lib/gcc/x86_64-linux-gnu/11/../../../x86_64-linux-gnu/libsecp256k1.a use -l:libsecp256k1.a or rename it to liblibsecp256k1.a
   /usr/lib/gcc/x86_64-linux-gnu/11/libgcc.a
   /usr/lib/gcc/x86_64-linux-gnu/11/libgcc_s.so
   /usr/lib/gcc/x86_64-linux-gnu/11/../../../x86_64-linux-gnu/libgcc_s.so.1
   /usr/lib/gcc/x86_64-linux-gnu/11/libgcc.a
   collect2: error: ld returned 1 exit status
   '
   
   

   My interpretation of this is, that the re logic in

   cpython/Lib/ctypes/util.py

   Line 139 in 8241a69

   res = re.findall(expr, trace)

   is not sufficient to handle the ld message.
4. cz6ace commented on Oct 12, 2022

   

   cz6ace

   on Oct 12, 2022

   More actions

   I face the same issue, when the output is polluted with message from ld, in my case:
   /usr/bin/ld: note to link with /usr/lib/gcc/x86_64-linux-gnu/11/../../../x86_64-linux-gnu/libusb-1.0.a use -l:libusb-1.0.a or rename it
   reproduced with using pyInstaller package, but can be reproduced easily, see below.

   Environment:
   Ubuntu 22.04 LTS
   uname: Linux ubu22-virtual 5.15.0-50-generic #56-Ubuntu SMP Tue Sep 20 13:23:26 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
   Python: 3.10.6

   Steps to reproduce:

   sudo apt install libusb-1.0-0-dev
   /usr/bin/gcc -Wl,-t -o /tmp/tmpp455oz01 -llibusb-1.0
   
   

   outputting

   /usr/lib/gcc/x86_64-linux-gnu/11/../../../x86_64-linux-gnu/Scrt1.o
   /usr/lib/gcc/x86_64-linux-gnu/11/../../../x86_64-linux-gnu/crti.o
   /usr/lib/gcc/x86_64-linux-gnu/11/crtbeginS.o
   /usr/bin/ld: cannot find -llibusb-1.0: No such file or directory
   /usr/bin/ld: note to link with /usr/lib/gcc/x86_64-linux-gnu/11/../../../x86_64-linux-gnu/libusb-1.0.a use -l:libusb-1.0.a or rename it to liblibusb-1.0.a
   /usr/lib/gcc/x86_64-linux-gnu/11/libgcc.a
   /usr/lib/gcc/x86_64-linux-gnu/11/libgcc_s.so
   /usr/lib/gcc/x86_64-linux-gnu/11/../../../x86_64-linux-gnu/libgcc_s.so.1
   /usr/lib/gcc/x86_64-linux-gnu/11/libgcc.a
   collect2: error: ld returned 1 exit status
   

   in python3

   from ctypes import util
   util.find_library('libusb-1.0')
   
   Traceback (most recent call last):
     File "<stdin>", line 1, in <module>
     File "/usr/lib/python3.10/ctypes/util.py", line 341, in find_library
       _get_soname(_findLib_gcc(name)) or _get_soname(_findLib_ld(name))
     File "/usr/lib/python3.10/ctypes/util.py", line 147, in _findLib_gcc
       if not _is_elf(file):
     File "/usr/lib/python3.10/ctypes/util.py", line 99, in _is_elf
       with open(filename, 'br') as thefile:
   FileNotFoundError: [Errno 2] No such file or directory: b'liblibusb-1.0.a'
   

   If the gcc produces only the list of files, then prepending ^ as looking for whole line could fix the issue.
   expr = os.fsencode(r'[^\(\)\s]*lib%s\.[^\(\)\s]*' % re.escape(name))
   to
   expr = os.fsencode(r'^[^\(\)\s]*lib%s\.[^\(\)\s]*' % re.escape(name))
5. serhiy-storchaka

   added this to Ctypes issueson Jan 1, 2024
6. encukou commented on Feb 23, 2026

   

   encukou

   on Feb 23, 2026

   Member

   More actions

   I'm proposing to soft-deprecate ctypes.util.find_library: https://discuss.python.org/t/106232
7. encukou commented on Mar 20, 2026

   

   encukou

   on Mar 20, 2026

   Member

   More actions

   It is now soft-deprecated. It cannot be made to work correctly and consistently on all platforms, and any change runs a chance to break whatever workarounds people already have in place.
   I'm prioritizing stability over new features, and encourage solving this outside ctypes itself.
8. encukou

   closed this as not plannedon Mar 20, 2026
9. github-project-automation

   moved this to Done in Ctypes issueson Mar 20, 2026

Issue actions