Description

abravalheri

opened on Apr 19, 2023

Issue body actions

Bug report

I noticed recently that Setuptools CI started failing for Python 3.12.0a7 with a SystemError that is difficult to understand:

SystemError: <built-in method startswith of str object at 0x7fe399991298> returned a result with an exception set

At a first glance this error does not make much sense because str.startswith is a built-in method in a built-in data structure. I am struggling to see how it can have an exception set attached to the result.

I managed to create the following simplified reproducer:

docker run --rm -it python:3.12.0a7-bullseye
Python 3.12.0a7 (main, Apr 12 2023, 14:13:09) [GCC 10.2.1 20210110] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> class Distribution:
...     @property
...     def version(self):
...         try:
...             return self._version
...         except AttributeError as e:
...             # ... complex code omitted for the sake of simplification ...
...             raise ValueError("Missing Version")
...     def __getattr__(self, attr):
...         if attr.startswith("_"):
...             raise AttributeError(attr)
...         # ... complex code omitted for the sake of simplification ...
...         return 42
...
>>> dist = Distribution()
>>> dist.version
Traceback (most recent call last):
  File "<stdin>", line 5, in version
  File "<stdin>", line 11, in __getattr__
AttributeError: _version. Did you mean: 'version'?

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<stdin>", line 8, in version
ValueError: Missing Version

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<stdin>", line 10, in __getattr__
SystemError: <method 'startswith' of 'str' objects> returned a result with an exception set
>>>

The error seems to be related to the combination of the following factors:

1. A property is being used¹
2. The property getter access an undefined attribute and triggers __getattr__
3. __getattr__ raises an AttributeError
4. The property getter is supposed to catch the AttributeError exception and raise a different exeception.

My expectation is that the SystemError never gets triggered with the confusing message. Instead the example should end up with a ValueError.

Please note that the example is very simplified. For the realistic implementation please consider pkg_resources.Distribution.

Your environment

• CPython versions tested on: 3.12.0a7
• Operating system and architecture:
  Reproducer tested on both Ubuntu 20.04.6 LTS machine and python:3.12.0a7-bullseye container

Problem also identified in GitHub Actions runners: ubuntu-latest, macos-latest, windows-latest (see https://github.com/pypa/setuptools/actions/runs/4710601389?pr=3893) for Python 3.12-dev.

Footnotes

1. If we replace the property with a regular method, the example works as expected. ↩

Activity

1. abravalheri

   added

   type-bugAn unexpected behavior, bug, or errorAn unexpected behavior, bug, or error

   on Apr 19, 2023
2. sunmy2019 commented on Apr 20, 2023

   

   sunmy2019

   on Apr 20, 2023

   Member

   More actions

   It should be fixed by #103332

   Reacted by Shantanu
3. sobolevn commented on Apr 21, 2023

   

   sobolevn

   on Apr 21, 2023

   Member

   More actions

   I agree that this should be fixed in main. Please, reopen if it is not fixed.
4. sobolevn

   closed this as not plannedon Apr 21, 2023
5. abravalheri commented on Apr 21, 2023

   

   abravalheri

   on Apr 21, 2023

   Author

   More actions

   Thank you very much.

   I run the test above with an adapted Dockerfile that gets the latest `main` and it seems to work

   FROM buildpack-deps:bullseye
   
   # ensure local python is preferred over distribution python
   ENV PATH /usr/local/bin:$PATH
   
   # http://bugs.python.org/issue19846
   # > At the moment, setting "LANG=C" on a Linux system *fundamentally breaks Python 3*, and that's not OK.
   ENV LANG C.UTF-8
   
   # runtime dependencies
   RUN set -eux; \
       apt-get update; \
       apt-get install -y --no-install-recommends \
           libbluetooth-dev \
           tk-dev \
           uuid-dev \
       ; \
       rm -rf /var/lib/apt/lists/*
   
   ENV PYTHON_VERSION 3.12.0dev+3e0fec7
   
   
   RUN set -eux; \
       \
       wget -O /tmp/python.tar.gz "https://github.com/python/cpython/tarball/3e0fec7e07a71bdeeab7554e980110fbc47763b9"; \
       mkdir -p /usr/src/python; \
       tar --extract --directory /usr/src/python --strip-components=1 --file /tmp/python.tar.gz; \
       rm /tmp/python.tar.gz; \
       \
       cd /usr/src/python; \
       gnuArch="$(dpkg-architecture --query DEB_BUILD_GNU_TYPE)"; \
       ./configure \
           --build="$gnuArch" \
           --enable-loadable-sqlite-extensions \
           --enable-optimizations \
           --enable-option-checking=fatal \
           --enable-shared \
           --with-lto \
           --with-system-expat \
           --without-ensurepip \
       ; \
       nproc="$(nproc)"; \
       EXTRA_CFLAGS="$(dpkg-buildflags --get CFLAGS)"; \
       LDFLAGS="$(dpkg-buildflags --get LDFLAGS)"; \
       make -j "$nproc" \
           "EXTRA_CFLAGS=${EXTRA_CFLAGS:-}" \
           "LDFLAGS=${LDFLAGS:-}" \
           "PROFILE_TASK=${PROFILE_TASK:-}" \
       ; \
   # https://github.com/docker-library/python/issues/784
   # prevent accidental usage of a system installed libpython of the same version
       rm python; \
       make -j "$nproc" \
           "EXTRA_CFLAGS=${EXTRA_CFLAGS:-}" \
           "LDFLAGS=${LDFLAGS:--Wl},-rpath='\$\$ORIGIN/../lib'" \
           "PROFILE_TASK=${PROFILE_TASK:-}" \
           python \
       ; \
       make install; \
       \
   # enable GDB to load debugging data: https://github.com/docker-library/python/pull/701
       bin="$(readlink -ve /usr/local/bin/python3)"; \
       dir="$(dirname "$bin")"; \
       mkdir -p "/usr/share/gdb/auto-load/$dir"; \
       cp -vL Tools/gdb/libpython.py "/usr/share/gdb/auto-load/$bin-gdb.py"; \
       \
       cd /; \
       rm -rf /usr/src/python; \
       \
       find /usr/local -depth \
           \( \
               \( -type d -a \( -name test -o -name tests -o -name idle_test \) \) \
               -o \( -type f -a \( -name '*.pyc' -o -name '*.pyo' -o -name 'libpython*.a' \) \) \
           \) -exec rm -rf '{}' + \
       ; \
       \
       ldconfig; \
       \
       python3 --version
   
   CMD ["python3"]

   Ref: https://github.com/docker-library/python/blob/cb6ecdbcbc7e30a180481a9911f5f1f470b9dba1/3.12-rc/bullseye/Dockerfile

Issue actions