GeezerVison is showing github.com — Create a release branch ABI stability regression test · Issue #87891 · python/cpython
Create a release branch ABI stability regression test #87891Description
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
assignee = None
closed_at = None
created_at = <Date 2021-04-04.04:35:13.558>
labels = ['build', '3.8', '3.9', 'expert-C-API', 'type-feature', 'tests', 'interpreter-core', '3.10']
title = 'Create a release branch ABI stability regression test'
updated_at = <Date 2021-10-15.14:16:28.469>
user = 'https://github.com/gpshead' bugs.python.org fields:
activity = <Date 2021-10-15.14:16:28.469>
actor = 'pablogsal'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = ['Build', 'Interpreter Core', 'Tests', 'C API']
creation = <Date 2021-04-04.04:35:13.558>
creator = 'gregory.p.smith'
dependencies = []
files = ['49935']
hgrepos = []
issue_num = 43725
keywords = ['patch']
message_count = 36.0
messages = ['390173', '390184', '390203', '390205', '390207', '390208', '390213', '390214', '390310', '390311', '390312', '390313', '390315', '390316', '390319', '390325', '390372', '390374', '390382', '390660', '390668', '390670', '390985', '390986', '390987', '390994', '391007', '391008', '391009', '391010', '391011', '392205', '392207', '392208', '404019', '404022']
nosy_count = 8.0
nosy_names = ['gregory.p.smith', 'vstinner', 'christian.heimes', 'ned.deily', 'petr.viktorin', 'lukasz.langa', 'steve.dower', 'pablogsal']
pr_nums = ['25188', '25230', '25232', '25322', '25323', '25394']
priority = 'normal'
resolution = None
stage = 'patch review'
status = 'open'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue43725'
versions = ['Python 3.8', 'Python 3.9', 'Python 3.10'] Linked PRs
Activitygpshead commented on Apr 4, 2021 In order to automate prevention of ABI regressions in stable releases, we could create an automated ABI stability test generator and check the specific ABI test it generates into each specific release branch.
I'm envisioning the main branch only having a code generator that creates such a test, and the release branches only having the output of that as Lib/tests/release_X_Y_ABI_stability_test.py and a policy of never updating that within a release branch without *extreme* attention to detail.
Such updates wouldn't happen by default in our current workflow as they're unique and versioned in every release branch so automated backport PRs wouldn't touch them - leaving CI to run them and highlight failures on attempted backports to do inadvertently cause an ABI shift. pablogsal commented on Apr 13, 2021 Ok, then we need to revert by latest 2 PRs and add a new label or something to skip vstinner commented on Apr 13, 2021
Ok, then we need to revert by latest 2 PRs and add a new label or something to skip
To skip ABI tests? I proposed to ignore ABI changes if they can be ignored safely. But I don't know right now which changes can be ignored or not. pablogsal commented on Apr 13, 2021
To skip ABI tests? I proposed to ignore ABI changes if they can be ignored safely. But I don't know right now which changes can be ignored or not.
As the situation stands we need to choose on keeping my changes to ignore _Py function changes or revert it. Further than that is unexplored land pablogsal commented on Apr 13, 2021
$ objdump -T /lib64/libpython3.10.so.1.0|grep _Py_M__importlib_bootstrap_external
# empty output
This was happening on 3.8 Victor:
root@d9c5942e274b:/src# objdump -T libpython3.8.so | grep _Py_M__importlib_bootstrap_external
0000000000335740 g DO .rodata 000000000000b062 Base _Py_M__importlib_bootstrap_external pablogsal commented on Apr 13, 2021 I am going to revert the 3.9 PR as 3.9+ is hiding symbols by default. vstinner commented on Apr 13, 2021 vstinner commented on Apr 28, 2021 I cannot merge my PR 25685 because the mandatory ABI CI job fails:
---------------
abidiff "libpython3.9.so" ./Doc/data/python3.9.abi --drop-private-types --no-architecture --no-added-syms
Functions changes summary: 0 Removed, 1 Changed, 0 Added function
Variables changes summary: 0 Removed, 0 Changed, 0 Added variable
1 function with some indirect sub-type change:
[C]'function int _PyInterpreterState_IDIncref(PyInterpreterState*)' at pystate.c:497:1 has some indirect sub-type changes:
return type changed:
type name changed from 'int' to 'void'
type size changed from 32 to 0 (in bits)
---------------
It is correct that my PR changes an internal C API on purpose.
Pablo suggests me to regenerate the ABI file but I don't know how to do that.
In Python 3.9, the GitHub Action uses:
---
check_abi:
name: 'Check if the ABI has changed'
runs-on: ubuntu-20.04
needs: check_source
if: needs.check_source.outputs.run_tests == 'true'
steps:
- uses: actions/checkout@v2
- uses: actions/setup-python@v2
- name: Install Dependencies
run: |
sudo ./.github/workflows/posix-deps-apt.sh
sudo apt-get install -yq abigail-tools
- name: Build CPython
env:
CFLAGS: -g3 -O0
run: |
# Build Python with the libpython dynamic library
./configure --enable-shared
make -j4
- name: Check for changes in the ABI
run: make check-abidump
---
I'm using Fedora 33 with "gcc (GCC) 11.0.1 20210324 (Red Hat 11.0.1-0)".
On Fedora, I used:
---
$ sudo dnf install -y libabigail
$ ./configure --enable-shared CFLAGS="-g3 -O0" && make -j10
$ make regen-abidump
$ git diff --stat
Doc/data/python3.9.abi | 28478 +++++++++++++++++++++++++++++++++
1 file changed, 18306 insertions(+), 10172 deletions(-)
---
There are tons of changes!
Also, "make check-abidump" lists many changes:
---
abidiff "libpython3.9.so" ./Doc/data/python3.9.abi --drop-private-types --no-architecture --no-added-syms
Functions changes summary: 0 Removed, 13 Changed (171 filtered out), 0 Added functions
Variables changes summary: 0 Removed, 6 Changed (2 filtered out), 0 Added variables
13 functions with some indirect sub-type change:
(...)
[C] 'function PyStatus _PyInterpreterState_Enable(_PyRuntimeState*)' at pystate.c:171:1 has some indirect sub-type changes:
[C] 'function int _Py_DecodeLocaleEx(const char*, wchar_t**, size_t*, const char**, int, _Py_error_handler)' at fileutils.c:574:1 has some indirect sub-type changes:
parameter 6 of type 'typedef _Py_error_handler' has sub-type changes:
[C] 'const unsigned char _Py_ctype_tolower[256]' was changed to 'const unsigned char[256] const _Py_ctype_tolower' at pyctype.h:26:1:
type of variable changed:
entity changed from 'const unsigned char[256]' to 'const unsigned char[256] const'
(...)
--- pablogsal commented on Apr 28, 2021 pablogsal commented on Apr 28, 2021 You are using Fedora, which is not the same docker container and likely the same compiler version that is used to check the dump vstinner commented on Oct 15, 2021
the dump needs to be generated in a docker container using the same compiler version that is used in the CI
I'm not used to docker and I don't know how to get a docker similar than the one used by GitHub Action. Is there a documentation somewhere giving commands to get the docker image and how to run it? pablogsal commented on Oct 15, 2021 I will add something to the devguide explaining how to do it. In any case, the RM should be ideally the one doing this because every potential ABI breakage need to be supervised by the RM (even if is a false positive). CAM-Gerlach commented on May 23, 2023 Can this be closed now? Seems like this was implemented (came across this seeing that it apparently needs to be added for 3.12 just now).
MetadataMetadataDevelopmentNo branches or pull requestsIssue actions
|