GeezerVison is showing github.com — Py_CLEAR/Py_SETREF() regression on the main branch with C++: SEGV with simple `numpy` program · Issue #157649 · python/cpython
Py_CLEAR/Py_SETREF() regression on the main branch with C++: SEGV with simple numpy program #157649Labels interpreter-core(Objects, Python, Grammar, and Parser dirs)(Objects, Python, Grammar, and Parser dirs)type-bugAn unexpected behavior, bug, or errorAn unexpected behavior, bug, or error DescriptionBug report
Bug description:
The program that triggers the crash:
import numpy
for _ in range(200):
numpy.prod(2, dtype=numpy.intp)Reproducer at https://github.com/lucascolley/cpython-np-prod-repro. This produces the SEGV on CPython main branch, but passes on the 3.15 branch.
❯ export BUILD_EDITABLE_PYTHON=false
❯ pixi run repro
✨ Pixi task (repro): python repro.py
Python: 3.16.0a0 (heads/master-dirty:d8a1072491, Sep 16 2026, 22:19:03) [Clang 21.1.8 ]
AddressSanitizer:DEADLYSIGNAL
=================================================================
==54023==ERROR: AddressSanitizer: SEGV on unknown address 0x000000000000 (pc 0x00010639c214 bp 0x00016afdf230 sp 0x00016afdf1d0 T0)
==54023==The signal is caused by a WRITE memory access.
==54023==HINT: address points to the zero page.
#0 0x00010639c214 in _PyGCHead_SET_NEXT pycore_gc.h:151
#1 0x00010639c214 in _PyObject_GC_UNTRACK pycore_gc.h:264
#2 0x00010639c214 in type_dealloc typeobject.c:6898
#3 0x000106338e40 in _Py_Dealloc object.c:3333
#4 0x00010e880a34 in PyUFunc_GenericReduction+0x119c (_multiarray_umath.cpython-316-darwin.so:arm64+0x704a34)
#5 0x000106237858 in _PyVectorcall_Call call.c:273
#6 0x00010652b77c in _PyEval_EvalFrameDefault generated_cases.c.h:2831
#7 0x0001065153bc in _PyEval_EvalFrame pycore_ceval.h:122
#8 0x0001065153bc in _PyEval_Vector ceval.c:2176
#9 0x000106237a78 in _PyObject_VectorcallTstate pycore_call.h:144
#10 0x000106237a78 in PyObject_Vectorcall call.c:327
#11 0x00010e54bd1c in dispatcher_vectorcall+0xc70 (_multiarray_umath.cpython-316-darwin.so:arm64+0x3cfd1c)
#12 0x000106237a78 in _PyObject_VectorcallTstate pycore_call.h:144
#13 0x000106237a78 in PyObject_Vectorcall call.c:327
#14 0x000106515938 in _Py_VectorCall_StackRefSteal ceval.c:728
#15 0x000106529170 in _PyEval_EvalFrameDefault generated_cases.c.h:3686
#16 0x000106514db0 in _PyEval_EvalFrame pycore_ceval.h:122
#17 0x000106514db0 in _PyEval_Vector ceval.c:2176
#18 0x000106514db0 in PyEval_EvalCode ceval.c:681
#19 0x00010665ba10 in run_mod pythonrun.c:1509
#20 0x00010665633c in _PyRun_File pythonrun.c:1332
#21 0x00010665633c in _PyRun_SimpleFile pythonrun.c:544
#22 0x000106655998 in _PyRun_AnyFile pythonrun.c:92
#23 0x0001066bd0b0 in pymain_run_file_obj main.c:478
#24 0x0001066bd0b0 in pymain_run_file main.c:494
#25 0x0001066bbe68 in pymain_run_python main.c:804
#26 0x0001066bbe68 in Py_RunMain main.c:891
#27 0x0001066bc830 in pymain_main main.c:921
#28 0x0001066bc9e0 in Py_BytesMain main.c:945
#29 0x000189febdfc in start+0x1b4c (dyld:arm64e+0x1fdfc)
==54023==Register values:
x[0] = 0x00006190000dde88 x[1] = 0x000000010639c054 x[2] = 0x000000016afde9d0 x[3] = 0x000061600009a870
x[4] = 0x0000000000000218 x[5] = 0x0000000000000218 x[6] = 0x0000000169fe4000 x[7] = 0x0000000000000001
x[8] = 0x00006190000dde80 x[9] = 0x0000000000000000 x[10] = 0x0000000000000000 x[11] = 0x00000fffffffffff
x[12] = 0x0000000000000000 x[13] = 0xffffffffffffffff x[14] = 0x0000000000000000 x[15] = 0x00007fffffffffff
x[16] = 0x0000000000000028 x[17] = 0x0000604000000410 x[18] = 0x0000000000000000 x[19] = 0x00006190000dde90
x[20] = 0x0000604000000438 x[21] = 0x0000000080805980 x[22] = 0x0000008000020000 x[23] = 0x0000000106c42940
x[24] = 0x0000008000020000 x[25] = 0x0000000000000000 x[26] = 0x000000016afdfb20 x[27] = 0x0000000000000000
x[28] = 0x00006110001f42e4 fp = 0x000000016afdf230 lr = 0x000000010639c1d8 sp = 0x000000016afdf1d0
AddressSanitizer can not provide additional info.
SUMMARY: AddressSanitizer: SEGV pycore_gc.h:151 in _PyGCHead_SET_NEXT
==54023==ABORTING@ngoldbaum perhaps you can reproduce?
[workspace]
channels = ["https://prefix.dev/conda-forge"]
platforms = ["osx-arm64"]
preview = ["pixi-build"]
requires-pixi = ">=0.74.0"
[tasks]
repro = "python repro.py"
[dependencies]
numpy = {
git = "https://github.com/numpy/numpy",
package = {
host-dependencies = {
python = {
flags = ["asan"],
# passing with 3.15 branch
# branch = "3.15",
# failing with main
branch = "main",
git = "https://github.com/python/cpython",
subdirectory = "Tools/pixi-packages",
},
meson-python = "*",
cython = "*",
},
build = {
backend.name = "pixi-build-python",
config.compilers = ["c", "cxx"],
},
},
}CPython versions tested on:
CPython main branch
Operating systems tested on:
macOS
Linked PRs
Activitylucascolley commented on Sep 16, 2026 ContributorAuthor More actions ngoldbaum commented on Sep 16, 2026 lucascolley commented on Sep 16, 2026 ContributorAuthor More actions runtime packages:
Details
cpython-np-prod-repro on 🎋 main via 🧚 v0.81.0
❯ pixi list
Installed for: osx-arm64
Name Version Build Size Kind Source
bzip2 1.0.8 h4e30115_10 122.04 KiB conda https://prefix.dev/conda-forge
ca-certificates 2026.7.22 hbd8a1cb_0 128.69 KiB conda https://prefix.dev/conda-forge
icu 78.3 py310h579977c_2 13.42 MiB conda https://prefix.dev/conda-forge
libcompiler-rt 23.1.1 hdb3d66b_0 1.32 MiB conda https://prefix.dev/conda-forge
libcxx 23.1.1 h55c6f16_0 565.00 KiB conda https://prefix.dev/conda-forge
libexpat 2.8.1 hf6b4638_1 67.74 KiB conda https://prefix.dev/conda-forge
libffi 3.7.0 h47dc5ef_1 42.61 KiB conda https://prefix.dev/conda-forge
liblzma 5.8.3 h8088a28_1 89.57 KiB conda https://prefix.dev/conda-forge
libmpdec 4.0.0 h84a0fba_2 71.57 KiB conda https://prefix.dev/conda-forge
libsqlite 3.53.4 hca69786_1 920.66 KiB conda https://prefix.dev/conda-forge
libzlib 1.3.2 h8088a28_3 46.70 KiB conda https://prefix.dev/conda-forge
ncurses 6.6 he64c551_1 785.45 KiB conda https://prefix.dev/conda-forge
numpy 2.6.0.dev0 hea27dcd_0 conda git+https://github.com/numpy/numpy#9e5c44c061f04992033e852aadc405b5b3b31564
openssl 3.6.4 h55eecbc_0 2.97 MiB conda https://prefix.dev/conda-forge
python 3.16 0_asan_cp316 conda git+https://github.com/python/cpython?subdirectory=Tools%2Fpixi-packages&rev=bba99f2b3272cbe47af9bee05a294961c0528829#bba99f2b3272cbe47af9bee05a294961c0528829
python_abi 3.16 0_asan_cp316 conda git+https://github.com/python/cpython?subdirectory=Tools%2Fpixi-packages&rev=bba99f2b3272cbe47af9bee05a294961c0528829#bba99f2b3272cbe47af9bee05a294961c0528829
readline 8.3 h8b90a29_1 307.03 KiB conda https://prefix.dev/conda-forge
zstd 1.5.7 hf451053_7 423.52 KiB conda https://prefix.dev/conda-forge
CPython build-time deps info:
Details
│ │ Resolved build dependencies(python-3.16-0_asan_cp316):
│ │ ╭─────────────────────────┬────────────────────────┬─────────────┬─────────────────────┬─────────────┬────────────╮
│ │ │ Package ┆ Spec ┆ Version ┆ Build ┆ Channel ┆ Size │
│ │ ╞═════════════════════════╪════════════════════════╪═════════════╪═════════════════════╪═════════════╪════════════╡
│ │ │ clang_osx-arm64 ┆ clang_osx-arm64 21.* ┆ 21.1.8 ┆ h15b5089_34 ┆ conda-forge ┆ 20.20 KiB │
│ │ │ clangxx_osx-arm64 ┆ clangxx_osx-arm64 21.* ┆ 21.1.8 ┆ h15b5089_34 ┆ conda-forge ┆ 18.91 KiB │
│ │ │ llvm-tools ┆ llvm-tools ┆ 21.1.8 ┆ hdb3d66b_1 ┆ conda-forge ┆ 87.02 KiB │
│ │ │ make ┆ make ┆ 4.4.1 ┆ h84a0fba_3 ┆ conda-forge ┆ 268.66 KiB │
│ │ │ pkg-config ┆ pkg-config ┆ 0.29.2 ┆ he0b766d_1013 ┆ conda-forge ┆ 267.48 KiB │
│ │ │ ca-certificates ┆ ┆ 2026.7.22 ┆ hbd8a1cb_0 ┆ conda-forge ┆ 128.69 KiB │
│ │ │ cctools_impl_osx-arm64 ┆ ┆ 1030.6.3 ┆ llvm21_1_h2e2604f_6 ┆ conda-forge ┆ 744.41 KiB │
│ │ │ cctools_osx-arm64 ┆ ┆ 1030.6.3 ┆ llvm21_1_h8e84c17_6 ┆ conda-forge ┆ 22.95 KiB │
│ │ │ clang-21 ┆ ┆ 21.1.8 ┆ default_h4399b93_6 ┆ conda-forge ┆ 800.13 KiB │
│ │ │ clang-scan-deps ┆ ┆ 21.1.8 ┆ default_h4399b93_6 ┆ conda-forge ┆ 98.90 KiB │
│ │ │ clang_impl_osx-arm64 ┆ ┆ 21.1.8 ┆ default_hc11f16d_6 ┆ conda-forge ┆ 27.69 KiB │
│ │ │ clangxx_impl_osx-arm64 ┆ ┆ 21.1.8 ┆ default_hc11f16d_6 ┆ conda-forge ┆ 27.48 KiB │
│ │ │ compiler-rt ┆ ┆ 21.1.8 ┆ hce30654_2 ┆ conda-forge ┆ 15.86 KiB │
│ │ │ compiler-rt21 ┆ ┆ 21.1.8 ┆ hdb3d66b_2 ┆ conda-forge ┆ 97.24 KiB │
│ │ │ compiler-rt21_osx-arm64 ┆ ┆ 21.1.8 ┆ hb8825d9_2 ┆ conda-forge ┆ 9.75 MiB │
│ │ │ compiler-rt_osx-arm64 ┆ ┆ 21.1.8 ┆ hce30654_2 ┆ conda-forge ┆ 16.02 KiB │
│ │ │ icu ┆ ┆ 78.3 ┆ py310h579977c_2 ┆ conda-forge ┆ 13.42 MiB │
│ │ │ ld64_osx-arm64 ┆ ┆ 956.6 ┆ llvm21_1_h1a10cc4_6 ┆ conda-forge ┆ 961.26 KiB │
│ │ │ libclang-cpp21.1 ┆ ┆ 21.1.8 ┆ default_h4399b93_6 ┆ conda-forge ┆ 13.07 MiB │
│ │ │ libclang-cpp23.1 ┆ ┆ 23.1.1 ┆ default_h79071a4_0 ┆ conda-forge ┆ 15.84 MiB │
│ │ │ libclang13 ┆ ┆ 23.1.1 ┆ default_hfbe3a31_0 ┆ conda-forge ┆ 10.13 MiB │
│ │ │ libcompiler-rt ┆ ┆ 21.1.8 ┆ hdb3d66b_2 ┆ conda-forge ┆ 1.27 MiB │
│ │ │ libcxx ┆ ┆ 23.1.1 ┆ h55c6f16_0 ┆ conda-forge ┆ 565.00 KiB │
│ │ │ libcxx-devel ┆ ┆ 21.1.8 ┆ h6dc3340_3 ┆ conda-forge ┆ 21.58 KiB │
│ │ │ libcxx-headers ┆ ┆ 21.1.8 ┆ h707e725_3 ┆ conda-forge ┆ 1.05 MiB │
│ │ │ libiconv ┆ ┆ 1.18 ┆ he4c29f2_3 ┆ conda-forge ┆ 733.22 KiB │
│ │ │ libllvm21 ┆ ┆ 21.1.8 ┆ h759d1ac_1 ┆ conda-forge ┆ 28.26 MiB │
│ │ │ libllvm23 ┆ ┆ 23.1.1 ┆ h759d1ac_0 ┆ conda-forge ┆ 30.12 MiB │
│ │ │ liblzma ┆ ┆ 5.8.3 ┆ h8088a28_1 ┆ conda-forge ┆ 89.57 KiB │
│ │ │ libsigtool ┆ ┆ 0.1.3 ┆ h98dc951_1 ┆ conda-forge ┆ 35.79 KiB │
│ │ │ libxml2 ┆ ┆ 2.15.4 ┆ h550178d_0 ┆ conda-forge ┆ 40.41 KiB │
│ │ │ libxml2-16 ┆ ┆ 2.15.4 ┆ heb56d2d_0 ┆ conda-forge ┆ 458.30 KiB │
│ │ │ libzlib ┆ ┆ 1.3.2 ┆ h8088a28_3 ┆ conda-forge ┆ 46.70 KiB │
│ │ │ llvm-tools-21 ┆ ┆ 21.1.8 ┆ h79441dc_1 ┆ conda-forge ┆ 17.68 MiB │
│ │ │ ncurses ┆ ┆ 6.6 ┆ he64c551_1 ┆ conda-forge ┆ 785.45 KiB │
│ │ │ openssl ┆ ┆ 3.6.4 ┆ h55eecbc_0 ┆ conda-forge ┆ 2.97 MiB │
│ │ │ sdkroot_env_osx-arm64 ┆ ┆ 26.0 ┆ ha3f98da_7 ┆ conda-forge ┆ 4.85 KiB │
│ │ │ sigtool-codesign ┆ ┆ 0.1.3 ┆ h98dc951_1 ┆ conda-forge ┆ 111.97 KiB │
│ │ │ tapi ┆ ┆ 1600.0.11.8 ┆ hb561403_3 ┆ conda-forge ┆ 195.70 KiB │
│ │ │ zstd ┆ ┆ 1.5.7 ┆ hf451053_7 ┆ conda-forge ┆ 423.52 KiB │
│ │ ╰─────────────────────────┴────────────────────────┴─────────────┴─────────────────────┴─────────────┴────────────╯
│ │ Resolved host dependencies(python-3.16-0_asan_cp316):
│ │ ╭─────────────────┬────────────────────────────────────────────────┬───────────┬─────────────────┬─────────────┬────────────╮
│ │ │ Package ┆ Spec ┆ Version ┆ Build ┆ Channel ┆ Size │
│ │ ╞═════════════════╪════════════════════════════════════════════════╪═══════════╪═════════════════╪═════════════╪════════════╡
│ │ │ bzip2 ┆ bzip2 ┆ 1.0.8 ┆ h4e30115_10 ┆ conda-forge ┆ 122.04 KiB │
│ │ │ expat ┆ expat ┆ 2.8.1 ┆ hf6b4638_1 ┆ conda-forge ┆ 132.87 KiB │
│ │ │ libcompiler-rt ┆ libcompiler-rt ┆ 23.1.1 ┆ hdb3d66b_0 ┆ conda-forge ┆ 1.32 MiB │
│ │ │ libcxx ┆ libcxx >=21 (RE of [build: clangxx_osx-arm64]) ┆ 23.1.1 ┆ h55c6f16_0 ┆ conda-forge ┆ 565.00 KiB │
│ │ │ libffi ┆ libffi ┆ 3.7.0 ┆ h47dc5ef_1 ┆ conda-forge ┆ 42.61 KiB │
│ │ │ liblzma-devel ┆ liblzma-devel ┆ 5.8.3 ┆ h8088a28_1 ┆ conda-forge ┆ 116.12 KiB │
│ │ │ libmpdec-devel ┆ libmpdec-devel ┆ 4.0.0 ┆ hce30654_2 ┆ conda-forge ┆ 14.78 KiB │
│ │ │ ncurses ┆ ncurses ┆ 6.6 ┆ he64c551_1 ┆ conda-forge ┆ 785.45 KiB │
│ │ │ openssl ┆ openssl 3.5.* ┆ 3.5.8 ┆ h55eecbc_0 ┆ conda-forge ┆ 2.92 MiB │
│ │ │ readline ┆ readline ┆ 8.3 ┆ h8b90a29_1 ┆ conda-forge ┆ 307.03 KiB │
│ │ │ sqlite ┆ sqlite ┆ 3.53.4 ┆ ha9f7ffe_1 ┆ conda-forge ┆ 175.69 KiB │
│ │ │ tk ┆ tk ┆ 9.0.4 ┆ hbeba79b_1 ┆ conda-forge ┆ 16.14 KiB │
│ │ │ xorg-libx11 ┆ xorg-libx11 ┆ 1.8.13 ┆ hf948f5a_1 ┆ conda-forge ┆ 740.99 KiB │
│ │ │ xorg-xorgproto ┆ xorg-xorgproto ┆ 2025.1 ┆ h74c22ad_3 ┆ conda-forge ┆ 583.54 KiB │
│ │ │ zlib ┆ zlib ┆ 1.3.2 ┆ h8088a28_3 ┆ conda-forge ┆ 79.83 KiB │
│ │ │ zstd ┆ zstd ┆ 1.5.7 ┆ hf451053_7 ┆ conda-forge ┆ 423.52 KiB │
│ │ │ ca-certificates ┆ ┆ 2026.7.22 ┆ hbd8a1cb_0 ┆ conda-forge ┆ 128.69 KiB │
│ │ │ icu ┆ ┆ 78.3 ┆ py310h579977c_2 ┆ conda-forge ┆ 13.42 MiB │
│ │ │ libexpat ┆ ┆ 2.8.1 ┆ hf6b4638_1 ┆ conda-forge ┆ 67.74 KiB │
│ │ │ liblzma ┆ ┆ 5.8.3 ┆ h8088a28_1 ┆ conda-forge ┆ 89.57 KiB │
│ │ │ libmpdec ┆ ┆ 4.0.0 ┆ h84a0fba_2 ┆ conda-forge ┆ 71.57 KiB │
│ │ │ libsqlite ┆ ┆ 3.53.4 ┆ hca69786_1 ┆ conda-forge ┆ 920.66 KiB │
│ │ │ libtcl ┆ ┆ 9.0.4 ┆ hbeba79b_1 ┆ conda-forge ┆ 1.13 MiB │
│ │ │ libtk ┆ ┆ 9.0.4 ┆ hbeba79b_1 ┆ conda-forge ┆ 1.32 MiB │
│ │ │ libtommath ┆ ┆ 1.3.0 ┆ h74c22ad_3 ┆ conda-forge ┆ 57.29 KiB │
│ │ │ libxcb ┆ ┆ 1.17.0 ┆ hbffa61f_2 ┆ conda-forge ┆ 316.63 KiB │
│ │ │ libzlib ┆ ┆ 1.3.2 ┆ h8088a28_3 ┆ conda-forge ┆ 46.70 KiB │
│ │ │ pthread-stubs ┆ ┆ 0.4 ┆ hb001647_1004 ┆ conda-forge ┆ 9.44 KiB │
│ │ │ xorg-libxau ┆ ┆ 1.0.12 ┆ hb001647_2 ┆ conda-forge ┆ 16.92 KiB │
│ │ │ xorg-libxdmcp ┆ ┆ 1.1.5 ┆ h84a0fba_2 ┆ conda-forge ┆ 19.03 KiB │
│ │ ╰─────────────────┴────────────────────────────────────────────────┴───────────┴─────────────────┴─────────────┴────────────╯
│ │ Resolved run dependencies(python-3.16-0_asan_cp316):
│ │ ╭──────────────────────┬───────────────────────────────────────────────╮
│ │ │ Name ┆ Spec │
│ │ ╞══════════════════════╪═══════════════════════════════════════════════╡
│ │ │ Run dependencies ┆ │
│ │ │ bzip2 ┆ >=1.0.8,<2.0a0 (RE of [host: bzip2]) │
│ │ │ libcompiler-rt ┆ >=23.1.1 (RE of [host: libcompiler-rt]) │
│ │ │ libcxx ┆ >=21 (RE of [build: clangxx_osx-arm64]) │
│ │ │ libexpat ┆ >=2.8.1,<3.0a0 (RE of [host: expat]) │
│ │ │ libffi ┆ >=3.7.0,<3.8.0a0 (RE of [host: libffi]) │
│ │ │ liblzma ┆ >=5.8.3,<6.0a0 (RE of [host: liblzma-devel]) │
│ │ │ libmpdec ┆ >=4.0.0,<5.0a0 (RE of [host: libmpdec-devel]) │
│ │ │ libsqlite ┆ >=3.53.4,<4.0a0 (RE of [host: sqlite]) │
│ │ │ libzlib ┆ >=1.3.2,<2.0a0 (RE of [host: zlib]) │
│ │ │ ncurses ┆ >=6.6,<7.0a0 (RE of [host: ncurses]) │
│ │ │ openssl ┆ >=3.5.8,<4.0a0 (RE of [host: openssl]) │
│ │ │ readline ┆ >=8.3,<9.0a0 (RE of [host: readline]) │
│ │ │ zstd ┆ >=1.5.7,<1.6.0a0 (RE of [host: zstd]) │
│ │ │ ┆ │
│ │ │ Run exports (Weak) ┆ │
│ │ │ python_abi ┆ 3.16.* *_asan_cp316 │
│ │ │ ┆ │
│ │ │ Run exports (Noarch) ┆ │
│ │ │ python ┆ │
│ │ ╰──────────────────────┴───────────────────────────────────────────────╯
ngoldbaum commented on Sep 17, 2026 Here's an AI-generated C++ example that triggers the same issue NumPy is triggering:
Details
// Standalone demonstration of the _Py_TYPEOF() C++ regression (gh-157649).
// Build against CPython main as C++ and run; every line marked FAIL shows
// Py_SETREF()/Py_XSETREF()/Py_CLEAR() operating on the wrong object.
#include <Python.h>
#include <cstdio>
#include <type_traits>
int main()
{
Py_Initialize();
PyObject *slots[1];
// decltype(slots[0]) is "PyObject *&" (a reference), so the temporary that
// is supposed to hold the *old* value aliases the slot itself.
printf("_Py_TYPEOF(slots[0]) is a reference type: %s\n",
std::is_reference<_Py_TYPEOF(slots[0])>::value ? "yes" : "no");
PyObject *a = PyList_New(0);
PyObject *b = PyList_New(0);
Py_INCREF(a); // a: 2
Py_INCREF(b); // b: 2
slots[0] = a;
Py_SETREF(slots[0], b); // should DECREF a, leave b alone
printf("after Py_SETREF(slots[0], b): refcnt(a)=%zd (expect 1), refcnt(b)=%zd (expect 2) %s\n",
Py_REFCNT(a), Py_REFCNT(b),
(Py_REFCNT(a) == 1 && Py_REFCNT(b) == 2) ? "OK" : "FAIL");
slots[0] = a;
Py_INCREF(a);
Py_INCREF(b);
Py_XSETREF(slots[0], b);
printf("after Py_XSETREF(slots[0], b): refcnt(a)=%zd (expect 1), refcnt(b)=%zd (expect 3) %s\n",
Py_REFCNT(a), Py_REFCNT(b),
(Py_REFCNT(a) == 1 && Py_REFCNT(b) == 3) ? "OK" : "FAIL");
// Py_CLEAR(slots[0]) expands to: store NULL into the slot, then
// Py_DECREF(<alias of the slot>) == Py_DECREF(NULL).
PyObject *c = PyList_New(0);
Py_INCREF(c);
slots[0] = c;
printf("calling Py_CLEAR(slots[0]) ...\n");
fflush(stdout);
Py_CLEAR(slots[0]);
printf("after Py_CLEAR(slots[0]): refcnt(c)=%zd (expect 1) %s\n",
Py_REFCNT(c), Py_REFCNT(c) == 1 ? "OK" : "FAIL");
Py_Finalize();
return 0;
}
The decltype of an lvalue expression like slots[0] is a reference type rather than an object type. That means with the new semantics, the "old value" temporary in Py_SETREF/Py_XSETREF/Py_CLEAR ends up as an alias of the slot being overwritten. Reacted by Stan Ulbrych Reacted by Lucas Colley StanFromIreland commented on Sep 17, 2026
Update: bisected to bba99f2 #157267
As I noted there, that's not possible as the commit was not backported to 3.15. It's only on main.
Sorry, I misread the issue, 3.15 is not failing if I understand correctly. Reacted by Lucas Colley and Chris Eibl lucascolley commented on Sep 17, 2026 ContributorAuthor More actions changed the title [-]ASan regression since 3.15: SEGV with simple `numpy` program[/-] [+]ASan regression on the main branch in Py_CLEAR/Py_SETREF: SEGV with simple `numpy` program[/+] on Sep 17, 2026vstinner commented on Sep 17, 2026 I built and installed Python in /opt/py316 with:
cd ~/cpython # https://github.com/python/cpython/ checkout
./configure --cache-file=../python-config.cache --with-pydebug --with-system-expat --prefix /opt/py316
make
make install
I built @ngoldbaum reproducer with:
$ cd ~/repro # with repro.cpp
g++ repro.cpp -o repro $(PKG_CONFIG_PATH=/opt/py316/lib/pkgconfig/ pkg-config python-3.16-embed --cflags --libs)
The reproducer does crash without ASAN:
$ ./repro
$ ./repro
_Py_TYPEOF(slots[0]) is a reference type: yes
after Py_SETREF(slots[0], b): refcnt(a)=2 (expect 1), refcnt(b)=1 (expect 2) FAIL
after Py_XSETREF(slots[0], b): refcnt(a)=3 (expect 1), refcnt(b)=1 (expect 3) FAIL
calling Py_CLEAR(slots[0]) ...
Segmentation fault (core dumped) ./repro
vstinner commented on Sep 17, 2026 When running g++ -E, the C++ code:
PyObject *slots[1];
Py_CLEAR(slots[0]);becomes:
PyObject *slots[1];
decltype(&(slots[0])) _tmp_op_ptr = &(slots[0]);
decltype(slots[0]) _tmp_old_op = (*_tmp_op_ptr);
if (_tmp_old_op != nullptr) {
*_tmp_op_ptr = nullptr;
Py_DECREF((PyObject*)_tmp_old_op);
}
I can reproduce the crash with this simple C++ code which doesn't use Python:
struct PyObject {
int data;
};
typedef struct PyObject PyObject;
void Py_DECREF(PyObject *obj)
{
obj->data = 0;
}
int main()
{
PyObject obj = {1};
PyObject *slots[1] = {&obj};
decltype(&(slots[0])) _tmp_op_ptr = &(slots[0]);
decltype(slots[0]) _tmp_old_op = (*_tmp_op_ptr);
if (_tmp_old_op != nullptr) {
*_tmp_op_ptr = nullptr;
Py_DECREF((PyObject*)_tmp_old_op);
}
return (obj.data == 0);
}The following C++ program doesn't crash, it uses Python 3.15 Py_CLEAR() implementation:
#include <string.h>
struct PyObject {
int data;
};
typedef struct PyObject PyObject;
void Py_DECREF(PyObject *obj)
{
obj->data = 0;
}
int main()
{
PyObject obj = {1};
PyObject *slots[1] = {&obj};
PyObject **_tmp_op_ptr = &(slots[0]);
PyObject *_tmp_old_op = (*_tmp_op_ptr);
if (_tmp_old_op != nullptr) {
PyObject *_null_ptr = nullptr;
memcpy(_tmp_op_ptr, &_null_ptr, sizeof(PyObject*));
Py_DECREF((PyObject*)_tmp_old_op);
}
return (obj.data == 0);
}lucascolley commented on Sep 17, 2026 ContributorAuthor More actions
The reproducer does crash without ASAN
(indeed, this diff reproduces the SIGSEGV (albeit without the memory address traceback))
diff --git a/pixi.toml b/pixi.toml
index 25be922..ed8b9cb 100644
--- a/pixi.toml
+++ b/pixi.toml
@@ -15,3 +15,3 @@ numpy = {
python = {
- flags = ["asan"],
+ flags = ["default"],
# passingchanged the title [-]ASan regression on the main branch in Py_CLEAR/Py_SETREF: SEGV with simple `numpy` program[/-] [+]Py_CLEAR/Py_SETREF() regression on the main branch with C++: SEGV with simple `numpy` program[/+] on Sep 17, 2026vstinner commented on Sep 28, 2026 Thanks for your bug report. The issue should now be fixed in the main branch, and tests have been added to check for non regression. lucascolley commented on Sep 28, 2026 ContributorAuthor More actions thank you @vstinner and all for the extensive work here! Glad to have helped find this :) Reacted by Victor Stinner vstinner commented on Oct 4, 2026 If there are still C++ experts around, I would like your opinion about gh-158810 issue (Py_ARRAY_LENGTH). I'm not sure how to fix it. vstinner commented on Nov 2, 2026 Py_SETREF() macros family is a gift which keeps giving: see issue gh-161185 about strict aliasing.
MetadataMetadatainterpreter-core(Objects, Python, Grammar, and Parser dirs)(Objects, Python, Grammar, and Parser dirs)type-bugAn unexpected behavior, bug, or errorAn unexpected behavior, bug, or error DevelopmentNo branches or pull requestsIssue actions
|