GeezerVison is showing github.com — 3.14.0b3 JIT build fails with `KeyError: 'R_X86_64_GOTPCREL'` · Issue #135690 · python/cpython
3.14.0b3 JIT build fails with KeyError: 'R_X86_64_GOTPCREL' #135690Labels 3.14bugs and security fixesbugs and security fixesinterpreter-core(Objects, Python, Grammar, and Parser dirs)(Objects, Python, Grammar, and Parser dirs)topic-JITtype-bugAn unexpected behavior, bug, or errorAn unexpected behavior, bug, or error DescriptionBug description:
The python-build-standalone experimental JIT build fails on 3.14.0b3 with:
cpython-3.14> | Traceback (most recent call last):
cpython-3.14> | File "/build/Python-3.14.0b3/Tools/jit/_targets.py", line 165, in _compile
cpython-3.14> | return await self._parse(o)
cpython-3.14> | ^^^^^^^^^^^^^^^^^^^^
cpython-3.14> | File "/build/Python-3.14.0b3/Tools/jit/_targets.py", line 106, in _parse
cpython-3.14> | self._handle_section(wrapped_section["Section"], group)
cpython-3.14> | ~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
cpython-3.14> | File "/build/Python-3.14.0b3/Tools/jit/_targets.py", line 352, in _handle_section
cpython-3.14> | hole = self._handle_relocation(base, relocation, stencil.body)
cpython-3.14> | File "/build/Python-3.14.0b3/Tools/jit/_targets.py", line 416, in _handle_relocation
cpython-3.14> | return _stencils.Hole(offset, kind, value, symbol, addend)
cpython-3.14> | ~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
cpython-3.14> | File "<string>", line 9, in __init__
cpython-3.14> | File "/build/Python-3.14.0b3/Tools/jit/_stencils.py", line 141, in __post_init__
cpython-3.14> | self.func = _PATCH_FUNCS[self.kind]
cpython-3.14> | ~~~~~~~~~~~~^^^^^^^^^^^
cpython-3.14> | KeyError: 'R_X86_64_GOTPCREL'
ref https://github.com/astral-sh/python-build-standalone/actions/runs/15737844881/job/44355511468?pr=663
I'm sort of confused that it's new? #130097 looks relevant but has been around for a bit. 3.14.0b2 works.
We use LLVM v20.1.4
CPython versions tested on:
3.14
Operating systems tested on:
Linux ActivityZheaoli commented on Jun 20, 2025 First, the official llvm support is 19. I think the JIT is not support 20 officially yet.
BTW, I can not reproduce it on 20.1.4, 20.1.7 and LLVM main branch
zanieb commented on Jun 20, 2025 ContributorAuthor More actions Did you change the target LLVM version? It's hard-coded to 19
|
_LLVM_VERSION = 19 |
|
_LLVM_VERSION_PATTERN = re.compile(rf"version\s+{_LLVM_VERSION}\.\d+\.\d+\S*\s+") |
|
_EXTERNALS_LLVM_TAG = "llvm-19.1.7.0" |
Zheaoli commented on Jun 21, 2025
Did you change the target LLVM version? It's hard-coded to 19
Yes Zheaoli commented on Jun 21, 2025 Would you mind giving me a codebase with a patched config? zanieb commented on Jun 21, 2025 ContributorAuthor More actions You can see the downstream build system at astral-sh/python-build-standalone#663
The JIT LLVM version patch is at
We also backport #134276
so our invocation looks like
/tools/host/bin/python3.14 ./Tools/jit/build.py x86_64-unknown-linux-gnu --output-dir . --pyconfig-dir . --cflags=" -fdebug-default-version=4 -fPIC -I/tools/deps/include -I/tools/deps/include/ncursesw"
Zheaoli commented on Jun 21, 2025 Thanks for the information. I might need a little bit time to reproduce it on my local environment Zheaoli commented on Jun 23, 2025 Sorry for taking some time. (My desktop crashed So I need to setup a new environment)
I think the root cause here you use -fPIC here. We don't handle this circumstance on x86_64(only in aarch64 if I'm correct)
I think the PIC flag will generate R_X86_64_GOTPCREL to calculate the address.(not sure about this
cc @brandtbucher AndPuQing commented on Jun 26, 2025 I think so. The -fPIC compilation flags were removed in gh-130097, which means that jit no longer uses GOT, so the support for R_X86_64_GOTPCREL was removed in _PATCH_FUNCS. I used the following patch, and now python-build-standalone works well.
The compilation instructions I used were
./build-linux.py --python cpython-3.14 --options pgo+lto --target-triple x86_64-unknown-linux-gnu diff --git a/cpython-unix/build-cpython.sh b/cpython-unix/build-cpython.sh
index 2296f2b..0b7cfda 100755
--- a/cpython-unix/build-cpython.sh
+++ b/cpython-unix/build-cpython.sh
@@ -90,12 +90,6 @@ else
patch -p1 -i ${ROOT}/patch-xopen-source-ios-legacy.patch
fi
-# See https://github.com/python/cpython/pull/135146
-# TODO(zanieb): Drop in 3.14b3
-if [ -n "${PYTHON_MEETS_MINIMUM_VERSION_3_14}" ]; then
- patch -p1 -i ${ROOT}/patch-static-remote-debug-3.14.patch
-fi
-
# LIBTOOL_CRUFT is unused and breaks cross-compiling on macOS. Nuke it.
# Submitted upstream at https://github.com/python/cpython/pull/101048.
if [ -n "${PYTHON_MEETS_MAXIMUM_VERSION_3_11}" ]; then
@@ -653,8 +647,13 @@ fi
# We patched configure.ac above. Reflect those changes.
autoconf
+# As of CPython 3.14 (GH-130097), the JIT compiler on x86-64 linux uses a
+# model that conflicts with -fPIC. We strip it from CFLAGS_JIT to prevent
+# compilation issues and ensure the correct model is used.
+CFLAGS_JIT="${CFLAGS//-fPIC/}"
+
# Ensure `CFLAGS` are propagated to JIT compilation for 3.13+
-CFLAGS=$CFLAGS CPPFLAGS=$CFLAGS CFLAGS_JIT=$CFLAGS LDFLAGS=$LDFLAGS \
+CFLAGS=$CFLAGS CPPFLAGS=$CFLAGS CFLAGS_JIT=$CFLAGS_JIT LDFLAGS=$LDFLAGS \
./configure ${CONFIGURE_FLAGS}
# Supplement produced Makefile with our modifications.zanieb commented on Jun 26, 2025 ContributorAuthor More actions brandtbucher commented on Jun 26, 2025 Nice, great work everybody! zanieb commented on Jun 27, 2025 ContributorAuthor More actions To be clear, it's expected that -fPIC won't be supported for the foreseeable future? I guess we should error ahead of time if that's present in the CFLAGS? brandtbucher commented on Jun 27, 2025 I don't think it's our job to parse and validate the flags being passed in here. The way I see it, CFLAGS_JIT is just an escape-hatch for distributors who really need it.
-fPIC, and all of the other flags we do and don't pass to Clang for stencil generation, are a super low-level implementation detail and subject to change in major version bumps without notice. My advice is to figure out the minimum flags that need to be passed to CFLAGS_JIT (in your case, probably just -I/tools/deps/include?) and only pass those to avoid messing up stencil generation in subtle ways.
zanieb commented on Jun 27, 2025 ContributorAuthor More actions
I don't think it's our job to parse and validate the flags being passed in here.
I agree in general, but this error message is quite bad and won't point a distributor to the root cause — if you don't want to parse the flags then there should probably be a better error message when this key (or some other key) is missing from _PATCH_FUNCS.
My advice is to figure out the minimum flags that need to be passed to CFLAGS_JIT (in your case, probably just -I/tools/deps/include?)
That feels just as feasible to break us in subtle ways, e.g., we could miss things like #134291 — hard to say one way is definitely better than the other.
MetadataMetadata3.14bugs and security fixesbugs and security fixesinterpreter-core(Objects, Python, Grammar, and Parser dirs)(Objects, Python, Grammar, and Parser dirs)topic-JITtype-bugAn unexpected behavior, bug, or errorAn unexpected behavior, bug, or error DevelopmentNo branches or pull requestsIssue actions
|
