GeezerVison is showing github.com — [pipeline-manager] Pipeline binaries over 2 GiB fail to link: mold resolves calls to undefined weak hooks in zstd and aws-lc to address 0 · Issue #7924 · feldera/feldera
[pipeline-manager] Pipeline binaries over 2 GiB fail to link: mold resolves calls to undefined weak hooks in zstd and aws-lc to address 0 #7924DescriptionA user's two generated programs (1.1 MB and 1.8 MB of SQL, 585 and 670 input tables) compile in SQL on 0.381.0 Enterprise. The final link of the pipeline binary then fails after 54 and 101 minutes. mold reports 12 errors for each program, all of this form:
mold: error: .../target/optimized/deps/libzstd_sys-af58253c2a06b4d1.rlib(fb80479a5fb81f6a-zstd_compress.o):(.text.ZSTD_compressBegin_internal): relocation R_X86_64_PLT32 against ZSTD_trace_compress_begin out of range: -2347874965 is not in [-2147483648, 2147483648)
mold: error: .../target/optimized/deps/libaws_lc_sys-0c67e737d76f9054.rlib(30a9a10d3f98970e-mem.o):(.text.aws_lc_0_44_0_OPENSSL_malloc): relocation R_X86_64_PLT32 against OPENSSL_memory_alloc out of range: -2352979459 is not in [-2147483648, 2147483648)
...
collect2: error: ld returned 1 exit status
| Hook functions called |
Crate |
Calls out of range |
Offset, program 1 |
Offset, program 2 |
ZSTD_trace_compress_begin, ZSTD_trace_compress_end, ZSTD_trace_decompress_begin, ZSTD_trace_decompress_end |
zstd-sys 2.1.2+zstd.1.5.7 |
8 |
-2.35 GB |
-2.80 GB |
OPENSSL_memory_alloc, OPENSSL_memory_free, OPENSSL_memory_get_size, OPENSSL_memory_realloc |
aws-lc-sys 0.44.0 |
4 |
-2.35 GB |
-2.80 GB |
mold reports no other error for either program.
Steps
The repro calls one undefined weak function after 2.4 GB of read-only data, in a PIE (rustc links a PIE by default on x86_64-unknown-linux-gnu). It uses mold 2.40.4, the version in the runtime image (deploy/Dockerfile:117).
hook.c:
__attribute__((weak)) void hook(void);
int main(void) { if (hook) hook(); return 0; }
filler.s:
.section .rodata.filler,"a"
.skip 0x90000000
run.sh:
#!/bin/bash
apt-get update -qq && apt-get install -y -qq gcc curl ca-certificates binutils > /dev/null
curl -sL https://github.com/rui314/mold/releases/download/v2.40.4/mold-2.40.4-x86_64-linux.tar.gz | tar -xz -C /opt
export PATH=/opt/mold-2.40.4-x86_64-linux/bin:$PATH
cd /tmp && as /w/filler.s -o filler.o && gcc -O2 -c /w/hook.c -o hook.o && gcc -O2 -fno-plt -c /w/hook.c -o hook_noplt.o
gcc -fuse-ld=mold -pie filler.o hook.o -o a.out; echo "rc=$?"
gcc -fuse-ld=mold -pie filler.o hook.o -Wl,-z,dynamic-undefined-weak -o a.out && ./a.out; echo "rc=$?"
gcc -fuse-ld=mold -pie filler.o hook_noplt.o -o a.out && ./a.out; echo "rc=$?"
docker run --rm --platform linux/amd64 -v "$PWD":/w ubuntu:24.04 bash /w/run.sh
Result
| Link |
Result |
-pie |
mold: error: hook.o:(.text.startup): relocation R_X86_64_PLT32 against hook out of range: -2415924884 is not in [-2147483648, 2147483648) |
-pie -Wl,-z,dynamic-undefined-weak |
Links (2.4 GB). The call goes to hook$pltgot. The program runs and exits with 0 |
-pie, hook.c compiled with -fno-plt |
Links. The call is call *hook$got. The program runs and exits with 0 |
Without the filler, mold links the same hook.o to call 0.
Cause
- zstd declares its trace hooks weak and turns tracing on when the compiler has weak symbols (zstd-sys 2.1.2:
zstd/lib/common/zstd_trace.h, lines 41 to 42 and 123 to 150). aws-lc declares its memory hooks weak (aws-lc-sys 0.44.0: aws-lc/crypto/mem.c, lines 67 to 70). Nothing in a pipeline binary defines them.
- mold resolves a direct call to an undefined weak function to address 0. An
R_X86_64_PLT32 call reaches only 2 GiB, so the link fails when the caller is more than 2 GiB from the start of the image.
- mold puts read-only data before
.text. In these binaries the zstd and aws-lc code is at 2.35 GB and 2.80 GB. The binaries are over 2 GiB, but mold reports no real call or data reference that spans 2 GiB. Only these 12 calls fail.
Expected
- A pipeline binary over 2 GiB links when no real reference in it spans 2 GiB. For example, add
-C link-arg=-Wl,-z,dynamic-undefined-weak to RUSTFLAGS in deploy/Dockerfile:122 and deploy/build.Dockerfile:219.
- We tested the flag only on the repro, not on the user's programs. If a binary grows further, the next limit is a real reference that spans 2 GiB.
Activitymihaibudiu commented on Nov 10, 2026 I assigned this to @gz, he's our linker expert ryzhyk commented on Nov 11, 2026 This failure is a mold bug fixed in mold 2.42.1. We'll need to bump the mold dependency to pickup the fix.
However even with the fix mold doesn't support binaries >2GiB, so we need to reduce the size of the binary.
MetadataMetadataDevelopmentNo branches or pull requestsIssue actions
|