GeezerVison

The World Wide Web, restored to 800 pixels of glory.

GeezerVison is showing github.com — ./Programs/_freeze_module fails with MSAN: Uninitialized value was created by an allocation of 'stat.i' · Issue #91043 · python/cpython

./Programs/_freeze_module fails with MSAN: Uninitialized value was created by an allocation of 'stat.i' #91043

New issue

Copy link

New issue

Copy link

Closed

./Programs/_freeze_module fails with MSAN: Uninitialized value was created by an allocation of 'stat.i'#91043

Copy link

Labels

3.11only security fixesonly security fixesbuildThe build process and cross-buildThe build process and cross-build

Description

vstinner

opened

on Mar 1, 2022

Issue body actions

BPO	46887
Nosy	@vstinner
PRs	bpo-46887: Work around clang MSAN bug on stat()/fstat() #31633

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = None
created_at = <Date 2022-03-01.09:38:30.397>
labels = ['build', '3.11']
title = "./Programs/_freeze_module fails with MSAN: Uninitialized value was created by an allocation of 'stat.i'"
updated_at = <Date 2022-03-01.15:01:07.589>
user = 'https://github.com/vstinner'

bugs.python.org fields:

activity = <Date 2022-03-01.15:01:07.589>
actor = 'vstinner'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = ['Build']
creation = <Date 2022-03-01.09:38:30.397>
creator = 'vstinner'
dependencies = []
files = []
hgrepos = []
issue_num = 46887
keywords = ['patch']
message_count = 6.0
messages = ['414249', '414264', '414267', '414268', '414269', '414271']
nosy_count = 1.0
nosy_names = ['vstinner']
pr_nums = ['31633']
priority = 'normal'
resolution = None
stage = 'patch review'
status = 'open'
superseder = None
type = None
url = 'https://bugs.python.org/issue46887'
versions = ['Python 3.11']

Activity

vstinner commented on Mar 1, 2022
vstinner
on Mar 1, 2022
MemberAuthor
More actions
"./Programs/_freeze_module zipimport" fails with MSAN:
---

$ make SHELL="bash -x" (...) + ./Programs/_freeze_module zipimport ./Lib/zipimport.py Python/frozen_modules/zipimport.h ==110524==WARNING: MemorySanitizer: use-of-uninitialized-value ==110523==WARNING: MemorySanitizer: use-of-uninitialized-value ==110526==WARNING: MemorySanitizer: use-of-uninitialized-value ==110525==WARNING: MemorySanitizer: use-of-uninitialized-value #0 0x4b8cfc in read_text /home/vstinner/python/main/Programs/_freeze_module.c:109:9 #1 0x4b8cfc in main /home/vstinner/python/main/Programs/_freeze_module.c:224:24 #2 0x7fe8acb8555f in __libc_start_call_main (/lib64/libc.so.6+0x2d55f) #3 0x7fe8acb8560b in __libc_start_main@GLIBC_2.2.5 (/lib64/libc.so.6+0x2d60b) #4 0x435f44 in _start (/home/vstinner/python/main/Programs/_freeze_module+0x435f44)

Uninitialized value was created by an allocation of 'stat.i' in the stack frame of function 'main'
#0 0x4b72e0 in main /home/vstinner/python/main/Programs/_freeze_module.c:205

SUMMARY: MemorySanitizer: use-of-uninitialized-value /home/vstinner/python/main/Programs/_freeze_module.c:109:9 in read_text
Exiting
make: *** [Makefile:1184: Python/frozen_modules/importlib._bootstrap_external.h] Error 1
make: *** Waiting for unfinished jobs....
#0 0x4b8cfc in read_text /home/vstinner/python/main/Programs/_freeze_module.c:109:9
#1 0x4b8cfc in main /home/vstinner/python/main/Programs/_freeze_module.c:224:24
#2 0x7f64ecd1355f in __libc_start_call_main (/lib64/libc.so.6+0x2d55f)
#3 0x7f64ecd1360b in __libc_start_main@GLIBC_2.2.5 (/lib64/libc.so.6+0x2d60b)
#4 0x435f44 in _start (/home/vstinner/python/main/Programs/_freeze_module+0x435f44)

Uninitialized value was created by an allocation of 'stat.i' in the stack frame of function 'main'
#0 0x4b72e0 in main /home/vstinner/python/main/Programs/_freeze_module.c:205

SUMMARY: MemorySanitizer: use-of-uninitialized-value /home/vstinner/python/main/Programs/_freeze_module.c:109:9 in read_text
Exiting
make: *** [Makefile:1176: Python/frozen_modules/getpath.h] Error 1
#0 0x4b8cfc in read_text /home/vstinner/python/main/Programs/_freeze_module.c:109:9
#1 0x4b8cfc in main /home/vstinner/python/main/Programs/_freeze_module.c:224:24
#2 0x7fb2a178d55f in __libc_start_call_main (/lib64/libc.so.6+0x2d55f)
#3 0x7fb2a178d60b in __libc_start_main@GLIBC_2.2.5 (/lib64/libc.so.6+0x2d60b)
#4 0x435f44 in _start (/home/vstinner/python/main/Programs/_freeze_module+0x435f44)

#0 0x4b8cfc in read_text /home/vstinner/python/main/Programs/_freeze_module.c:109:9 #1 0x4b8cfc in main /home/vstinner/python/main/Programs/_freeze_module.c:224:24 #2 0x7f4d471ca55f in __libc_start_call_main (/lib64/libc.so.6+0x2d55f) #3 0x7f4d471ca60b in __libc_start_main@GLIBC_2.2.5 (/lib64/libc.so.6+0x2d60b) #4 0x435f44 in _start (/home/vstinner/python/main/Programs/_freeze_module+0x435f44)

Uninitialized value was created by an allocation of 'stat.i' in the stack frame of function 'main'
Uninitialized value was created by an allocation of 'stat.i' in the stack frame of function 'main'
#0 0x4b72e0 in main /home/vstinner/python/main/Programs/_freeze_module.c:205

#0 0x4b72e0 in main /home/vstinner/python/main/Programs/_freeze_module.c:205

SUMMARY: MemorySanitizer: use-of-uninitialized-value /home/vstinner/python/main/Programs/_freeze_module.c:109:9 in read_text
SUMMARY: MemorySanitizer: use-of-uninitialized-value /home/vstinner/python/main/Programs/_freeze_module.c:109:9 in read_text
Exiting
Exiting
make: *** [Makefile:1181: Python/frozen_modules/importlib._bootstrap.h] Error 1
make: *** [Makefile:1187: Python/frozen_modules/zipimport.h] Error 1
---

Build Python with clang MSAN:
---
./configure --with-pydebug --with-memory-sanitizer CC=clang LD=clang
make
---
vstinner
added
3.11only security fixesonly security fixes

buildThe build process and cross-buildThe build process and cross-build
on Mar 1, 2022
vstinner commented on Mar 1, 2022
vstinner
on Mar 1, 2022
MemberAuthor
More actions
It looks like a bug in clang MSAN:
llvm/llvm-project#54131

vstinner commented on Mar 1, 2022

vstinner

on Mar 1, 2022

MemberAuthor

More actions

Hum, later "import ssl" in setup.py also fails. Simplified code:
---

import _ssl
print(_ssl.txt2obj('1.3.6.1.5.5.7.3.1', name=False)) # server OID

Error:
---

$ ./python x.py 
Uninitialized bytes in MemcmpInterceptorCommon at offset 0 inside [0x701000000032, 8)
==135651==WARNING: MemorySanitizer: use-of-uninitialized-value
    #0 0x49467c in memcmp (/home/vstinner/python/main/python+0x49467c)
    #1 0x7f5546825adc in OBJ_bsearch_ex_ (/lib64/libcrypto.so.1.1+0x14eadc)
    #2 0x7f5546826ff1 in OBJ_obj2nid (/lib64/libcrypto.so.1.1+0x14fff1)
    #3 0x7f554675b434  (/lib64/libcrypto.so.1.1+0x84434)
    #4 0x7f554675b6af in d2i_ASN1_OBJECT (/lib64/libcrypto.so.1.1+0x846af)
    #5 0x7f5546827702 in OBJ_txt2obj (/lib64/libcrypto.so.1.1+0x150702)
    #6 0x7f5546a78c31 in _ssl_txt2obj_impl /home/vstinner/python/main/Modules/_ssl.c:5276:11
    #7 0x7f5546a78c31 in _ssl_txt2obj /home/vstinner/python/main/Modules/clinic/_ssl.c.h:1216:20
    (...)

Uninitialized value was created by a heap allocation
#0 0x466642 in __interceptor_malloc (/home/vstinner/python/main/python+0x466642)
#1 0x7f55468276b3 in OBJ_txt2obj (/lib64/libcrypto.so.1.1+0x1506b3)

SUMMARY: MemorySanitizer: use-of-uninitialized-value (/home/vstinner/python/main/python+0x49467c) in memcmp
Exiting
---

The Python code calls OBJ_txt2obj("1.3.6.1.5.5.7.3.1", 0) in C: the OpenSSL function.

vstinner commented on Mar 1, 2022
vstinner
on Mar 1, 2022
MemberAuthor
More actions

It looks like a bug in clang MSAN:
llvm/llvm-project#54131

I wrote #75814 to work around the false alarm on stat() and fstat().
vstinner commented on Mar 1, 2022
vstinner
on Mar 1, 2022
MemberAuthor
More actions

The Python code calls OBJ_txt2obj("1.3.6.1.5.5.7.3.1", 0) in C: the OpenSSL function.

This error is unrelated to Python, but comes from OpenSSL.
vstinner commented on Mar 1, 2022
vstinner
on Mar 1, 2022
MemberAuthor
More actions

The Python code calls OBJ_txt2obj("1.3.6.1.5.5.7.3.1", 0) in C: the OpenSSL function.

I reported this issue to OpenSSL: openssl/openssl#17784
ezio-melotti
transferred this issue from on Apr 10, 2022
vstinner commented on Nov 3, 2022
vstinner
on Nov 3, 2022
MemberAuthor
More actions
It's unclear to me if it's an issue in Python itself, Clang/GCC, or how the glibc was built. For example, openssl/openssl#17784 (comment) was closed OpenSSL must be built with -fsanitize=memory.

Anyway, I close the issue.
vstinner
closed this as completed on Nov 3, 2022

illia-v commented on Mar 7, 2023

illia-v

on Mar 7, 2023

Contributor

More actions

It's unclear to me if it's an issue in Python itself, Clang/GCC, or how the glibc was built. For example, openssl/openssl#17784 (comment) was closed OpenSSL must be built with -fsanitize=memory.

I've recently tried building and compiling CPython with CC=clang ./configure --with-memory-sanitizer && make -j12 against the system OpenSSL.

After making the #102510 change, a slightly different use-of-uninitialized-value error tracing to /lib/x86_64-linux-gnu/libcrypto.so.3 appeared.

So, I tried compiling it against OpenSSL built with -fsanitize=memory, and that fixed an error.

But it looks like there is a sequence of other use-of-uninitialized-value appearing one by one.
The next one looks to be fixable by this

diff --git a/Python/dynload_shlib.c b/Python/dynload_shlib.c
index 6761bba457..ac379d5d98 100644
--- a/Python/dynload_shlib.c
+++ b/Python/dynload_shlib.c
@@ -56,7 +56,7 @@ _PyImport_FindSharedFuncptr(const char *prefix,
 {
     dl_funcptr p;
     void *handle;
-    char funcname[258];
+    char funcname[258] = "";
     char pathbuf[260];
     int dlopenflags=0;

I stopped digging on this one, but noticed the symbol lookup error: /home/user/openssl/lib64/libcrypto.so.3: undefined symbol: __msan_retval_tls warnings:

Uninitialized bytes in __interceptor_strlen at offset 0 inside [0x701000003040, 1)
==91306==WARNING: MemorySanitizer: use-of-uninitialized-value
/usr/bin/llvm-symbolizer-15: symbol lookup error: /home/user/openssl/lib64/libcrypto.so.3: undefined symbol: __msan_retval_tls
==91306==WARNING: external symbolizer didn't start up correctly!
==91306==WARNING: Can't write to symbolizer at fd 6
/usr/bin/llvm-symbolizer-15: symbol lookup error: /home/user/openssl/lib64/libcrypto.so.3: undefined symbol: __msan_retval_tls
==91306==WARNING: external symbolizer didn't start up correctly!
==91306==WARNING: Can't write to symbolizer at fd 6
/usr/bin/llvm-symbolizer-15: symbol lookup error: /home/user/openssl/lib64/libcrypto.so.3: undefined symbol: __msan_retval_tls
==91306==WARNING: external symbolizer didn't start up correctly!
==91306==WARNING: Can't write to symbolizer at fd 6
/usr/bin/llvm-symbolizer-15: symbol lookup error: /home/user/openssl/lib64/libcrypto.so.3: undefined symbol: __msan_retval_tls
==91306==WARNING: external symbolizer didn't start up correctly!
==91306==WARNING: Can't write to symbolizer at fd 6
/usr/bin/llvm-symbolizer-15: symbol lookup error: /home/user/openssl/lib64/libcrypto.so.3: undefined symbol: __msan_retval_tls
==91306==WARNING: external symbolizer didn't start up correctly!
==91306==WARNING: Failed to use and restart external symbolizer!
    #0 0x7f98fe69104b  (/lib/x86_64-linux-gnu/libreadline.so.8+0x3204b) (BuildId: 990587ed58ff96de9f94756458d4aaa2f51aaa3d)
    #1 0x7f98fe880ab2  (/home/user/cpython/build/lib.linux-x86_64-3.12/readline.cpython-312-x86_64-linux-gnu.so+0x3ab2) (BuildId: a50d39ef2c15d0868c490d8e02f64b74edea405b)
    #2 0x557f32c393af  (/home/user/cpython/python+0x9a03af) (BuildId: 61b467aa3df7c96f1560120596af9896ed67767d)
    #3 0x557f32c36cb8  (/home/user/cpython/python+0x99dcb8) (BuildId: 61b467aa3df7c96f1560120596af9896ed67767d)
    #4 0x557f327b7d68  (/home/user/cpython/python+0x51ed68) (BuildId: 61b467aa3df7c96f1560120596af9896ed67767d)
    #5 0x557f326002e9  (/home/user/cpython/python+0x3672e9) (BuildId: 61b467aa3df7c96f1560120596af9896ed67767d)
    #6 0x557f326012b5  (/home/user/cpython/python+0x3682b5) (BuildId: 61b467aa3df7c96f1560120596af9896ed67767d)
    #7 0x557f326016e3  (/home/user/cpython/python+0x3686e3) (BuildId: 61b467aa3df7c96f1560120596af9896ed67767d)
    #8 0x557f32afd88f  (/home/user/cpython/python+0x86488f) (BuildId: 61b467aa3df7c96f1560120596af9896ed67767d)
    #9 0x557f32ad0cc2  (/home/user/cpython/python+0x837cc2) (BuildId: 61b467aa3df7c96f1560120596af9896ed67767d)
    #10 0x557f32cc4ce3  (/home/user/cpython/python+0xa2bce3) (BuildId: 61b467aa3df7c96f1560120596af9896ed67767d)
    #11 0x557f32cbbefe  (/home/user/cpython/python+0xa22efe) (BuildId: 61b467aa3df7c96f1560120596af9896ed67767d)
    #12 0x557f32cba316  (/home/user/cpython/python+0xa21316) (BuildId: 61b467aa3df7c96f1560120596af9896ed67767d)
    #13 0x557f32d6975b  (/home/user/cpython/python+0xad075b) (BuildId: 61b467aa3df7c96f1560120596af9896ed67767d)
    #14 0x557f32d6b41a  (/home/user/cpython/python+0xad241a) (BuildId: 61b467aa3df7c96f1560120596af9896ed67767d)
    #15 0x557f32d6b5cd  (/home/user/cpython/python+0xad25cd) (BuildId: 61b467aa3df7c96f1560120596af9896ed67767d)
    #16 0x557f324375d8  (/home/user/cpython/python+0x19e5d8) (BuildId: 61b467aa3df7c96f1560120596af9896ed67767d)
    #17 0x7f990342350f  (/lib/x86_64-linux-gnu/libc.so.6+0x2350f) (BuildId: d1704d25fbbb72fa95d517b883131828c0883fe9)
    #18 0x7f99034235c8  (/lib/x86_64-linux-gnu/libc.so.6+0x235c8) (BuildId: d1704d25fbbb72fa95d517b883131828c0883fe9)
    #19 0x557f323b0e54  (/home/user/cpython/python+0x117e54) (BuildId: 61b467aa3df7c96f1560120596af9896ed67767d)

  Uninitialized value was created by a heap allocation
    #0 0x557f323e4f00  (/home/user/cpython/python+0x14bf00) (BuildId: 61b467aa3df7c96f1560120596af9896ed67767d)
    #1 0x7f98fe699ddc  (/lib/x86_64-linux-gnu/libreadline.so.8+0x3addc) (BuildId: 990587ed58ff96de9f94756458d4aaa2f51aaa3d)

SUMMARY: MemorySanitizer: use-of-uninitialized-value (/lib/x86_64-linux-gnu/libreadline.so.8+0x3204b) (BuildId: 990587ed58ff96de9f94756458d4aaa2f51aaa3d)

Metadata

Assignees

No one assigned

Labels

3.11only security fixesonly security fixesbuildThe build process and cross-buildThe build process and cross-build

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions

You are visitor 01402503 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.

GeezerVison

./Programs/_freeze_module fails with MSAN: Uninitialized value was created by an allocation of 'stat.i' #91043

Description

Activity

vstinner commented on Mar 1, 2022

vstinner commented on Mar 1, 2022

vstinner commented on Mar 1, 2022

vstinner commented on Mar 1, 2022

vstinner commented on Mar 1, 2022

vstinner commented on Mar 1, 2022

vstinner commented on Nov 3, 2022

illia-v commented on Mar 7, 2023

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions