GeezerVison is showing github.com — `linecache.cache` sometimes has an entry for `<string>` under Python 3.13.0a5 · Issue #117174 · python/cpython
linecache.cache sometimes has an entry for <string> under Python 3.13.0a5 #117174
DescriptionBug report
Bug description:
I noticed this via Hypothesis' pretty-printer for lambda functions, and tracked the divergence through the inspect module to linecache:
import linecache
def test():
print(linecache.cache["<string>"]) # expected to raise KeyError
assert FalseIf I run this snippet with python3.13 -Wignore -m pytest repro.py, or any older Python version, you'll get the expected KeyError. But if I append -n2 to the command, it prints (44, None, ['import sys;exec(eval(sys.stdin.readline()))\n'], '<string>')!
(python3.13 -m pip install pytest pytest-xdist will get the dependencies for this)
That's the popen bootstrap line from execnet, which handles running code across multiple processes for pytest-xdist. At this point I've found linecache.cache.pop("<string>", None) to be an acceptable workaround, and since I don't have any particular knowledge of either the CPython or execnet code that's as far as I investigated.
CPython versions tested on:
3.8, 3.9, 3.10, 3.11, 3.12, 3.13
Operating systems tested on:
Linux
Linked PRs
Activitygaogaotiantian commented on Mar 26, 2024 I believe this is a feature from #110805 which allows the traceback to show source code with python -c.
python -c "import linecache;print(linecache.cache);1/0"
3.12:
{}
Traceback (most recent call last):
File "<string>", line 1, in <module>
ZeroDivisionError: division by zero
3.13:
{'<string>': (44, None, ['import linecache;print(linecache.cache);1/0\n'], '<string>')}
Traceback (most recent call last):
File "<string>", line 1, in <module>
import linecache;print(linecache.cache);1/0
~^~
ZeroDivisionError: division by zero
However, I can see that the secret <string> key in the cache causing issues for inspect because that module heavily relies on it. An example:
./python -c "import inspect;x=eval('lambda x: x');print(inspect.getsource(x))"
The code above should trigger an exception because we should not be able to get the source of x (it's dynamically generated) but now it gives the full string.
I think a proper fix would be to change the code object name <string> for code passed with -c. <string> is commonly used for many dynamically generated code objects and if we want to do something special with the source code from -c we should name it differently. It might not be 100% backward compatible but I don't see a giant gap there. We can even keep the <string> in the traceback like what we did for python inputs.
A simpler way would be just disable this feature for -c - it's designed for interactive inputs and I don't even know if this is intentional or a pleasant side effect.
Or we could do even more aggressive and do similar numbering for dynamically compiled code and always give source code, which is a bit ambitious and definitely beyond this issue.
@pablogsal thoughts on this? I can do the PR if you want. Reacted by Zac Hatfield-Dodds pablogsal commented on Apr 3, 2024 I have opened #117500 to do a renaming of the string to something less common. We still want to preserve this behaviour for -c since this still leads to better error locations and tracebacks. added a commit that references this issue on Mar 10, 2025gh-117174: Add a new route in linecache to fetch interactive source c… added 3 commits that reference this issue on Mar 10, 2025gh-117174: Adapt test_multiple_statements_fail_early now that we have… encukou commented on Mar 11, 2025 After this was merged I'm seeing failures like this:
======================================================================
ERROR: test_frames (test.test_gdb.test_pretty_print.PrettyPrintTests.test_frames)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/buildbot/buildarea/3.x.cstratak-RHEL8-x86_64.lto/build/Lib/test/test_gdb/test_pretty_print.py", line 426, in test_frames
gdb_output = self.get_stack_trace('''
~~~~~~~~~~~~~~~~~~~~^^^^
import sys
^^^^^^^^^^
...<7 lines>...
cmds_after_breakpoint=['print (PyFrameObject*)x']
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
)
^
File "/home/buildbot/buildarea/3.x.cstratak-RHEL8-x86_64.lto/build/Lib/test/test_gdb/util.py", line 242, in get_stack_trace
out, err = run_gdb(*args, PYTHONHASHSEED=PYTHONHASHSEED)
~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/buildbot/buildarea/3.x.cstratak-RHEL8-x86_64.lto/build/Lib/test/test_gdb/util.py", line 67, in run_gdb
raise Exception(f"{cmd_text} failed with exit code {proc.returncode}, "
...<2 lines>...
f"stderr={stderr!r}")
Exception: /usr/bin/gdb --batch -nx --init-eval-command 'add-auto-load-safe-path /home/buildbot/buildarea/3.x.cstratak-RHEL8-x86_64.lto/build/python-gdb.py' '--eval-command=set breakpoint pending yes' '--eval-command=break _typing__idfunc' '--eval-command=set print address off' --eval-command=run '--eval-command=set print entry-values no' '--eval-command=print (PyFrameObject*)x' --args /home/buildbot/buildarea/3.x.cstratak-RHEL8-x86_64.lto/build/python -bb -E -Wdefault -S -c '
import sys
from _typing import _idfunc
def foo(a, b, c):
return sys._getframe(0)
f = foo(3, 4, 5)
_idfunc(f)' failed with exit code 1, expected exit code 0:
stdout='Breakpoint 1 at 0x43c9e0: file ./Include/object.h, line 270.\n[Thread debugging using libthread_db enabled]\nUsing host libthread_db library "/lib64/libthread_db.so.1".\n\nBreakpoint 1, _typing__idfunc () at ./Include/object.h:270\n270\t return ob->ob_type;\n'
stderr='No symbol "x" in current context.\n'
pablogsal commented on Mar 11, 2025 How this was not detected in the CI?
I will take a look later today encukou commented on Mar 12, 2025 vstinner commented on Mar 12, 2025 tl; dr I created PR gh-131143 to skip test_gdb if Python is built with LTO.
There are two important things on the AMD64 RHEL 8 buildbot:
- It uses an old GCC version: gcc (GCC) 8.5.0 20210514 (Red Hat 8.5.0-24)
- LTO is always used on RHEL buildbots
I already noticed weird gdb behaviors when Python is built with LTO.
@pablogsal:
Looking at the failures this is super wrong: (...)
_typing__idfunc is certainly not at ./Include/object.h:270
It's maybe time to declare game over and skip test_gdb if Python is built with LTO :-( We already skip test_gdb if Python is built with PGO or BOLT:
if support.check_cflags_pgo():
raise unittest.SkipTest("test_gdb is not reliable on PGO builds")
if support.check_bolt_optimized():
raise unittest.SkipTest("test_gdb is not reliable on BOLT optimized builds")By the way, we recently modified test_gdb to also skip it if Python is built with clang -Og (commit 129db32). pablogsal commented on Mar 12, 2025
tl; dr I created PR gh-131143 to skip test_gdb if Python is built with LTO.
There are two important things on the AMD64 RHEL 8 buildbot:
- It uses an old GCC version: gcc (GCC) 8.5.0 20210514 (Red Hat 8.5.0-24)
- LTO is always used on RHEL buildbots
I already noticed weird gdb behaviors when Python is built with LTO.
@pablogsal:
Looking at the failures this is super wrong: (...)
_typing__idfunc is certainly not at ./Include/object.h:270
It's maybe time to declare game over and skip test_gdb if Python is built with LTO :-( We already skip test_gdb if Python is built with PGO or BOLT:
if support.check_cflags_pgo():
raise unittest.SkipTest("test_gdb is not reliable on PGO builds")
if support.check_bolt_optimized():
raise unittest.SkipTest("test_gdb is not reliable on BOLT optimized builds")By the way, we recently modified test_gdb to also skip it if Python is built with clang -Og (commit 129db32).
Please let's not do it. The whole point of test gdb is to test that gdb works with released versions of CPython. If we skip it then gdb will be broken and we never know that the functionality works with the versions that people use it pablogsal commented on Mar 12, 2025 I fixed the test failures by using a different function that doesn't get super optimised to wrong DWARF so at least we are fine for now vstinner commented on Mar 12, 2025 Oh, I didn't see that you already fixed the issue. That's why I was unable to reproduce it :-)
Ok, I abandon my PR skipping test_gdb is Python is built with LTO. added a commit that references this issue on Mar 28, 2025gh-117174: Adapt `test_multiple_statements_fail_early` to new REPL be…
MetadataMetadataDevelopmentNo branches or pull requestsIssue actions |