GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — gh-98894: Quote test_dtrace tracer subcommands by stratakis · Pull Request #152901 · python/cpython

Conversation

@stratakis

@stratakis stratakis commented Jul 2, 2026 •
edited by bedevere-app Bot
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Contributor

Use shlex.join() when building tracer -c command strings for avoiding issues with spaces.

  • Issue: Some DTrace probes are broken in 3.11 #98894

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
@bedevere-app bedevere-app Bot added tests Tests in the Lib/test dir awaiting review labels Jul 2, 2026
@bedevere-app bedevere-app Bot mentioned this pull request Jul 2, 2026
Some DTrace probes are broken in 3.11 #98894
Open
@stratakis
pythongh-98894: Quote test_dtrace tracer subcommands
eea0a12
Use shlex.join() when building tracer -c command strings
for avoiding issues with spaces.
@stratakis
stratakis force-pushed the shlex_dtrace branch from d97d139 to eea0a12 Compare July 2, 2026 15:28
@stratakis

stratakis commented Jul 2, 2026

Copy link
Copy Markdown
Contributor Author

cc @vstinner .

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

vstinner
vstinner reviewed Jul 3, 2026
View reviewed changes

@vstinner vstinner left a comment

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you also modify f"Command {' '.join(command)!r} failed " to use shlex.join() in run_readelf() and trace()?

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
@vstinner vstinner added skip news needs backport to 3.15 bugs and security fixes labels Jul 3, 2026
@stratakis
More shlex
0fa221b
@vstinner

vstinner commented Jul 9, 2026

Copy link
Copy Markdown
Member

I tried to create a path with a space to test if the command/paths are properly quoted.

I renamed the dtracedata/ directory to dtrace data/: mv Lib/test/dtracedata/ "Lib/test/dtrace data/".

I applied this patch:

diff --git a/Lib/test/test_dtrace.py b/Lib/test/test_dtrace.py
index 30731b8f90a..b890f467807 100644
--- a/Lib/test/test_dtrace.py
+++ b/Lib/test/test_dtrace.py
@@ -22,7 +22,7 @@
 
 
 def abspath(filename):
-    return os.path.abspath(findfile(filename, subdir="dtracedata"))
+    return os.path.abspath(findfile(filename, subdir="dtrace data"))
 
 
 def normalize_trace_output(output):

For example, this bpftrace program test fails:

sudo ./python -m test -v test_dtrace -m test.test_dtrace.BPFTraceOptimizedTests.test_gc

I extracted the executed command and simplified it:

$ sudo bpftrace -e 'usdt:/home/vstinner/python/main/python:python:function__entry {}' -c "/home/vstinner/python/main/python -O -O '/home/vstinner/python/main/Lib/test/dtrace data/gc.py'"
Attached 1 probe
/home/vstinner/python/main/python: can't open file "/home/vstinner/python/main/'/home/vstinner/python/main/Lib/test/dtrace": [Errno 2] No such file or directory

So python fails to find its filename parameter.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@vstinner

vstinner commented Jul 9, 2026

Copy link
Copy Markdown
Member

Same error if I try to create a Python executable path with a space in assert_usable():

$ git diff
diff --git a/Lib/test/test_dtrace.py b/Lib/test/test_dtrace.py
index 30731b8f90a..011507bd5c3 100644
--- a/Lib/test/test_dtrace.py
+++ b/Lib/test/test_dtrace.py
@@ -314,9 +314,15 @@ def assert_usable(self):
         # Check if bpftrace is available and can attach to USDT probes
         program = f'usdt:{sys.executable}:python:function__entry {{ printf("probe: success\\n"); exit(); }}'
         try:
+            exe = sys.executable + ' .exe'
+            import shutil
+            shutil.copy2(sys.executable, exe)
+            cmd = shlex.join([exe, "-c", "pass"])
+            print("CMD", cmd)
+            cmd = ["bpftrace", "-e", program, "-c", cmd]
+            print("CMD", cmd)
             proc = create_process_group(
-                ["bpftrace", "-e", program, "-c",
-                 shlex.join([sys.executable, "-c", "pass"])],
+                cmd,
                 stdout=subprocess.PIPE,
                 stderr=subprocess.PIPE,
                 universal_newlines=True,

If I run manually bpftrace, it also fails:

$ sudo bpftrace -e 'usdt:/home/vstinner/python/main/python:python:function__entry { printf("probe: success\\n"); exit(); }' '-c' "'/home/vstinner/python/main/python .exe' -c pass"
ERROR: Failed to fork child: path ''/home/vstinner/python/main/python' does not exist or is not executable
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

vstinner
vstinner approved these changes Jul 9, 2026
View reviewed changes

@vstinner vstinner left a comment

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. The AssertionError changes are useful. I'm less sure that the bpftrace are useful, but well, they look correct at least. It's just that bpftrace doesn't handle properly commands (-c argument) with quotes.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
Comment thread Lib/test/test_dtrace.py Resolved

Uh oh!

There was an error while loading. Please reload this page.

@bedevere-app bedevere-app Bot added awaiting merge and removed awaiting review labels Jul 9, 2026
@vstinner
vstinner merged commit a159c68 into python:main Jul 9, 2026
52 checks passed

Uh oh!

There was an error while loading. Please reload this page.

@bedevere-app bedevere-app Bot removed the awaiting merge label Jul 9, 2026
@miss-islington-app

miss-islington-app Bot commented Jul 9, 2026

Copy link
Copy Markdown

Thanks @stratakis for the PR, and @vstinner for merging it 🌮🎉.. I'm working now to backport this PR to: 3.15.
🐍🍒⛏🤖

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@bedevere-app

bedevere-app Bot commented Jul 9, 2026

Copy link
Copy Markdown

GH-153459 is a backport of this pull request to the 3.15 branch.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@bedevere-app bedevere-app Bot removed the needs backport to 3.15 bugs and security fixes label Jul 9, 2026
@vstinner

vstinner commented Jul 9, 2026

Copy link
Copy Markdown
Member

Merged. Thanks for the fix!

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

vstinner pushed a commit that referenced this pull request Jul 9, 2026
@miss-islington @stratakis
[3.15] gh-98894: Quote test_dtrace tracer subcommands (GH-152901) (#1…
ef5569e
…53459)

gh-98894: Quote test_dtrace tracer subcommands (GH-152901)

Use shlex.join() when building tracer -c command strings
for avoiding issues with spaces.
(cherry picked from commit a159c68)

Co-authored-by: stratakis <[email protected]>
@stratakis
stratakis deleted the shlex_dtrace branch July 9, 2026 22:03
vstinner added a commit that referenced this pull request Oct 16, 2026
@vstinner @stratakis
[3.14] gh-156701: Backport test_dtrace from main (#159596)
566f815
* gh-98894: Fix DTrace test_check_probes for shared builds (#151122)

When building with --enable-shared, the SystemTap/DTrace notes
live in libpython. Add detection logic to be used by readelf.

Force the C locale on readelf output.

(cherry picked from commit 3a08e93)

* gh-151496: Use process groups in test_dtrace (#151512)

Create a new process group to run bpftrace commands, so it's possible
to kill also child processes on timeout.

(cherry picked from commit a064b32)

* gh-151496: Use process groups in TraceBackend in test_dtrace (#152039)

Run the generic DTrace/SystemTap commands in a new process group and
terminate the whole group on timeout.

This prevents a forked tracer child from keeping stdout/stderr pipes
open after the direct tracer process is killed.

(cherry picked from commit 1785f4b)

* gh-98894: Check readelf failures in test_dtrace (#152345)

Report readelf command failures directly instead of later failing with
missing-probe assertions.

(cherry picked from commit d60a677)

* gh-98894: Check tracer exit status in test_dtrace (#152893)

Fail functional dtrace and SystemTap cases directly when the tracer exits
non-zero.

(cherry picked from commit 311e7e7)

* gh-98894: Quote test_dtrace tracer subcommands (#152901)

Use shlex.join() when building tracer -c command strings
for avoiding issues with spaces.

(cherry picked from commit a159c68)

* gh-98894: Fix dtrace tests in shared builds (#153372)

Generate SystemTap probe definitions targeting libpython for shared
builds and use centralized USDT probe object discovery for readelf
and BPFTrace.

(cherry picked from commit 298bef3)

* gh-156703: Set LD_LIBRARY_PATH in SystemTap trace commands (GH-156424)

Prefix commands passed to stap -c with LD_LIBRARY_PATH via env, when
probes reside in libpython. This allows the interpreter to find
libpython when staprun does not preserve the loader path.

(cherry picked from commit faf4e52)

* gh-156722, test_dtrace: Terminate tracing processes gracefully on timeout (#156753)

Send SIGTERM to the tracing process group before falling back to SIGKILL.

(cherry picked from commit 55ecc0a)

* gh-156701: Increase the SystemTap usability timeout (#156720)

SystemTap may compile a kernel module during its usability check
(if no cached module is available). We need to allow a larger timeout
for that case.

(cherry picked from commit e1be613)

---------

Co-authored-by: stratakis <[email protected]>
Co-authored-by: stratakis <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@vstinner vstinner vstinner approved these changes

Uh oh!

There was an error while loading. Please reload this page.

Assignees

No one assigned

Labels

skip news tests Tests in the Lib/test dir

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Uh oh!

There was an error while loading. Please reload this page.

2 participants

@stratakis @vstinner
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
You are visitor 01402503 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.