GeezerVison is showing github.com — gh-135953: Add Gecko reporter to sampling profiler by ivonastojanovic · Pull Request #139364 · python/cpython
Conversation
ivonastojanovic
commented
Sep 26, 2025
•
edited by bedevere-app
Bot
Loading
Uh oh!
There was an error while loading. Please reload this page.
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull Request Overview
This PR adds Gecko reporter functionality to the sampling profiler, enabling export of profiling data in Gecko Profile format for web-based visualization tools. The implementation includes a new GeckoCollector class and supporting data structures.
- Adds new gecko_format.py module with dataclasses and builder for Gecko profile format
- Implements GeckoCollector class in stack_collector.py that exports profiling data as JSON
- Updates sample.py to support --gecko command line option and integrate the new collector
Reviewed Changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated 2 comments.
| File |
Description |
| Lib/profiling/sampling/gecko_format.py |
New module defining Gecko profile format dataclasses and builder |
| Lib/profiling/sampling/stack_collector.py |
Adds GeckoCollector class and category constants |
| Lib/profiling/sampling/sample.py |
Integrates Gecko collector option and updates CLI help text |
Comments suppressed due to low confidence (1)
Lib/profiling/sampling/stack_collector.py:1
- GeckoCollector is instantiated without the skip_idle parameter that other collectors receive. Consider whether skip_idle functionality should be supported for consistency with other collectors.
Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
|
A Python core developer has requested some changes be made to your pull request before we can consider merging it. If you could please address their requests along with any other requests in other reviews from core developers that would be appreciated.
Once you have made the requested changes, please leave a comment on this pull request containing the phrase I have made the requested changes; please review again. I will then notify any core developers who have left a review that you're ready for them to take another look at this pull request.
|
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
|
There are several problems here:
- The file doesn't really work. If you try to lead it into the firefox profiler it just fails with "Couldn’t read the file or parse the profile in it".
- Generating the profiler with
./python -m profiling.sampling -i 1 -d100000 --realtime-stats --all-threads --gecko -o flamegraph.gecko -m test test_asyncio takes 2GB of memory. That's unfortunately not acceptable
- Generating the profiler when the process ends is super slow (takes 40 seconds on my machine).
|
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
|
This also will need some tests
|
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
|
@ivonastojanovic I have pushed a draft version of a simpler collector that uses raw dictionaries. This can be processed by the firefox profiler, doesn't blow up the memory in my experiments and also is relatively faster. It needs some polishing but should cover most problems.
|
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️
Hi! The buildbot AMD64 Ubuntu Shared 3.x (tier-1) has failed when building commit 75b1afe.
What do you need to do:
- Don't panic.
- Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
- Go to the page of the buildbot that failed (https://buildbot.python.org/#/builders/506/builds/11528) and take a look at the build logs.
- Check if the failure is related to this commit (75b1afe) or if it is a false positive.
- If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.
You can take a look at the buildbot page here:
https://buildbot.python.org/#/builders/506/builds/11528
Failed tests:
- test_profiling
- test_pyrepl
Failed subtests:
- test_sample_target_script - test.test_profiling.test_sampling_profiler.TestSampleProfilerIntegration.test_sample_target_script
- test_repl_eio - test.test_pyrepl.test_unix_console.TestUnixConsoleEIOHandling.test_repl_eio
Summary of the results of the build (if available):
==
Click to see traceback logs
Traceback (most recent call last):
File "/srv/buildbot/buildarea/3.x.bolen-ubuntu/build/Lib/subprocess.py", line 1137, in __del__
_warn("subprocess %s is still running" % self.pid,
~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ResourceWarning, source=self)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ResourceWarning: subprocess 3224727 is still running
Warning -- Unraisable exception
Exception ignored while finalizing file <_io.FileIO name=8 mode='rb' closefd=True>:
Traceback (most recent call last):
File "/srv/buildbot/buildarea/3.x.bolen-ubuntu/build/Lib/test/libregrtest/single.py", line 165, in _load_run_test
regrtest_runner(result, test_func, runtests)
~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/srv/buildbot/buildarea/3.x.bolen-ubuntu/build/Lib/test/libregrtest/single.py", line 118, in regrtest_runner
test_result = test_func()
File "/srv/buildbot/buildarea/3.x.bolen-ubuntu/build/Lib/test/libregrtest/single.py", line 162, in test_func
return run_unittest(test_mod, runtests)
File "/srv/buildbot/buildarea/3.x.bolen-ubuntu/build/Lib/test/libregrtest/single.py", line 42, in run_unittest
return _run_suite(tests)
File "/srv/buildbot/buildarea/3.x.bolen-ubuntu/build/Lib/test/libregrtest/single.py", line 105, in _run_suite
raise support.TestFailedWithDetails(err, errors, failures, stats=stats)
Traceback (most recent call last):
File "/srv/buildbot/buildarea/3.x.bolen-ubuntu/build/Lib/test/test_profiling/test_sampling_profiler.py", line 1924, in test_sample_target_script
self.assertIn("slow_fibonacci", output)
~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^
AssertionError: 'slow_fibonacci' not found in 'Captured 9986 samples in 1.00 seconds\nSample rate: 9985.99 samples/sec\nError rate: 0.00%\nWarning: missed 14 samples from the expected total of 10000 (0.14%)\n\x1b[1;34mProfile Stats:\x1b[0m\n\x1b[1;34m nsamples\x1b[0m \x1b[1;34m sample%\x1b[0m \x1b[1;34mtottime (ms)\x1b[0m \x1b[1;34m cumul%\x1b[0m \x1b[1;34mcumtime (ms)\x1b[0m \x1b[1;34mfilename:lineno(function)\x1b[0m\n 0/9986 0.0 0.000 100.0 998.600 \x1b[32mtmpy9ith0_b\x1b[0m:\x1b[33m63\x1b[0m(\x1b[36m<module>\x1b[0m)\n 0/9918 0.0 0.000 99.3 991.800 \x1b[32mtmpy9ith0_b\x1b[0m:\x1b[33m52\x1b[0m(\x1b[36mmain_loop\x1b[0m)\n 5005/5005 50.1 500.500 50.1 500.500 \x1b[32mtmpy9ith0_b\x1b[0m:\x1b[33m15\x1b[0m(\x1b[36mcpu_intensive_work\x1b[0m)\n 3232/3232 32.4 323.200 32.4 323.200 \x1b[32mtmpy9ith0_b\x1b[0m:\x1b[33m16\x1b[0m(\x1b[36mcpu_intensive_work\x1b[0m)\n 1651/1651 16.5 165.100 16.5 165.100 \x1b[32mtmpy9ith0_b\x1b[0m:\x1b[33m14\x1b[0m(\x1b[36mcpu_intensive_work\x1b[0m)\n 1/68 0.0 0.100 0.7 6.800 \x1b[32mtmpy9ith0_b\x1b[0m:\x1b[33m40\x1b[0m(\x1b[36mnested_calls\x1b[0m)\n 0/68 0.0 0.000 0.7 6.800 \x1b[32mtmpy9ith0_b\x1b[0m:\x1b[33m58\x1b[0m(\x1b[36mmain_loop\x1b[0m)\n 0/67 0.0 0.000 0.7 6.700 \x1b[32mtmpy9ith0_b\x1b[0m:\x1b[33m38\x1b[0m(\x1b[36mnested_calls.<locals>.level1.<locals>.level2\x1b[0m)\n 0/67 0.0 0.000 0.7 6.700 \x1b[32mtmpy9ith0_b\x1b[0m:\x1b[33m39\x1b[0m(\x1b[36mnested_calls.<locals>.level1\x1b[0m)\n 42/42 0.4 4.200 0.4 4.200 \x1b[32mtmpy9ith0_b\x1b[0m:\x1b[33m24\x1b[0m(\x1b[36mmedium_computation\x1b[0m)\n 30/30 0.3 3.000 0.3 3.000 \x1b[32mtmpy9ith0_b\x1b[0m:\x1b[33m17\x1b[0m(\x1b[36mcpu_intensive_work\x1b[0m)\n 25/25 0.3 2.500 0.3 2.500 \x1b[32mtmpy9ith0_b\x1b[0m:\x1b[33m23\x1b[0m(\x1b[36mmedium_computation\x1b[0m)\n\n\x1b[1;34mLegend:\x1b[0m\n \x1b[33mnsamples\x1b[0m: Direct/Cumulative samples (direct executing / on call stack)\n \x1b[33msample%\x1b[0m: Percentage of total samples this function was directly executing\n \x1b[33mtottime\x1b[0m: Estimated total time spent directly in this function\n \x1b[33mcumul%\x1b[0m: Percentage of total samples when this function was on the call stack\n \x1b[33mcumtime\x1b[0m: Estimated cumulative time (including time in called functions)\n \x1b[33mfilename:lineno(function)\x1b[0m: Function location and name\n\n\x1b[1;34mSummary of Interesting Functions:\x1b[0m\n\n\x1b[1;34mFunctions with Highest Direct/Cumulative Ratio (Hot Spots):\x1b[0m\n 1.000 direct/cumulative ratio, 99.3% direct samples: \x1b[32mtmpy9ith0_b\x1b[0m:\x1b[33m\x1b[0m(\x1b[36mcpu_intensive_work\x1b[0m)\n 1.000 direct/cumulative ratio, 0.7% direct samples: \x1b[32mtmpy9ith0_b\x1b[0m:\x1b[33m\x1b[0m(\x1b[36mmedium_computation\x1b[0m)\n 0.015 direct/cumulative ratio, 0.0% direct samples: \x1b[32mtmpy9ith0_b\x1b[0m:\x1b[33m\x1b[0m(\x1b[36mnested_calls\x1b[0m)\n\n\x1b[1;34mFunctions with Highest Call Frequency (Indirect Calls):\x1b[0m\n 9986 indirect calls, 100.0% total stack presence: \x1b[32mtmpy9ith0_b\x1b[0m:\x1b[33m\x1b[0m(\x1b[36m<module>\x1b[0m)\n 9986 indirect calls, 100.0% total stack presence: \x1b[32mtmpy9ith0_b\x1b[0m:\x1b[33m\x1b[0m(\x1b[36mmain_loop\x1b[0m)\n 67 indirect calls, 0.7% total stack presence: \x1b[32mtmpy9ith0_b\x1b[0m:\x1b[33m\x1b[0m(\x1b[36mnested_calls\x1b[0m)\n\n\x1b[1;34mFunctions with Highest Call Magnification (Cumulative/Direct):\x1b[0m\n 68.0x call magnification, 67 indirect calls from 1 direct: \x1b[32mtmpy9ith0_b\x1b[0m:\x1b[33m\x1b[0m(\x1b[36mnested_calls\x1b[0m)\n'
Traceback (most recent call last):
File "/srv/buildbot/buildarea/3.x.bolen-ubuntu/build/Lib/test/support/__init__.py", line 837, in gc_collect
gc.collect()
~~~~~~~~~~^^
ResourceWarning: unclosed file <_io.FileIO name=8 mode='rb' closefd=True>
Warning -- Unraisable exception
Exception ignored while finalizing file <_io.FileIO name=10 mode='rb' closefd=True>:
Traceback (most recent call last):
File "/srv/buildbot/buildarea/3.x.bolen-ubuntu/build/Lib/test/libregrtest/single.py", line 165, in _load_run_test
regrtest_runner(result, test_func, runtests)
~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/srv/buildbot/buildarea/3.x.bolen-ubuntu/build/Lib/test/libregrtest/single.py", line 118, in regrtest_runner
test_result = test_func()
File "/srv/buildbot/buildarea/3.x.bolen-ubuntu/build/Lib/test/libregrtest/single.py", line 162, in test_func
return run_unittest(test_mod, runtests)
File "/srv/buildbot/buildarea/3.x.bolen-ubuntu/build/Lib/test/libregrtest/single.py", line 42, in run_unittest
return _run_suite(tests)
File "/srv/buildbot/buildarea/3.x.bolen-ubuntu/build/Lib/test/libregrtest/single.py", line 105, in _run_suite
raise support.TestFailedWithDetails(err, errors, failures, stats=stats)
Traceback (most recent call last):
File "/srv/buildbot/buildarea/3.x.bolen-ubuntu/build/Lib/test/libregrtest/single.py", line 165, in _load_run_test
regrtest_runner(result, test_func, runtests)
~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/srv/buildbot/buildarea/3.x.bolen-ubuntu/build/Lib/test/libregrtest/single.py", line 118, in regrtest_runner
test_result = test_func()
File "/srv/buildbot/buildarea/3.x.bolen-ubuntu/build/Lib/test/libregrtest/single.py", line 162, in test_func
return run_unittest(test_mod, runtests)
File "/srv/buildbot/buildarea/3.x.bolen-ubuntu/build/Lib/test/libregrtest/single.py", line 42, in run_unittest
return _run_suite(tests)
File "/srv/buildbot/buildarea/3.x.bolen-ubuntu/build/Lib/test/libregrtest/single.py", line 105, in _run_suite
raise support.TestFailedWithDetails(err, errors, failures, stats=stats)
Traceback (most recent call last):
File "/srv/buildbot/buildarea/3.x.bolen-ubuntu/build/Lib/subprocess.py", line 1137, in __del__
_warn("subprocess %s is still running" % self.pid,
~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ResourceWarning, source=self)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ResourceWarning: subprocess 3247738 is still running
Warning -- Unraisable exception
Exception ignored while finalizing file <_io.FileIO name=8 mode='rb' closefd=True>:
Traceback (most recent call last):
File "/srv/buildbot/buildarea/3.x.bolen-ubuntu/build/Lib/test/libregrtest/single.py", line 165, in _load_run_test
regrtest_runner(result, test_func, runtests)
~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/srv/buildbot/buildarea/3.x.bolen-ubuntu/build/Lib/test/libregrtest/single.py", line 118, in regrtest_runner
test_result = test_func()
File "/srv/buildbot/buildarea/3.x.bolen-ubuntu/build/Lib/test/libregrtest/single.py", line 162, in test_func
return run_unittest(test_mod, runtests)
File "/srv/buildbot/buildarea/3.x.bolen-ubuntu/build/Lib/test/libregrtest/single.py", line 42, in run_unittest
return _run_suite(tests)
File "/srv/buildbot/buildarea/3.x.bolen-ubuntu/build/Lib/test/libregrtest/single.py", line 105, in _run_suite
raise support.TestFailedWithDetails(err, errors, failures, stats=stats)
Traceback (most recent call last):
File "/srv/buildbot/buildarea/3.x.bolen-ubuntu/build/Lib/test/support/__init__.py", line 837, in gc_collect
gc.collect()
~~~~~~~~~~^^
ResourceWarning: unclosed file <_io.FileIO name=10 mode='rb' closefd=True>
Traceback (most recent call last):
File "/srv/buildbot/buildarea/3.x.bolen-ubuntu/build/Lib/test/test_pyrepl/test_unix_console.py", line 362, in test_repl_eio
_, err = proc.communicate(timeout=5) # sleep for pty to settle
~~~~~~~~~~~~~~~~^^^^^^^^^^^
File "/srv/buildbot/buildarea/3.x.bolen-ubuntu/build/Lib/subprocess.py", line 1219, in communicate
stdout, stderr = self._communicate(input, endtime, timeout)
~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^
File "/srv/buildbot/buildarea/3.x.bolen-ubuntu/build/Lib/subprocess.py", line 2126, in _communicate
self._check_timeout(endtime, orig_timeout, stdout, stderr)
~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/srv/buildbot/buildarea/3.x.bolen-ubuntu/build/Lib/subprocess.py", line 1266, in _check_timeout
raise TimeoutExpired(
...<2 lines>...
stderr=b''.join(stderr_seq) if stderr_seq else None)
subprocess.TimeoutExpired: Command '['/srv/buildbot/buildarea/3.x.bolen-ubuntu/build/python', '-E', '-S', '/srv/buildbot/buildarea/3.x.bolen-ubuntu/build/Lib/test/test_pyrepl/eio_test_script.py']' timed out after 5 seconds
|
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
|
This looks unrelated to this PR as this PR only adds test_gecko_collector_basic
|
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
Reviewers
+1 more reviewer
Reviewers whose approvals may not affect merge requirements
Uh oh!
There was an error while loading. Please reload this page.
Assignees
No one assigned
Milestone
No milestone
Development
Successfully merging this pull request may close these issues.
Uh oh!
There was an error while loading. Please reload this page.
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
|