GeezerVisonThe World Wide Web, restored to 800 pixels of glory. |
| Home|About|Gaming cabinet|Archives|How it works|Starter sites |
GeezerVison is showing github.com — Hard to debug subprocess crashes when using concurrent.futures.ProcessPoolExecutor · Issue #139462 · python/cpython Closed Closed Copy link Labels stdlibStandard Library Python modules in the Lib/ directoryStandard Library Python modules in the Lib/ directorytopic-multiprocessingtype-featureA feature request or enhancementA feature request or enhancement DescriptionFeature or enhancementProposal:BackgroundWhen using a This doesn't tell you two very important bits of information:
When running a nontrivial application, odds are that you'll run into a BrokenProcessPool, whether because a subprocess segfaulted, OOMed, or something else, and having some basic information about what exactly died and why would be incredibly helpful for correlating crashes with logs/telemetry/etc and aiding further investigation. As a possible but suboptimal workaround, you can inspect the private from concurrent.futures import as_completed, ProcessPoolExecutor from concurrent.futures.process import BrokenProcessPool import ctypes import os import traceback def worker_ok() -> str: print(f"worker_ok {os.getpid()=}", flush=True) return "ok" def worker_segfault(): print(f"worker_segfault {os.getpid()=}", flush=True) ctypes.string_at(0) if __name__ == "__main__": with ProcessPoolExecutor(max_workers=2) as executor: futures = [ executor.submit(worker_ok), executor.submit(worker_segfault), ] for future in as_completed(futures): try: future.result() except BrokenProcessPool as e: # Need to look at private _processes to figure out what died and why for pid, p in executor._processes.items(): if p.exitcode: print(f"Process {pid} failed with exitcode {p.exitcode}") traceback.print_exception(e) Accessing ProposalThere is already a concept in the codebase of a I've implemented the above approach, and running the example script above with my change, I now see: Note the If this looks good to everyone, I'll raise a PR shortly. Has this already been discussed elsewhere?This is a minor feature, which does not need previous discussion elsewhere Links to previous discussion of this feature:No response Linked PRsReacted by Gregory P. Smith Activity
MetadataMetadataAssigneesLabelsstdlibStandard Library Python modules in the Lib/ directoryStandard Library Python modules in the Lib/ directorytopic-multiprocessingtype-featureA feature request or enhancementA feature request or enhancement ProjectsMilestoneRelationshipsDevelopmentIssue actions |
| You are visitor 01402503 — GeezerVison does not store the pages you look up. Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required. |