GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — gh-109295: Clean up multiprocessing in test_asyncio and test_compileall by vstinner · Pull Request #109298 · python/cpython

Conversation

@vstinner

@vstinner vstinner commented Sep 12, 2023 •
edited by bedevere-app Bot
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Member

test_asyncio and test_compileall now clean up multiprocessing by calling _cleanup_tests() method: explicitly clean up resources and stop background processes like the resource tracker.

  • Issue: test_asyncio and test_compileall leak temporary files/directories #109295

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
All reactions
@vstinner
pythongh-109295: test_asyncio and test_compileall clean multiprocessing
3830ca6 
test_asyncio and test_compileall now clean up multiprocessing by
calling multiprocessing _cleanup_tests(): explicitly clean up
resources and stop background processes like the resource tracker.
@vstinner vstinner added tests Tests in the Lib/test dir needs backport to 3.11 only security fixes needs backport to 3.12 only security fixes labels Sep 12, 2023
@vstinner
vstinner requested review from 1st1, asvetlov, gvanrossum, kumaraditya303 and willingc as code owners September 12, 2023 00:13
@bedevere-app bedevere-app Bot added the awaiting review label Sep 12, 2023
@bedevere-app bedevere-app Bot mentioned this pull request Sep 12, 2023
test_asyncio and test_compileall leak temporary files/directories #109295
Closed
@vstinner vstinner added the skip news label Sep 12, 2023
@vstinner vstinner mentioned this pull request Sep 12, 2023
gh-109276: libregrtest: add RunTests.work_dir #109290
Closed
@gvanrossum gvanrossum changed the title gh-109295: test_asyncio and test_compileall clean multiprocessing gh-109295: Clean up multiprocessing in test_asyncio and test_compileall Sep 12, 2023
gvanrossum
gvanrossum reviewed Sep 12, 2023
View reviewed changes
Comment thread Lib/test/test_asyncio/test_events.py Resolved

Uh oh!

There was an error while loading. Please reload this page.
Comment thread Lib/test/test_compileall.py Resolved

Uh oh!

There was an error while loading. Please reload this page.
@vstinner
Use addCleanup()
94e8c75
gvanrossum
gvanrossum approved these changes Sep 12, 2023
View reviewed changes

@gvanrossum gvanrossum left a comment

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Gah. Why do we have compileall and py_compile in the stdlib. :-( (That's a rhetorical question.)

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
All reactions
@bedevere-app bedevere-app Bot added awaiting core review and removed awaiting review labels Sep 12, 2023
@vstinner
vstinner merged commit 09ea4b8 into python:main Sep 12, 2023
@vstinner
vstinner deleted the cleanup_tests branch September 12, 2023 01:31
@miss-islington

miss-islington commented Sep 12, 2023

Copy link
Copy Markdown
Contributor

Thanks @vstinner for the PR 🌮🎉.. I'm working now to backport this PR to: 3.11, 3.12.
🐍🍒⛏🤖
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
@bedevere-app bedevere-app Bot removed the awaiting core review label Sep 12, 2023
miss-islington pushed a commit to miss-islington/cpython that referenced this pull request Sep 12, 2023
@vstinner @miss-islington
pythongh-109295: Clean up multiprocessing in test_asyncio and test_co…
b95540d 
…mpileall (pythonGH-109298)

test_asyncio and test_compileall now clean up multiprocessing by
calling multiprocessing _cleanup_tests(): explicitly clean up
resources and stop background processes like the resource tracker.
(cherry picked from commit 09ea4b8)

Co-authored-by: Victor Stinner <[email protected]>
miss-islington pushed a commit to miss-islington/cpython that referenced this pull request Sep 12, 2023
@vstinner @miss-islington
pythongh-109295: Clean up multiprocessing in test_asyncio and test_co…
ce50dad 
…mpileall (pythonGH-109298)

test_asyncio and test_compileall now clean up multiprocessing by
calling multiprocessing _cleanup_tests(): explicitly clean up
resources and stop background processes like the resource tracker.
(cherry picked from commit 09ea4b8)

Co-authored-by: Victor Stinner <[email protected]>
@bedevere-app

bedevere-app Bot commented Sep 12, 2023

Copy link
Copy Markdown

GH-109301 is a backport of this pull request to the 3.12 branch.
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
@bedevere-app bedevere-app Bot removed the needs backport to 3.12 only security fixes label Sep 12, 2023
@bedevere-app

bedevere-app Bot commented Sep 12, 2023

Copy link
Copy Markdown

GH-109302 is a backport of this pull request to the 3.11 branch.
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
@bedevere-app bedevere-app Bot removed the needs backport to 3.11 only security fixes label Sep 12, 2023
@vstinner

vstinner commented Sep 12, 2023

Copy link
Copy Markdown
Member Author

Gah. Why do we have compileall and py_compile in the stdlib. :-( (That's a rhetorical question.)

Well, why do we have asyncio and multiprocessing in the stlidb? Why do you have a stdlib? (That's a rhetorical question.) 😂
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
@vstinner

vstinner commented Sep 12, 2023

Copy link
Copy Markdown
Member Author

I would prefer that multiprocessing and concurrent.futures would have a more explicit API to manage resources. Rather than spawning processes in the background with no public API to be aware of that, nor an API to cleanup these processes. There are temporary directories, processes, shared memory, etc. Not trivial resources like a single large memory block.
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
vstinner added a commit that referenced this pull request Sep 12, 2023
@miss-islington @vstinner
[3.11] gh-109295: Clean up multiprocessing in test_asyncio and test_c…
abf3a68 
…ompileall (GH-109298) (#109302)

gh-109295: Clean up multiprocessing in test_asyncio and test_compileall (GH-109298)

test_asyncio and test_compileall now clean up multiprocessing by
calling multiprocessing _cleanup_tests(): explicitly clean up
resources and stop background processes like the resource tracker.
(cherry picked from commit 09ea4b8)

Co-authored-by: Victor Stinner <[email protected]>
Yhg1s pushed a commit that referenced this pull request Sep 12, 2023
@miss-islington @vstinner
[3.12] gh-109295: Clean up multiprocessing in test_asyncio and test_c…
8acfb42 
…ompileall (GH-109298) (#109301)

gh-109295: Clean up multiprocessing in test_asyncio and test_compileall (GH-109298)

test_asyncio and test_compileall now clean up multiprocessing by
calling multiprocessing _cleanup_tests(): explicitly clean up
resources and stop background processes like the resource tracker.
(cherry picked from commit 09ea4b8)

Co-authored-by: Victor Stinner <[email protected]>
vstinner added a commit to vstinner/cpython that referenced this pull request Sep 13, 2023
@vstinner
pythongh-109295: Clean up multiprocessing in test_asyncio and test_co…
2a94b5f 
…mpileall (python#109298)

test_asyncio and test_compileall now clean up multiprocessing by
calling multiprocessing _cleanup_tests(): explicitly clean up
resources and stop background processes like the resource tracker.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@gvanrossum gvanrossum gvanrossum approved these changes
@1st1 1st1 Awaiting requested review from 1st1 1st1 is a code owner
@asvetlov asvetlov Awaiting requested review from asvetlov asvetlov is a code owner
@kumaraditya303 kumaraditya303 Awaiting requested review from kumaraditya303 kumaraditya303 is a code owner
@willingc willingc Awaiting requested review from willingc willingc is a code owner

Uh oh!

There was an error while loading. Please reload this page.

Assignees

No one assigned

Labels

skip news tests Tests in the Lib/test dir

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Uh oh!

There was an error while loading. Please reload this page.

3 participants

@vstinner @miss-islington @gvanrossum
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
You are visitor 01402503 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.