GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — bpo-31783: Fix a race condition creating workers during shutdown by brianquinlan · Pull Request #13171 · python/cpython

Conversation

@brianquinlan

@brianquinlan brianquinlan commented May 7, 2019 •
edited by bedevere-bot
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Contributor

https://bugs.python.org/issue31783

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
@brianquinlan
bpo-31783: Fix a race condition while creating workers during interpr…
b4222fd
…eter shutdown
@the-knights-who-say-ni the-knights-who-say-ni added the CLA signed label May 7, 2019
@bedevere-bot bedevere-bot added the awaiting review label May 7, 2019
@blurb-it
📜🤖 Added by blurb_it.
e1f0027
@markshannon

markshannon commented May 7, 2019

Copy link
Copy Markdown
Member

Can you add a test for this, thanks.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@brianquinlan

brianquinlan commented May 8, 2019 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Contributor Author

The race condition occurs with the following set of context switches:

Main Thread                           Thread  #1
========= .                         ========
1.                                   ThreadPoolExecutor.submit()
                                        ...
                                        if _shutdown:

                    <=====

2. thread._python_exit
      _shutdown = true
      items = list(_threads_queues.items())

                    =====>

3.                                      ...
                                         _threads_queues[t] = self._work_queue

                    <=====

4. 
    for t, q in items:
        q.put(None)
    for t, q in items:
        t.join()

At this point there is a new thread added to _threads_queues but thread._python_exit won't join it because it was added to _threads_queues after a copy was made (i.e.
items = list(_threads_queues.items()))

The bug has a patch with time.sleep() to force the pathological sequencing and a test script to demonstrate the problem.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@brianquinlan

brianquinlan commented May 8, 2019

Copy link
Copy Markdown
Contributor Author

I don't think writing a test for this would be feasible because provoking the race condition requires very specific timing that, at least on my machine, I can't provoke without adding sleep calls to the code.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

brianquinlan added 2 commits May 8, 2019 14:12
@brianquinlan
Merge branch 'master' into shutdown-race
217a5ff
@brianquinlan
Merge branch 'master' into shutdown-race
c1e6922
@brianquinlan brianquinlan closed this Jun 28, 2019
@brianquinlan brianquinlan reopened this Jun 28, 2019
@brianquinlan
brianquinlan merged commit 242c26f into python:master Jun 28, 2019
@bedevere-bot

bedevere-bot commented Jun 28, 2019

Copy link
Copy Markdown

@brianquinlan: Please replace # with GH- in the commit message next time. Thanks!

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@bedevere-bot bedevere-bot removed the awaiting review label Jun 28, 2019
@brianquinlan brianquinlan added needs backport to 3.7 labels Jul 1, 2019
@miss-islington

miss-islington commented Jul 1, 2019

Copy link
Copy Markdown
Contributor

Thanks @brianquinlan for the PR 🌮🎉.. I'm working now to backport this PR to: 3.7.
🐍🍒⛏🤖

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@miss-islington

miss-islington commented Jul 1, 2019

Copy link
Copy Markdown
Contributor

Thanks @brianquinlan for the PR 🌮🎉.. I'm working now to backport this PR to: 3.8.
🐍🍒⛏🤖 I'm not a witch! I'm not a witch!

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@miss-islington

miss-islington commented Jul 1, 2019

Copy link
Copy Markdown
Contributor

Sorry, @brianquinlan, I could not cleanly backport this to 3.8 due to a conflict.
Please backport using cherry_picker on command line.
cherry_picker 242c26f53edb965e9808dd918089e664c0223407 3.8

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@miss-islington miss-islington assigned brianquinlan Jul 1, 2019
@miss-islington

miss-islington commented Jul 1, 2019

Copy link
Copy Markdown
Contributor

Sorry @brianquinlan, I had trouble checking out the 3.7 backport branch.
Please backport using cherry_picker on command line.
cherry_picker 242c26f53edb965e9808dd918089e664c0223407 3.7

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

lisroach pushed a commit to lisroach/cpython that referenced this pull request Sep 10, 2019
@brianquinlan @lisroach
bpo-31783: Fix a race condition creating workers during shutdown (pyt…
4627477
…hon#13171)

* bpo-31783: Fix a race condition while creating workers during interpreter shutdown

* 📜🤖 Added by blurb_it.
DinoV pushed a commit to DinoV/cpython that referenced this pull request Jan 14, 2020
@brianquinlan @DinoV
bpo-31783: Fix a race condition creating workers during shutdown (pyt…
adbfdfc
…hon#13171)

* bpo-31783: Fix a race condition while creating workers during interpreter shutdown

* 📜🤖 Added by blurb_it.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Uh oh!

There was an error while loading. Please reload this page.

Assignees

@brianquinlan brianquinlan

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Uh oh!

There was an error while loading. Please reload this page.

5 participants

@brianquinlan @markshannon @bedevere-bot @miss-islington @the-knights-who-say-ni
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
You are visitor 01402503 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.