GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — Make multiprocessing `SemLock` thread-safe in the free-threaded build · Issue #117435 · python/cpython

Make multiprocessing SemLock thread-safe in the free-threaded build #117435

New issue
Copy link
New issue
Copy link
Closed
Closed
Make multiprocessing SemLock thread-safe in the free-threaded build#117435
Copy link
Labels
topic-free-threadingtype-featureA feature request or enhancementA feature request or enhancement

Description

@colesbury
colesbury
opened on Apr 1, 2024
Issue body actions

Feature or enhancement

The multiprocessing SemLock code maintains an internal count field. When operating as a recursive mutex, count is the number of times the thread has acquired the mutex (i.e., 0, 1, .. N). When operating as a semaphore, count is often 0 or 1, but can be negative if the SemLock is initialized with maxvalue > 1

The modification of count is not thread-safe without the GIL within the process. Note that the count field is not shared across processes, unlike the underlying sem_t or HANDLE.

In particular, the modification of count after the semaphore is released is not thread-safe without the GIL:

cpython/Modules/_multiprocessing/semaphore.c

Lines 444 to 448 in 9dae05e

if (sem_post(self->handle) < 0)
return PyErr_SetFromErrno(PyExc_OSError);
--self->count;
Py_RETURN_NONE;

I think this is the source of deadlocks when running test_multiprocessing_forkserver.test_processes with the GIL disabled.

Linked PRs

  • gh-117435: Make SemLock thread-safe in free-threaded build #117436

Activity

  1. colesbury
    added
    type-featureA feature request or enhancementA feature request or enhancement
    topic-free-threading
    on Apr 1, 2024
  2. colesbury
    changed the title [-]Make multiprocessing semaphore thread-safe in the free-threaded build[/-] [+]Make multiprocessing `SemLock` thread-safe in the free-threaded build[/+] on Apr 1, 2024
  3. colesbury
    added 4 commits that reference this issue on Apr 1, 2024

    pythongh-117435: Make `SemLock` thread-safe in free-threaded build

    dcdff0d

    Merge branch 'pythongh-117435-semlock' into nogil-integration

    367ccf0

    Merge branch 'pythongh-117435-semlock' into nogil-integration

    c9e481a

    gh-117435: Make `SemLock` thread-safe in free-threaded build (#117436)

    de5ca0b
  4. colesbury
    closed this as completedon Apr 10, 2024
  5. diegorusso
    added a commit that references this issue on Apr 17, 2024

    pythongh-117435: Make `SemLock` thread-safe in free-threaded build (p…

    2b7b6b3
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    topic-free-threadingtype-featureA feature request or enhancementA feature request or enhancement

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions

      You are visitor 01402503 — GeezerVison does not store the pages you look up.
      Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.