GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — gh-116738: Make bisect module thread-safe by yoney · Pull Request #137021 · python/cpython

Conversation

@yoney

@yoney yoney commented Jul 22, 2025 •
edited by bedevere-app Bot
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Contributor

Modify the insort_left() and insort_right() functions in the bisect module to be thread-{aware,safe} by adding a lock to the sequence argument. While these functions did not crash before, it is more intuitive for them to lock the sequence they are about to modify. These functions locate the correct position to insert an item and then perform the insertion. By adding a lock, the sequence remains safe from concurrent modifications by other threads during this process.

The new tests failed before introducing the lock.

cc: @mpage @Yhg1s @colesbury

  • Issue: Audit all built-in modules for thread safety #116738

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
@yoney
pythongh-116738: Make bisect module thread-safe
2eb25b1
@bedevere-app bedevere-app Bot mentioned this pull request Jul 22, 2025
Audit all built-in modules for thread safety #116738
Open
@yoney
yoney marked this pull request as ready for review July 22, 2025 20:03
@yoney
yoney requested a review from rhettinger as a code owner July 22, 2025 20:03
@bedevere-app bedevere-app Bot added the awaiting review label Jul 22, 2025
@mpage
mpage requested review from Yhg1s, colesbury and mpage July 22, 2025 20:09
@ZeroIntensity ZeroIntensity added the topic-free-threading label Jul 22, 2025
colesbury
colesbury reviewed Jul 22, 2025
View reviewed changes

@colesbury colesbury left a comment

Copy link
Copy Markdown
Contributor

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The pure Python implementations in bsiect.py are still not going to be thread safe.

While these functions did not crash before, it is more intuitive for them to lock the sequence they are about to modify...

I'm not fully convinced this is worth it. I think it's okay for bisect not to be thread safe, but I don't feel very strongly about this one way or the other.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

yoney reacted with thumbs up emoji
All reactions
  • 👍 1 reaction
Comment thread Lib/test/test_free_threading/test_bisect.py
from test.support import import_helper, threading_helper, subTests
from test.support.threading_helper import run_concurrently

bisect = import_helper.import_module("bisect")

@colesbury colesbury Jul 22, 2025

Copy link
Copy Markdown
Contributor

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can this just be import bisect?

If I understand correctly, import_helper.import_module() handles missing optional modules but that doesn't seem to be relevant here.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

yoney reacted with thumbs up emoji
All reactions
  • 👍 1 reaction
@yoney

yoney commented Jul 22, 2025

Copy link
Copy Markdown
Contributor Author

I'm not fully convinced this is worth it. I think it's okay for bisect not to be thread safe, but I don't feel very strongly about this one way or the other.

I don't feel strongly about landing it. If we agree current state is good enough, we can go ahead and mark the module as safe and close the PR.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@yoney

yoney commented Jul 22, 2025

Copy link
Copy Markdown
Contributor Author

The pure Python implementations in bsiect.py are still not going to be thread safe.

Yes, pure Python version would be different, which is a good reason to consider dropping this change.
Just to clarify my understanding; there is noway to lock the mutex on the object from Python, right?

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@colesbury

colesbury commented Jul 22, 2025

Copy link
Copy Markdown
Contributor

Just to clarify my understanding; there is noway to lock the mutex on the object from Python, right?

No, we don't expose a way to lock the object. Critical sections are not very useful from Python because the eval breaker checks can run arbitrary code (such as via the GC) and that can temporarily suspend the active critical section. So things would only be "atomic most of the time", which is not a very useful property.

yoney reacted with thumbs up emoji
All reactions
  • 👍 1 reaction

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@ZeroIntensity

ZeroIntensity commented Jul 22, 2025

Copy link
Copy Markdown
Member

The pure Python implementations in bsiect.py are still not going to be thread safe.

I think this is an issue with several other modules already. For example, the pure-Python implementation of heapq is not thread-safe while the C implementation is.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@rhettinger
rhettinger removed their request for review July 23, 2025 02:46
@rhettinger

rhettinger commented Jul 23, 2025

Copy link
Copy Markdown
Contributor

I think this is an issue with several other modules already. For example, the pure-Python implementation of heapq is not thread-safe while the C implementation is.

ISTM that all we want in these cases is for the C implementation to not segfault. There is no need to promise more than the pure python version promises. If the heapq module was edited to do more, that should probably be reverted. The C versions are intended to just be accelerators.

yoney reacted with thumbs up emoji
All reactions
  • 👍 1 reaction

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@colesbury

colesbury commented Jul 23, 2025

Copy link
Copy Markdown
Contributor

I think with heapq the locks were necessary to make it not segfault under race conditions because it directly accessed list internals. I don't think that's the case here because we're using the abstract sequence API instead.

yoney reacted with thumbs up emoji
All reactions
  • 👍 1 reaction

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@kumaraditya303

kumaraditya303 commented Jul 23, 2025

Copy link
Copy Markdown
Contributor

While these functions did not crash before, it is more intuitive for them to lock the sequence they are about to modify.

I think instead of adding locks for this, we should document the thread safety of this module like as in #136555

ZeroIntensity and yoney reacted with thumbs up emoji
All reactions
  • 👍 2 reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@ZeroIntensity

ZeroIntensity commented Jul 23, 2025

Copy link
Copy Markdown
Member

To clarify, I'm not saying we should necessarily add locking to bisect, I just think we shouldn't reject the idea on the basis that it would create differences between the C and Python implementation, because we already do that in other modules.

There is no need to promise more than the pure python version promises. If the heapq module was edited to do more, that should probably be reverted. The C versions are intended to just be accelerators.

The problem is that by adding memory safety to things like heapq (i.e., making it not crash via locking), we're also incidentally adding thread safety. But, then the Python implementation remains memory safe without thread safety (i.e., doesn't crash but also doesn't really work).

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@serhiy-storchaka
serhiy-storchaka requested a review from rhettinger July 23, 2025 11:24
@serhiy-storchaka

serhiy-storchaka commented Jul 23, 2025

Copy link
Copy Markdown
Member

If we make the C implementation thread-safe, users will start writing code that depends on this and won't work with a pure Python implementation.

ZeroIntensity reacted with thumbs up emoji
All reactions
  • 👍 1 reaction

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@yoney

yoney commented Jul 23, 2025

Copy link
Copy Markdown
Contributor Author

I think with heapq the locks were necessary to make it not segfault under race conditions because it directly accessed list internals. I don't think that's the case here because we're using the abstract sequence API instead.

Exactly, locks were necessary for heapq because it directly accesses the internals of a list. Without the locks, it would segfault.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@yoney

yoney commented Jul 24, 2025

Copy link
Copy Markdown
Contributor Author

I think instead of adding locks for this, we should document the thread safety of this module like as in #136555

I'm closing this PR in favor of documenting it. I believe #136555 will be merged soon.

Thank you all for your reviews and feedback!

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@yoney yoney closed this Jul 24, 2025
@corona10 corona10 mentioned this pull request Aug 5, 2025
Segfault from calling StringIO methods in threads on free-threading debug build #135410
Closed
@yoney
yoney deleted the ft_bisect branch October 19, 2025 03:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@colesbury colesbury colesbury left review comments
@Yhg1s Yhg1s Awaiting requested review from Yhg1s
@mpage mpage Awaiting requested review from mpage
@rhettinger rhettinger Awaiting requested review from rhettinger rhettinger is a code owner

Uh oh!

There was an error while loading. Please reload this page.

Assignees

No one assigned

Labels

awaiting review topic-free-threading

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Uh oh!

There was an error while loading. Please reload this page.

6 participants

@yoney @colesbury @ZeroIntensity @rhettinger @kumaraditya303 @serhiy-storchaka
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
You are visitor 01402503 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.