Description

sum12

opened on Apr 14, 2022

Issue body actions

Summary

https://rustpython.github.io/pages/whats-left says _queue is missing. The module does not supply much apart from the native implementation. I would like to give it a try and replicate it here.

Detailed Explanation

Drawbacks, Rationale, and Alternatives

Not sure if it add any value, since the rustpython implementation seems good enough ?

Unresolved Questions

Reacted by Snowapril

Activity

1. sum12

   added

   RFCRequest for commentsRequest for comments

   on Apr 14, 2022
2. youknowone commented on Apr 14, 2022

   

   youknowone

   on Apr 14, 2022

   Member

   More actions

   Hi, welcome to the project. For now, it falls back to pure python implementation, which is also a part of official CPython library. If you add _queue, it will enhance the performance. That's the value.
   There is already a working version and we lack of many other features, so it is not the first-grade prioritized, but at the same time, no reason to not to do. If you are motivated by that, go fot it! It will be an easy start point to dig in the project.

   If you are looking for easy start point to learn how to start RustPython, I will also recommend this features: #3609
3. youknowone

   added

   A-stdlib

   on May 8, 2022
4. arihant2math

   added

   good first issueGood for newcomersGood for newcomers

   on May 12, 2025
5. lsahn-gh commented on May 15, 2026

   

   lsahn-gh

   on May 15, 2026

   Contributor

   More actions

   @youknowone,

   Can I take this if no one does ?
6. roryyorke commented on May 31, 2026

   

   roryyorke

   on May 31, 2026

   Contributor

   More actions

   SimpleQueue was added to CPython in python/cpython#3346 to address python/cpython#59181 .

   The problem it solves is that queue.Queue is not guaranteed to be reentrant, and, at time of the issue, it was not (see https://codewithoutrules.com/2017/08/16/concurrency-python/ , linked in the CPython issue).

   I've had a go at this, see https://github.com/roryyorke/RustPython/tree/rory/_queue , which passes the CSimpleQueueTest suite in test_queue.py; this suite includes a reentrancy test.

   I'd like to submit it for merge, but have some points for discussion first:

   • I suspect the reentrancy property must not only be tested for, but carefully reasoned about
   • clippy says I have unnecessary .clone()s. I added these during initial implementation because I thought they were needed for correct reference counting, but now I can't find the documentation that led me to think these calls were necessary
   • the get and wait timeouts are implemented as a polling loop, with a 100ms interval, so that signals (SIGINT, etc.) are serviced. Queue.get (etc. etc.) don't have this property, so maybe this complexity is not needed now
   • I initially use parking_lot::Condvar with rustpython_common::lock::PyMutex, but a recent-ish commit to main broke that, so I've used a parking_lot::Mutex instead. I don't know the wider implications of this.

   Finally, while trying to understand the reentrancy idea I ran the example code in https://codewithoutrules.com/2017/08/16/concurrency-python/ with recent (3.13.5) CPython, and to my great surprise it did not deadlock. git bisection revealed that python/cpython@70af994 changed threading.Condition.notify so that queue.Queue, both in CPython and RustPython, no longer deadlock with this particular example. queue._PySimpleQueue, which uses a threading.Semaphore, does still deadlock with this example (again, in both CPython and RustPython) --- queue._PySimpleQueue has a comment noting that it does not have the reentrant property.
7. youknowone commented on May 31, 2026

   

   youknowone

   on May 31, 2026

   Member

   More actions

   @roryyorke Thank you for the comment.
   @ShaharNaveh could you follow-up this as author of #7994 ?

   Reacted by Shahar Naveh
8. ShaharNaveh commented on May 31, 2026

   

   ShaharNaveh

   on May 31, 2026

   Contributor

   More actions

   @roryyorke is very similar to an earlier implementation of #7994 https://github.com/ShaharNaveh/RustPython/blob/5501877c85230034dff29a0598bbdc2e1b4d5754/crates/stdlib/src/_queue.rs

   That implementation has 1 major issue that it can be deadlocked by CPython test suite, specifically test_multiprocessing_fork.test_manager. I am now testing things with ReentrantMutex to hopefully fix that potential deadlock.

   Also, there are some small stuff like making weakref.ref(_queue.Empty) to pass, but that's a relatively easy fix
9. youknowone commented on Oct 6, 2026

   

   youknowone

   on Oct 6, 2026

   Member

   More actions

   #7994 is merged, thank you all!
10. youknowone

    closed this as completedon Oct 6, 2026

Issue actions