GeezerVison is showing github.com — gh-116738: Make _csv module thread-safe by aisk · Pull Request #118344 · python/cpython
Conversation
aisk
commented
Apr 27, 2024
•
edited
Loading
Uh oh!
There was an error while loading. Please reload this page.
This extension module has some module state values which may be read or written in different threads:
field_limit: A simple integer value to store the maximum field size for the parser;
dialects: A dict value to store all the dialects by string names.
field_limit
There is a public function field_size_limit that can read and write it, using a critical section to protect it. For other internal usages, _Py_atomic_load_long and _Py_atomic_store_long are used to make it thread safe.
dialects
There are three public functions get_dialect, unregister_dialect and list_dialects that can access this field. For the first two functions, a critical section is used to protect it.
And the list_dialects function just returns the dict's keys with PyDict_Keys. I think its result is readonly and PyDict_Keys itself is thread safe, so we don't need to do anything.
Other internal usages of dialects involve accessing its value with get_dialect_from_registry, which simply calls PyDict_GetItemRef and returns the result. I think we don't need to change it too.
Update: I think these APIs is just call dict APIs which is already thread-safe, and there are no other state changes inside these APIs, and no intermediate variables need to protect in these functions, so we don't need add locks on them.
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM, Sorry for the late.
@colesbury Do you have any comments about this?
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
aisk reacted with heart emoji
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Sorry for the long delay in reviewing this. Two requests:
- Let's change
field_limit to a Py_ssize_t. That seems more appropriate here and avoids the need for adding new atomic bindings. I'd prefer to avoid adding additional atomic bindings because its more code to maintain and its easy to add subtle bugs in the atomic bindings. Additionally, field_limit is compared to field_len, which is already a Py_ssize_t.
- Let's use relaxed atomics for reading and writing
field_limit, e.g., FT_ATOMIC_STORE_SSIZE_RELAXED. Relaxed ordering is fine for these sorts of global settings.
For the data type change:
%ld -> %zd
PyLong_AsLong -> PyLong_AsSsize_t
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
aisk reacted with heart emoji
Sorry for the long delay in reviewing this. Two requests:
* Let's change `field_limit` to a `Py_ssize_t`. That seems more appropriate here and avoids the need for adding new atomic bindings. I'd prefer to avoid adding additional atomic bindings because its more code to maintain and its easy to add subtle bugs in the atomic bindings. Additionally, `field_limit` is compared to `field_len`, which is already a `Py_ssize_t`.
* Let's use relaxed atomics for reading and writing `field_limit`, e.g., `FT_ATOMIC_STORE_SSIZE_RELAXED`. Relaxed ordering is fine for these sorts of global settings.
For the data type change:
* `%ld` -> `%zd`
* `PyLong_AsLong` -> `PyLong_AsSsize_t`
Thanks for the review, updated!
|
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
|
Thanks @aisk for the PR, and @kumaraditya303 for merging it 🌮🎉.. I'm working now to backport this PR to: 3.13.
🐍🍒⛏🤖
|
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
Reviewers
corona10
corona10 approved these changes
Uh oh!
There was an error while loading. Please reload this page.
Milestone
No milestone
Development
Successfully merging this pull request may close these issues.
Uh oh!
There was an error while loading. Please reload this page.
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
|