GeezerVison is showing github.com — copy.copy and copy.deepcopy scale poorly with free-threading · Issue #132657 · python/cpython
copy.copy and copy.deepcopy scale poorly with free-threading #132657DescriptionThe copy.copy and copy.deepcopy operations running on different objects in different threads should be able to run independently. The scaling is not good: output of the ftscalingbench.py with added benchmarks:
Running benchmarks with 8 threads
shallow_copy 4.6x slower
deepcopy 2.3x slower
object_cfunction 5.9x faster
cmodule_function 5.9x faster
object_lookup_special 5.5x faster
mult_constant 5.6x faster
generator 4.9x faster
pymethod 5.2x faster
pyfunction 4.5x faster
module_function 5.0x faster
load_string_const 5.6x faster
load_tuple_const 5.9x faster
create_pyobject 2.3x faster
create_closure 7.1x faster
create_dict 3.3x faster
thread_local_read 3.3x faster
There are at least two reasons:
- The
copy module uses module level variables in the copy.copy and copy.deepcopy methods.
- The
_copy_atomic_types (and some similar data structures) is a set which requires locking for membership testing.
Linked PRs
Activitychanged the title [-]copy.copy and copy.deepcopy scale with free-threading[/-] [+]copy.copy and copy.deepcopy scale poorly with free-threading[/+] on Apr 17, 2025eendebakpt commented on Apr 17, 2025 ContributorAuthor More actions I have no good approach yet to improve performance of the module level variables. This diff
diff --git a/Lib/copy.py b/Lib/copy.py
index c64fc076179..084b2873b65 100644
--- a/Lib/copy.py
+++ b/Lib/copy.py
@@ -67,7 +67,8 @@ def copy(x):
cls = type(x)
- if cls in _copy_atomic_types:
+ _local_atomic_types = {int, float, bool, complex, bytes, str, type, range, property}
+ if cls in _local_atomic_types or cls in _atomic_types:
return x
if cls in _copy_builtin_containers:
return cls.copy(x)
improves the scaling of copy.deepcopy a lot, but the solution is not very elegant and does not work for all test cases. colesbury commented on Apr 18, 2025
The copy module uses module level variables in the copy.copy and copy.deepcopy methods
We've discussed making module level variables use deferred reference counting, which would address this problem, but haven't decided exactly how we want to implement it. (All global variables? Only some frequently accessed ones?) eendebakpt commented on Apr 21, 2025 ContributorAuthor More actions Ok, I will leave this open until the deferred reference counting is in place. I tried making the relevant variables (e.g. copy._copy_atomic_types immortal, but that did not seem to improve performance). eendebakpt commented on Jun 26, 2025 ContributorAuthor More actions There is a bit more going on than just refcount contention. The code
s = frozenset({1, 2, 3})
z in s
scales poorly in the FT build.
- The reason is that the
z in s is specialized to _CONTAINS_OP_SET which calls _PySet_Contains (here) which has a lock
|
_PySet_Contains(PySetObject *so, PyObject *key) |
|
{ |
|
int rv; |
|
Py_BEGIN_CRITICAL_SECTION(so); |
|
rv = set_contains_lock_held(so, key); |
|
Py_END_CRITICAL_SECTION(); |
|
return rv; |
|
} |
- A second reason is there might be refcount contention in
set_lookkey
|
table = so->table; |
|
Py_INCREF(startkey); |
|
cmp = PyObject_RichCompareBool(startkey, key, Py_EQ); |
|
Py_DECREF(startkey); |
|
if (cmp < 0) |
|
return NULL; |
|
if (table != so->table || entry->key != startkey) |
|
return set_lookkey(so, key, hash); |
The increment is there because startkey is a borrowed reference and the rich compare might remove startkey from the set. For a frozenset this cannot happen, so we can skip the incref/decref. For a frozenset we can also skip the guard if (table != so->table || entry->key != startkey).
To address the issues we can either
- Make dedicated methods for frozenset (this requires some duplication of code and a new _CONTAINS_OP_FROZENSET bytecode)
- Add a quick check using
PyFrozenSet_CheckExact at the appropriate places and act accordingly.
@colesbury @markshannon Any opinion on which way to go? colesbury commented on Jun 26, 2025 I think adding a code path for PyFrozenSet_CheckExact that avoids the locking and incref/decref makes sense. nascheme commented on Dec 15, 2025 @colesbury Perhaps this can be closed now? Results below are for 8 threads after merging #132290 . The modified ftscaling script is linked from that PR. Maybe we want to revise the benchmark before closing this.
| FT scaling benchmark |
3.14.2t |
main |
| dict_contains |
7.6x faster |
7.9x faster |
| tuple_contains |
7.7x faster |
7.8x faster |
| list_contains |
7.6x faster |
7.6x faster |
| frozenset_contains |
7.5x faster |
7.8x faster |
| frozenset_contains_dunder |
7.7x faster |
7.8x faster |
| set_contains |
4.8x slower |
7.5x faster |
| set_contains_dunder |
3.4x slower |
7.7x faster |
| shallow_copy |
2.3x slower |
1.2x faster |
| deepcopy |
1.6x slower |
3.0x faster |
Reacted by Sergey Miryanov colesbury commented on Dec 15, 2025 @nascheme - that seems improved, but it looks like there are still scaling bottlenecks. I guess they're from reference count contention when loading the same global variable in multiple threads? eendebakpt commented on Dec 15, 2025 ContributorAuthor More actions @nascheme @colesbury There is still contention on the module level attributes. By addressing that the scaling can be improved:
Main:
shallow_copy_atomic_type 1.6x slower
shallow_copy_list 1.2x faster
deepcopy 2.6x faster
deepcopy_dataclass 3.3x faster
PR
shallow_copy_atomic_type 5.8x faster
shallow_copy_list 4.7x faster
deepcopy 4.4x faster
deepcopy_dataclass 3.0x faster
This improvement is by adding the following 5 lines to the copy module:
import testcapi
testcapi.pyobject_enable_deferred_refcount(_copy_atomic_types)
_testcapi.pyobject_enable_deferred_refcount(_copy_builtin_containers)
_testcapi.pyobject_enable_deferred_refcount(_atomic_types)
_testcapi.pyobject_enable_deferred_refcount(_deepcopy_dispatch)
I do not really like the solution, as we could end up with many of these statemements in the codebase. (I would be in favor of adding pyobject_enable_deferred_refcount to sys , but there is issue #134819 for that already)
We could also make all module level attributes use deferred reference counting. A branch to test this is
main...eendebakpt:cpython:module_attributes_deferred_ref_counting
(the implementation needs some work though) nascheme commented on Dec 16, 2025 Maybe a crazy idea but what about something like the following. The idea is when we specialize to LOAD_GLOBAL_MODULE we also enable deferred ref counting for the value.
--- a/Python/specialize.c
+++ b/Python/specialize.c
@@ -1305,6 +1305,13 @@ specialize_load_global_lock_held(
SPECIALIZATION_FAIL(LOAD_GLOBAL, SPEC_FAIL_OUT_OF_RANGE);
goto fail;
}
+#ifdef Py_GIL_DISABLED
+ PyObject *value;
+ if (PyDict_GetItemRef(globals, name, &value) == 1) {
+ PyUnstable_Object_EnableDeferredRefcount(value);
+ Py_DECREF(value);
+ }
+#endif
cache->index = (uint16_t)index;
cache->module_keys_version = (uint16_t)keys_version;
specialize(instr, LOAD_GLOBAL_MODULE);
With this change, the copy/deepcopy benchmarks scale quite a bit better:
shallow_copy 5.6x faster
deepcopy 6.2x faster
Better change, still quick and dirty:
#142843
It seems we only need to do it for frozensets, at least to help the copy scaling. Reacted by Pieter Eendebak Reacted by Sergey Miryanov colesbury commented on Dec 16, 2025 I think something like that makes sense. The downside is that it means that the value won't get freed immediately if someone overwrites the global variable -- it will only get freed during GC. We might want to limit it to globals that are accessed by another thread.
@nascheme - do you want to open a PR that we can use as a basis for discussion and investigation? encukou commented on Jan 19, 2026 Commit bb25f72 broke free-threading refleak tests:
$ ./python -m test -R2:3 test__interpchannels
[...]
0:00:00 load avg: 1.25 [1/1] test__interpchannels
beginning 5 repetitions. Showing number of leaks (. for 0 or less, X for 10 or more)
12:345
XX 222
test__interpchannels leaked [1, 1, 1] references, sum=3
test__interpchannels leaked [2, 2, 2] memory blocks, sum=6
0:00:07 load avg: 1.29 [1/1/1] test__interpchannels failed (reference leak)
nascheme commented on Jan 19, 2026 eendebakpt commented on Jan 20, 2026 ContributorAuthor More actions On current main I have these results (8 threads) which look good:
shallow_copy 5.1x faster
deepcopy 4.3x faster
deepcopy_dataclass 3.5x faster
For the dataclass performance there is a separate issue open: #139103.
If no objections, we can close the issue as completed.
MetadataMetadataDevelopmentNo branches or pull requestsIssue actions
|