GeezerVison is showing github.com — Dramatic slowdown with random.random on free-threading build. · Issue #118392 · python/cpython
Dramatic slowdown with random.random on free-threading build. #118392Description
This slowdown was found with one of my favorite benchmarks, which is calculating the pi value with the Monte Carlo method.
import os
import random
import time
from threading import Thread
def monte_carlo_pi_part(n: int, idx: int, results: list[int]) -> None:
count = 0
for i in range(n):
x = random.random()
y = random.random()
if x*x + y*y <= 1:
count += 1
results[idx] = count
n = 10000
threads = []
num_threads = 100
results = [0] * num_threads
a = time.time()
for i in range(num_threads):
t = Thread(target=monte_carlo_pi_part, args=(n, i, results))
t.start()
threads.append(t)
while threads:
t = threads.pop()
t.join()
b = time.time()
print(sum(results) / (n * num_threads) * 4)
print(b-a)Acquiring critical sections for random methods causes this slowdown.
Removing @critical_section from the method, which uses genrand_uint32 and then updating genrand_uint32 to use atomic operation makes the performance acceptable.
| Build |
Elapsed |
PI |
| Default (with specialization) |
0.16528010368347168 |
3.144508 |
| Free-threading (with no specialization) |
0.548654317855835 |
3.1421 |
| Free-threading with my patch (with no specialization) |
0.2606849670410156 |
3.141108 |
Linked PRs
Activitycorona10 commented on Apr 29, 2024 colesbury commented on Apr 29, 2024 It'd be great to have the default random number generator be safe and efficient when called from multiple threads, but I think that's a bigger change that requires more discussion and maybe a PEP.
In the meantime, I think the advice should be to use individual Random objects per-thread. In other words, the program should be written as:
def monte_carlo_pi_part(n: int, idx: int, results: list[int]) -> None:
r = random.Random()
count = 0
for i in range(n):
x = r.random()
y = r.random()
if x*x + y*y <= 1:
count += 1
results[idx] = countI think that you also would need #118112 (or deferred reference counting) for the program to be efficient, or otherwise the reference count contention is going to be a bottleneck. Reacted by Donghee Na, Joongi Kim, Erlend E. Aasland and Juhan Oskar Hennoste corona10 commented on Apr 29, 2024
In the meantime, I think the advice should be to use individual Random objects per-thread. In other words, the program should be written as:
Yeah, it works. so it should be included somewhere in the migration guide. corona10 commented on Apr 29, 2024 colesbury commented on Apr 29, 2024 We could also consider adopting a strategy like Go does, at least for the free-threaded build:
- If
seed() has not been called, use a fast per-thread RNG for the global calls
- Once
seed() is called, switch to the shared state RNG to preserve old behavior.
Reacted by Donghee Na and Erlend E. Aasland
MetadataMetadataDevelopmentNo branches or pull requestsIssue actions
|
