GeezerVison

Thanks for doing this! Do you have any benchmark results on how much the perf impact is? versus say using Py_BEGIN_CRITICAL_SECTION ?

Thanks for doing this! Do you have any benchmark results on how much the perf impact is? versus say using

I will try to prepare :)

@Fidget-Spinner @colesbury

I am not sure this will be a fair benchmark, but it shows different strengths in terms of technique and size of N.

Single thread

import pyperf

runner = pyperf.Runner()
runner.timeit(name="bench iter single",
              stmt="""
data = list(it)
""",
              setup = """
N = 100
it = iter(range(N))
"""
              )

• pyperf compare_to single_base.json single_cas.json
  Benchmark hidden because not significant (1): bench iter single

Mutl thread

import pyperf

runner = pyperf.Runner()
runner.timeit(name="bench iter multi",
              stmt="""
for _ in range(100):
    it = iter(range(N))
    with concurrent.futures.ThreadPoolExecutor() as executor:
        data = list(executor.map(lambda _: next(it), range(N)))
""",
              setup = """
import concurrent.futures
N = 100
"""
              )

• Mean +- std dev: [multi_baseline] 284 ms +- 11 ms -> [multi_cas] 277 ms +- 20 ms: 1.03x faster

Hmmm the perf difference is not that large. That might imply that we should just use critical section for simplicity, and hope that someone on PyPI publishes a "prange()" or something like that.

That might imply that we should just use critical section for simplicity, and hope that someone on PyPI publishes a "prange()" or something like that.

Yeah, I prefer this one.

See: #120540