Description

kumaraditya303

opened on Aug 22, 2026

Issue body actions

OpenSSL allows overriding its allocator with CRYPTO_set_mem_functions(), currently _ssl and _hashopenssl use the default system malloc. I am filing this issue to investigate if we can set it to PyMem_Raw* so that on free-threading it uses mimalloc which would scale better. Previously, there have been reports of high memory usage of ssl on glibc allocator, it would be worth checking if doing this fixes that as well.

Linked PRs

• gh-156228: Use PyMem_Raw* allocators for OpenSSL in _ssl and _hashlib #156229

Activity

1. kumaraditya303

   added

   topic-SSL

   performancePerformance or resource usagePerformance or resource usage

   on Aug 22, 2026
2. picnixz

   added

   extension-modulesC modules in the Modules dirC modules in the Modules dir

   type-featureA feature request or enhancementA feature request or enhancement

   on Aug 22, 2026
3. kumaraditya303 commented on Aug 23, 2026

   

   kumaraditya303

   on Aug 23, 2026

   ContributorAuthor

   More actions

   I analyzed the effect of #156229 on the reproducer from #109534 with Claude. GIL build.

   Two worktrees off the same commit, only difference is #156229. gcc -O3 -DNDEBUG --with-mimalloc, glibc 2.39, OpenSSL 3.0.13. Reproducer from #109534 at 2500 concurrent connections x 4 waves. RSS measured after all connections close and gc.collect(), then again after malloc_trim(0). Baseline ~24 MB, average of 2 runs.

   GIL build	RSS after gc	malloc_trim(0) reclaims	RSS after trim
   main, default	789 MB	759 MB	30 MB
   + #156229, default	800 MB	770 MB	30 MB
   main, PYTHONMALLOC=mimalloc	177 MB	133 MB	44 MB
   + #156229, PYTHONMALLOC=mimalloc	47 MB	0 MB	unchanged

   With the default allocator there is no effect, and there can't be: PYRAW_ALLOC is MALLOC_ALLOC and _PyMem_RawMalloc is a bare malloc(), so #156229 sends OpenSSL from malloc to a wrapper around malloc. Same arena, same retention. The difference in the table is noise.

   With PYTHONMALLOC=mimalloc it works as intended — OpenSSL's allocations leave the glibc arena. malloc_trim(0) reclaims 133 MB on main and nothing with #156229, which puts OpenSSL's share under this load at 133 MB. Post-gc RSS goes from ~177 MB down to ~47 MB.
4. kumaraditya303

   added a commit that references this issue on Sep 2, 2026

   gh-156228: Use PyMem_Raw allocators for OpenSSL in `_ssl` and `_hashl…

   39a9a47
5. kumaraditya303

   closed this as completedon Sep 2, 2026
6. clin1234

   added a commit that references this issue on Sep 12, 2026

   pythongh-156228: Use PyMem_Raw allocators for OpenSSL in `_ssl` and `…

   a93ed33

Issue actions