GeezerVison is showing github.com — SSLContext.load_verify_locations leaks memory on Linux · Issue #84904 · python/cpython
SSLContext.load_verify_locations leaks memory on Linux #84904Description
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
assignee = None
closed_at = None
created_at = <Date 2020-05-22.09:29:10.619>
labels = ['expert-SSL', '3.8', '3.9', 'performance', '3.7', 'library', 'expert-asyncio']
title = 'SSLContext.load_verify_locations leaks memory on Linux in async code'
updated_at = <Date 2020-05-22.10:32:24.242>
user = 'https://github.com/Recursing' bugs.python.org fields:
activity = <Date 2020-05-22.10:32:24.242>
actor = 'Recursing'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = ['Library (Lib)', 'asyncio', 'SSL']
creation = <Date 2020-05-22.09:29:10.619>
creator = 'Recursing'
dependencies = []
files = []
hgrepos = []
issue_num = 40727
keywords = []
message_count = 6.0
messages = ['369573', '369578', '369582', '369583', '369584', '369586']
nosy_count = 4.0
nosy_names = ['christian.heimes', 'asvetlov', 'yselivanov', 'Recursing']
pr_nums = []
priority = 'normal'
resolution = None
stage = None
status = 'open'
superseder = None
type = 'resource usage'
url = 'https://bugs.python.org/issue40727'
versions = ['Python 3.6', 'Python 3.7', 'Python 3.8', 'Python 3.9'] Reacted by Zachary Levonian and Simon Gurcke ActivityRecursing commented on May 22, 2020 MannequinAuthor More actions Minimal code to reproduce:
import ssl
import certifi
import gc
import asyncio
ca_path = certifi.where()
async def make_async_context() -> None:
context = ssl.SSLContext(ssl.PROTOCOL_TLS)
context.load_verify_locations(ca_path)
await asyncio.sleep(1)
async def main(n: int) -> None:
await asyncio.wait([make_async_context() for _ in range(n)])
gc.collect()
asyncio.run(main(2000))
input("Finished run, still using lots of memory :(")
gc.collect()
input("gc.collect() does not help :(")
Running this code on several linux machines (with python from 3.6.9 to 3.9.0a5, and openSSL from 1.1.1 11 Sep 2018 to 1.1.1g 21 Apr 2020) causes a significant memory leak, while on windows memory usage peaks around 1 GB but gets freed tiran commented on May 22, 2020 Does it also leak without asyncio? Recursing commented on May 22, 2020 MannequinAuthor More actions Removing the await asyncio.sleep(1) removes the leak, while changing it to await asyncio.sleep(0) seems to keep it sw55555 commented on Nov 17, 2022 Leaks happen on Python3.11 too, but only on linux(compiled with openssl 1.1.1q, same as python on Windows), after running test, memory usage:
Ubuntu 22.04.1 LTS :
Line # Mem usage Increment Occurrences Line Contents
=============================================================
8 21.6 MiB 21.6 MiB 1 @profile
9 def leak_test():
10 21.6 MiB 0.0 MiB 1 ca_certs = certifi.where()
11 26.7 MiB 5.2 MiB 303 contexts = [SSLContext(PROTOCOL_TLS_CLIENT) for _ in range(300)]
12 238.5 MiB 0.0 MiB 301 for context in contexts:
13 238.5 MiB 211.7 MiB 300 context.load_verify_locations(ca_certs)
14
15 238.5 MiB 0.0 MiB 1 del contexts
16 238.5 MiB 0.0 MiB 1 gc.collect()
17
18 238.5 MiB 0.0 MiB 1 print('Test complete!')Windows 10 :
Line # Mem usage Increment Occurrences Line Contents
=============================================================
8 22.7 MiB 22.7 MiB 1 @profile
9 def leak_test():
10 22.9 MiB 0.3 MiB 1 ca_certs = certifi.where()
11 24.6 MiB 1.6 MiB 303 contexts = [SSLContext(PROTOCOL_TLS_CLIENT) for _ in range(300)]
12 251.8 MiB 0.0 MiB 301 for context in contexts:
13 251.8 MiB 227.2 MiB 300 context.load_verify_locations(ca_certs)
14
15 35.5 MiB -216.3 MiB 1 del contexts
16 35.5 MiB 0.0 MiB 1 gc.collect()
17
18 35.5 MiB 0.0 MiB 1 print('Test complete!')My code to reproduce this bug:
import gc
import certifi
from ssl import SSLContext, PROTOCOL_TLS_CLIENT
from memory_profiler import profile
@profile
def leak_test():
ca_certs = certifi.where()
contexts = [SSLContext(PROTOCOL_TLS_CLIENT) for _ in range(300)]
for context in contexts:
context.load_verify_locations(ca_certs)
del contexts
gc.collect()
print('Test complete!')
if __name__ == "__main__":
leak_test()sw55555 commented on Nov 17, 2022 Test on Python 3.10.6(Ubuntu 22.04.1 LTS), it leaks too changed the title [-]SSLContext.load_verify_locations leaks memory on Linux in async code[/-] [+]SSLContext.load_verify_locations leaks memory on Linux[/+] on Nov 17, 2022gvanrossum commented on Nov 17, 2022 On macOS (Ventura 13.1) I also see most memory being returned, like on Windows 10 above. So it does seem to be Linux specific. Since the latest repro doesn't involve async code I removed that from the subject and removed the expert-asyncio label.
@mhils may well be right that this is some other leak together with a weird allocation pattern. vminfant commented on Jun 19, 2023 Python 3.10.12 and 3.11.2 on (Debian GNU/Linux 12 (bookworm), has memory leak as well. Is there any progress or any recommendation to avoid this issue? vEpiphyte commented on Feb 20, 2024 This is still reproducing in 3.12.2 on ubuntu 22.04, compiled against openssl 3.0.2 gvanrossum commented on Feb 20, 2024 @vEpiphyte
This is still reproducing in 3.12.2 on ubuntu 22.04, compiled against openssl 3.0.2
What exactly did you try? vEpiphyte commented on Feb 21, 2024 @gvanrossum Here is the code I used. It is a variant of the original - I am holding an reference to the SSLContext objects in a dictionary to avoid any nebulous object lifetime issues in the original make_async_context; which is then cleared(). The resident memory usage was observed with external tools.
With the malloc_trim(0) call from @mhils the resident memory decreases when the cache is cleared. Without it, the memory stays allocated after clearing the dictionary. This was consistent with and without del statements on the SSLContext objects.
import gc
import sys
import ssl
import time
import ctypes
import certifi
N = 1000
ca_path = certifi.where()
context_cache = {}
def clear_cache(cache):
cache.clear()
print('Calling maclloc_trim(0)')
# Per https://github.com/python/cpython/issues/84904#issuecomment-1123760269 this helps...
# ctypes.CDLL('libc.so.6').malloc_trim(0)
def make_context(i) -> None:
context = ssl.SSLContext(ssl.PROTOCOL_TLS)
context.load_verify_locations(ca_path)
context_cache[i] = context
return None
def main(n: int) -> None:
for i in range(n):
if i % 100 == 0:
print(f'{i=}')
make_context(i)
print('done gather, sleeping 6')
time.sleep(6)
print('clearing cache')
clear_cache(context_cache)
print('done clearing, sleeping 6')
time.sleep(6)
gc.collect()
main(N)
input("Finished run, still using lots of memory :(")
gc.collect()
input("Does gc.collect() help?")
time.sleep(6)
sys.exit(0)
gvanrossum commented on Feb 21, 2024 But if malloc_trim() releases the memory, the problem you are experiencing is not leakage but fragmentation. vEpiphyte commented on Feb 22, 2024 I was just wanting to report that the s/leak/fragmentation/ was still observable with Linux on 3.12.2. I do not know enough about python memory management to know if that behavior is a linux specific artifact or not; but it does feel a bit odd to only be observed on that platform.
MetadataMetadataStatusShow more project fields
DevelopmentNo branches or pull requestsIssue actions
|