GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — Optimize non buffered protocol reads in SSLProtocol by tarasko · Pull Request #726 · MagicStack/uvloop

Conversation

@tarasko

@tarasko tarasko commented Jan 31, 2026 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Contributor

Calling SSLObject.read(SSL_READ_MAX_SIZE) has a big performance issue. Internally it first allocates bytes object of SSL_READ_MAX_SIZE bytes, then reads into it, then shrinks it down to the actual number of bytes read.
Given that SSL_READ_MAX_SIZE = 256 * 1024,
we allocate 256K every time we call SSLObject.read from SSLProtocol._do_read__copied.
I've fixed it by allocating our own buffer and passing it to SSLObject.read.

I have attached perf output before and after this change.

Other changes:

  • fix cython warning: uvloop/handles/pipe.pyx:159:4: Overriding a c(p)def method with a def method
  • Use bytearray instead of raw memory allocations for the buffer that is provided to UVStream for simplicity
  • Reduce default size of read buffer from 256K to 64K. 64K is actually a default suggested size from libuv. Read buffer can still grow up to 256K if libuv wants more memory.
  • Cache _app_protocol.data_received for performance
  • Cache SSL_READ_MAX_SIZE as a python object, so that cython don't have to create a new object every time we call SSLObject.read.
  • It is ok to call SSLObject.read and pass a buffer that is smaller than the len argument. SSLObject.read will correctly read up to the buffer size, not up to len argument

Before:

30.79%     0.14%            14  python          _ssl.cpython-313-x86_64-linux-gnu.so                  [.] _ssl__SSLSocket_read
        |          
         --30.65%--_ssl__SSLSocket_read
                   |          
                   |--16.98%--PyBytes_FromStringAndSize
                   |          |          
                   |           --13.91%--PyObject_Malloc
                   |                     |          
                   |                      --13.87%--malloc
                   |                                |          
                   |                                 --13.79%--_int_malloc
                   |                                           |          
                   |                                            --13.48%--sysmalloc_mmap.isra.0
                   |                                                      |          
                   |                                                       --7.11%--__mmap
                   |          
                   |--8.66%--_PyBytes_Resize
                   |          |          
                   |           --8.50%--realloc
                   |                     |          
                   |                      --8.43%--__GI___mremap
                   |          
                    --4.37%--SSL_read_ex
                              |          
                               --4.19%--ssl3_read_internal
                                         |          
                                          --4.18%--ssl3_read_bytes
                                                    |          
                                                     --3.24%--ssl3_get_record
                                                               |          
                                                                --2.24%--tls1_enc
                                                                          |          
                                                                          |--1.12%--EVP_DecryptUpdate
                                                                          |          ossl_gcm_stream_update
                                                                          |          |          
                                                                          |           --1.02%--gcm_cipher_internal
                                                                          |                     |          
                                                                          |                      --0.81%--ossl_gcm_one_shot
                                                                          |                                |          
                                                                          |                                 --0.67%--generic_aes_gcm_cipher_update
                                                                          |          
                                                                           --0.78%--EVP_CIPHER_CTX_ctrl

After:

 8.47%     0.21%            19  python          _ssl.cpython-313-x86_64-linux-gnu.so                  [.] _ssl__SSLSocket_read
        |          
         --8.25%--_ssl__SSLSocket_read
                   |          
                   |--6.93%--SSL_read_ex
                   |          |          
                   |           --6.77%--ssl3_read_internal
                   |                     |          
                   |                      --6.68%--ssl3_read_bytes
                   |                                |          
                   |                                |--5.28%--ssl3_get_record
                   |                                |          |          
                   |                                |           --3.81%--tls1_enc
                   |                                |                     |          
                   |                                |                     |--1.92%--EVP_DecryptUpdate
                   |                                |                     |          ossl_gcm_stream_update
                   |                                |                     |          |          
                   |                                |                     |           --1.91%--gcm_cipher_internal
                   |                                |                     |                     |          
                   |                                |                     |                      --1.56%--ossl_gcm_one_shot
                   |                                |                     |                                |          
                   |                                |                     |                                 --1.26%--generic_aes_gcm_cipher_update
                   |                                |                     |                                           |          
                   |                                |                     |                                            --0.50%--CRYPTO_gcm128_decrypt_ctr32
                   |                                |                     |          
                   |                                |                      --1.24%--EVP_CIPHER_CTX_ctrl
                   |                                |          
                   |                                 --0.59%--ssl3_setup_read_buffer
                   |          
                    --0.69%--PyArg_ParseTuple
                              |          
                               --0.60%--vgetargs1_impl

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
taras added 2 commits January 31, 2026 05:21
taras
WIP
33d0ea4
taras
Cleanup
5385ebe
@tarasko tarasko changed the title Optimize non buffered reads in SSLProtocol Optimize non buffered protocol reads in SSLProtocol Jan 31, 2026
taras added 7 commits February 1, 2026 00:36
taras
Fix memory leak in SSLProtocol and test_create_connection_memory_leak…
58ce0be
… test
taras
Add some comments
d33c852
taras
Optimize and cleanup _do_read_into_void
649f8de
taras
Fix bytearray allocation
1345960
taras
Format code
3e1a8a8
taras
Cleanup for PR
64ff06e
taras
Cleanup cython imports
cde915d
@tarasko

tarasko commented Feb 1, 2026 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Contributor Author

Hi @fantix, would you mind taking a look at this PR as well? It significantly improves SSL read performance in case of non-buffered user protocol

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

taras and others added 4 commits February 8, 2026 21:44
taras
Better name for variable
6f91a52
@tarasko
Merge branch 'master' into optimize_ssl_nonbuffered_read
7e66d06
@fantix
Merge branch 'master' into optimize_ssl_nonbuffered_read
b771722
@tarasko
Merge branch 'master' into optimize_ssl_nonbuffered_read
d3e0314
@parikls

parikls commented Sep 25, 2026

Copy link
Copy Markdown

Thanks @tarasko. Turns out this also fixes a memory-leak problem we've hit in a production. Here are our numbers:

A FastAPI/uvicorn service of ours grew ~110 MiB RSS per pod per day and never leveled off. It talks to Postgres through asyncpg over TLS. asyncpg isn't a BufferedProtocol, so every DB read goes through _do_read__copied. There, each SSLObject.read(256 KiB) allocates a 256 KiB bytes object and then shrinks it. By then, blocks freed at startup have raised glibc dynamic mmap threshold above 256 KiB. So these buffers are carved out of the main heap, which fragments and never goes back to the OS.

We replayed our production traffic, 150k requests per run, with master and this PR side by side:

┌─────────────────┬─────────────────────────────────┬────────────┐
│     uvloop      │ 256 KiB allocations per request │ RSS growth │
├─────────────────┼─────────────────────────────────┼────────────┤
│ 0.22.1          │ 37.6                            │ +36 MiB    │
├─────────────────┼─────────────────────────────────┼────────────┤
│ master 0582f94  │ 19.3                            │ +33 MiB    │
├─────────────────┼─────────────────────────────────┼────────────┤
│ this PR d3e0314 │ 0                               │ +4 MiB     │
└─────────────────┴─────────────────────────────────┴────────────┘

With this PR, memory flattens after warm-up, the same as when Postgres runs over plain TCP. There were no errors and CPU per request didn't change.

Setup: Python 3.13.15, glibc 2.41, Linux, uvicorn 0.42 + httptools 0.8, asyncpg 0.31, TLS 1.3.

MALLOC_MMAP_THRESHOLD_=131072 works around it.

Would be great to see this merged and released.

CC @fantix @elprans @1st1

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@tarasko

tarasko commented Sep 26, 2026 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Contributor Author

Thanks @tarasko. Turns out this also fixes a memory-leak problem we've hit in a production. Here are our numbers:

A FastAPI/uvicorn service of ours grew ~110 MiB RSS per pod per day and never leveled off. It talks to Postgres through asyncpg over TLS. asyncpg isn't a BufferedProtocol, so every DB read goes through _do_read__copied. There, each SSLObject.read(256 KiB) allocates a 256 KiB bytes object and then shrinks it. By then, blocks freed at startup have raised glibc dynamic mmap threshold above 256 KiB. So these buffers are carved out of the main heap, which fragments and never goes back to the OS.

We replayed our production traffic, 150k requests per run, with master and this PR side by side:

┌─────────────────┬─────────────────────────────────┬────────────┐
│     uvloop      │ 256 KiB allocations per request │ RSS growth │
├─────────────────┼─────────────────────────────────┼────────────┤
│ 0.22.1          │ 37.6                            │ +36 MiB    │
├─────────────────┼─────────────────────────────────┼────────────┤
│ master 0582f94  │ 19.3                            │ +33 MiB    │
├─────────────────┼─────────────────────────────────┼────────────┤
│ this PR d3e0314 │ 0                               │ +4 MiB     │
└─────────────────┴─────────────────────────────────┴────────────┘

With this PR, memory flattens after warm-up, the same as when Postgres runs over plain TCP. There were no errors and CPU per request didn't change.

Setup: Python 3.13.15, glibc 2.41, Linux, uvicorn 0.42 + httptools 0.8, asyncpg 0.31, TLS 1.3.

MALLOC_MMAP_THRESHOLD_=131072 works around it.

Would be great to see this merged and released.

CC @fantix @elprans @1st1

@parikls
Merging this PR and releasing uvloop, might take some time.

But perhaps you would be interested to try out aiofastnet.

aiofastnet still works on top of uvloop, but it replaces transport layer with its own implementation which is even more efficient than this PR.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Uh oh!

There was an error while loading. Please reload this page.

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Uh oh!

There was an error while loading. Please reload this page.

3 participants

@tarasko @parikls @fantix
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
You are visitor 01402503 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.