GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — [3.14] gh-119452: Read/write CGI data using worker threads by encukou · Pull Request #142181 · python/cpython

Conversation

@encukou

@encukou encukou commented Dec 2, 2025 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Member

This reads/writes data as available, making the CGI application responsible for managing any timeouts when receiving data from clients.

Data is read in chunks of bounded size, and passed on immediately (except stderr, which is combined into a single message as before).

This does need 3 threads. (As does process.communicate.)

  • Issue: OOM a potential denial of service in the CGI server on Windows #119452

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
All reactions
@encukou
[3.14] pythongh-142176: Read/write CGI data using worker threads
a773018 
This reads/writes data as available, making the CGI application
responsible for managing any timeouts when receiving data from
clients.

Data is read in chunks of bounded size, and passed on immediately
(except stderr, which is combined into a single message as before).
This does need 3 threads. (As does process.communicate.)
@bedevere-app bedevere-app Bot added the awaiting core review label Dec 2, 2025
@encukou encukou added the skip news label Dec 2, 2025
@encukou encukou changed the title [3.14] gh-142176: Read/write CGI data using worker threads [3.14] gh-119452: Read/write CGI data using worker threads Dec 2, 2025
@bedevere-app bedevere-app Bot mentioned this pull request Dec 2, 2025
OOM a potential denial of service in the CGI server on Windows #119452
Closed
@encukou
Always close stdin
5216564
@encukou
encukou requested a review from serhiy-storchaka December 2, 2025 11:09
This was referenced Dec 2, 2025
[3.14] gh-119452: Block until data is read #142176
Closed
[3.14] Revert "gh-119452: Fix a potential virtual memory allocation denial of service in http.server (GH-119455)" #142184
Merged
[3.13] Revert "gh-119452: Fix a potential virtual memory allocation denial of service in http.server (GH-119455) (GH-142130)" #142185
Merged
encukou added 2 commits December 2, 2025 14:02
@encukou
Merge docs fix from 3.14
9f24d8b
@encukou
Merge the 3.14 branch, keeping code from this one
cb6fc3e
@encukou

encukou commented Dec 2, 2025

Copy link
Copy Markdown
Member Author

@serhiy-storchaka See here.
Do you think it's better than loading all request data to memory, then sending it to the process all at once?
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
@serhiy-storchaka

serhiy-storchaka commented Dec 2, 2025

Copy link
Copy Markdown
Member

This approach has great advantages, I like it, but creating a thread is not cheap. This is also much bigger change, so we should evaluate all the pros and cons.

Can the same select() loop be used for input and output? Can it be ran in the original thread?
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
@encukou

encukou commented Dec 2, 2025

Copy link
Copy Markdown
Member Author

creating a thread is not cheap

Sure, but can you make this cheaper? Note that Popen.communicate also creates 3 threads (generally: one for each pipe, except if there's only one pipe).

Can the same select() loop be used for input and output?

On Windows, select only works on sockets.
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
gpshead
gpshead reviewed Dec 2, 2025
View reviewed changes
Comment thread Lib/http/server.py
# already closed?
pass
if self.command.lower() == "post" and nbytes > 0:
data = self.rfile.read(nbytes)

@gpshead gpshead Dec 2, 2025

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The issue was only around the memory allocation of an untrusted nbytes value here. introducing a thread seems unnecessary. There wasn't a resource exhaustion problem with this blocking for the issue at hand.

The memory-address-space-DoS consumption point is to only allocate an amount close in magnitude to the total data that actually arrives rather than the untrusted number up front. A simple loop reading in chunk increments as the earlier PR #119455 did, but without a select at all would handle the issue fine. (ie: just remove the select from the earlier pr entirely)

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
All reactions
@encukou encukou closed this Dec 4, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@gpshead gpshead gpshead left review comments
@serhiy-storchaka serhiy-storchaka Awaiting requested review from serhiy-storchaka

Uh oh!

There was an error while loading. Please reload this page.

Assignees

No one assigned

Labels

awaiting core review skip news

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Uh oh!

There was an error while loading. Please reload this page.

3 participants

@encukou @serhiy-storchaka @gpshead
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
You are visitor 01402503 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.