Description

kenballus

opened on Apr 3, 2023

Issue body actions

Sending a null byte in an HTTP request path or URL triggers an uncaught ValueError in CPython 3.10 and below.

Reproduction Instructions

Open two terminals and do the following (requires python <=3.10; current 3.10 branch included):

Terminal 1:

$ ./python -m http.server --bind 127.0.0.1
Serving HTTP on 127.0.0.1 port 8000 (http://127.0.0.1:8000/) ...

Terminal 2:

$ printf 'GET \x00 HTTP/1.1\r\n\r\n' | nc localhost 8000

Terminal 1:

----------------------------------------
Exception occurred during processing of request from ('127.0.0.1', 56900)
Traceback (most recent call last):
  File "/home/bkallus/clones/cpython/Lib/socketserver.py", line 683, in process_request_thread
    self.finish_request(request, client_address)
  File "/home/bkallus/clones/cpython/Lib/http/server.py", line 1304, in finish_request
    self.RequestHandlerClass(request, client_address, self,
  File "/home/bkallus/clones/cpython/Lib/http/server.py", line 668, in __init__
    super().__init__(*args, **kwargs)
  File "/home/bkallus/clones/cpython/Lib/socketserver.py", line 747, in __init__
    self.handle()
  File "/home/bkallus/clones/cpython/Lib/http/server.py", line 433, in handle
    self.handle_one_request()
  File "/home/bkallus/clones/cpython/Lib/http/server.py", line 421, in handle_one_request
    method()
  File "/home/bkallus/clones/cpython/Lib/http/server.py", line 672, in do_GET
    f = self.send_head()
  File "/home/bkallus/clones/cpython/Lib/http/server.py", line 727, in send_head
    f = open(path, 'rb')
ValueError: embedded null byte
----------------------------------------

This bug is fixed in python 3.11+, but I haven't yet figured out why. As far as I can tell, the relevant portions of Lib/http/server.py and Lib/socketserver.py remain unchanged. I think the issue boils down to catching only OSErrors, but not ValueErrors, even though open can raise ValueErrors. I would guess that this happened because the docs do not clearly state that open(path, 'rb') can raise a ValueError.

Environment:

Arch Linux, x86_64
Python 3.10.10+

Activity

1. kenballus

   added

   type-bugAn unexpected behavior, bug, or errorAn unexpected behavior, bug, or error

   on Apr 3, 2023
2. kenballus

   changed the title [-]Uncaught exception in `http.server` request handling (<=3.10 branch)[/-] [+]Uncaught exception in `http.server` request handling (<=3.10)[/+] on Apr 3, 2023
3. arhadthedev

   added

   stdlibStandard Library Python modules in the Lib/ directoryStandard Library Python modules in the Lib/ directory

   3.10 (EOL)end of lifeend of life

   on Apr 4, 2023
4. CCLDArjun commented on Apr 4, 2023

   

   CCLDArjun

   on Apr 4, 2023

   Contributor

   More actions

   did some debugging:

   looks like translate_path() in send_head() in 3.10 returns the path with a null character at the end, whereas in 3.11, it doesn't. Since in translate_path():

   cpython/Lib/http/server.py

   Line 848 in 9a8ce95

   path = posixpath.normpath(path)

   is called. the behavior of posixpath.normpath varies in 3.10 and 3.11:

   python3.11 -c "import pathlib; print(ord(pathlib.posixpath.normpath('\x00')))"
   46
   
   python3.10 -c "import pathlib; print(ord(pathlib.posixpath.normpath('\x00')))"
   0

   We error on this line in 3.10 because open() doesn't like null characters in the file path
   

   cpython/Lib/http/server.py

   Line 727 in 9a8ce95

   f = open(path, 'rb')

   So, should we catch the ValueError, like @kenballus suggested? Unsure whether either of the posixpath.normpath()s are broken, but it's suspicious that they're returning different values.
5. iritkatriel commented on Apr 5, 2023

   

   iritkatriel

   on Apr 5, 2023

   Member

   More actions

   3.10 is in security-fix only mode now, so this is out of date.

   Reacted by Ben Kallus
6. iritkatriel

   closed this as not plannedon Apr 5, 2023
7. kenballus commented on Apr 5, 2023

   

   kenballus

   on Apr 5, 2023

   ContributorAuthor

   More actions

   Oops! Was going based on the chart on python.org and didn't notice that 3.10.11 came out.

Issue actions