GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — Upgrade libuv to v1.48.0 by niklasr22 · Pull Request #600 · MagicStack/uvloop

Conversation

@niklasr22

@niklasr22 niklasr22 commented Mar 12, 2024 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Contributor

Upgrades libuv to v1.48.0 which fixes a security vulnerability.

I removed two DNS test cases because they raise an error intended by libuv.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
niklasr22 added 2 commits March 12, 2024 11:11
@niklasr22
Upgrade libuv version to 1.48.0
fc3beb5
@niklasr22
Remove test cases
281dc2c
@niklasr22

niklasr22 commented Mar 12, 2024

Copy link
Copy Markdown
Contributor Author

I think the pipeline should be able to pass as it did in the PR in my fork repo. Could someone please trigger a retry?

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@fantix

fantix commented Mar 13, 2024

Copy link
Copy Markdown
Member

yeah that error in CI looks like an old flake

niklasr22 reacted with thumbs up emoji
All reactions
  • 👍 1 reaction

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@tapple-cisco

tapple-cisco commented Jun 21, 2024 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown

I did some investigating about that venerability. I checked if I could reproduce the ‘truncate after 256 bytes’ venerability. I cannot exploit it if I pass hostname as a string, due to this idna encoding line; uvloop accidentally protects you from the libuv venerability:

>>> payload = f'0x{"0"*246}7f000001.example.com'
>>> payload
'0x0000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000007f000001.example.com'

>>> import uvloop, asyncio
>>> async def loop_getaddrinfo(addr, port): return await asyncio.get_running_loop().getaddrinfo(addr, port)
>>> uvloop.run(loop_getaddrinfo(payload, 80))
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/mfulmer/miniconda3/envs/lm2/lib/python3.9/site-packages/uvloop/__init__.py", line 82, in run
    return loop.run_until_complete(wrapper())
  File "uvloop/loop.pyx", line 1517, in uvloop.loop.Loop.run_until_complete
  File "/home/mfulmer/miniconda3/envs/lm2/lib/python3.9/site-packages/uvloop/__init__.py", line 61, in wrapper
    return await main
  File "<stdin>", line 1, in loop_get_addrinfo
  File "uvloop/loop.pyx", line 1528, in getaddrinfo
  File "uvloop/loop.pyx", line 905, in uvloop.loop.Loop._getaddrinfo
UnicodeError: encoding with 'idna' codec failed (UnicodeError: label too long)

This is a similar error that socket gives:

>>> import socket
>>> socket.getaddrinfo(payload, "80")
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/mfulmer/miniconda3/envs/lm2/lib/python3.9/socket.py", line 954, in getaddrinfo
    for res in _socket.getaddrinfo(host, port, family, type, proto, flags):
UnicodeError: encoding with 'idna' codec failed (UnicodeError: label too long)

However, if I pass the hostname as bytes, I can bypass the accidental uvloop protection and exploit libuv:

>>> payload = f'0x{"0"*246}7f000001.example.com'.encode()
>>> payload
b'0x0000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000007f000001.example.com'
>>> uvloop.run(loop_getaddrinfo(payload, 80))
[(<AddressFamily.AF_INET: 2>, <SocketKind.SOCK_STREAM: 1>, 6, '', ('127.0.0.1', 80)), (<AddressFamily.AF_INET: 2>, <SocketKind.SOCK_DGRAM: 2>, 17, '', ('127.0.0.1', 80)), (<AddressFamily.AF_INET: 2>, <SocketKind.SOCK_RAW: 3>, 0, '', ('127.0.0.1', 80))]

socket, however, isn’t fooled:

>>> socket.getaddrinfo(payload, "80")
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/mfulmer/miniconda3/envs/lm2/lib/python3.9/socket.py", line 954, in getaddrinfo
    for res in _socket.getaddrinfo(host, port, family, type, proto, flags):
socket.gaierror: [Errno -2] Name or service not known

I didn’t know about this "0x7f000001" form of hostnames, but, apparently it’s a thing:

>>> socket.getaddrinfo("0x7f000001", "80")
[(<AddressFamily.AF_INET: 2>, <SocketKind.SOCK_STREAM: 1>, 6, '', ('127.0.0.1', 80)), (<AddressFamily.AF_INET: 2>, <SocketKind.SOCK_DGRAM: 2>, 17, '', ('127.0.0.1', 80)), (<AddressFamily.AF_INET: 2>, <SocketKind.SOCK_RAW: 3>, 0, '', ('127.0.0.1', 80))]

I can’t find any documentation about why that’s considered a valid hostname. It’s obviously a hex encoding of a 4-byte ipv4 address, but, I’ve never seen it written that way

Anyway, maybe you can turn my investigation into a unit test for the security venerability

niklasr22 reacted with thumbs up emoji
All reactions
  • 👍 1 reaction

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@tapple-cisco

tapple-cisco commented Jun 24, 2024 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown

regarding the idna encoding error, there's some discussion of whether that error should be handled a different way in the python standard library or not. Just for reference: python/cpython#77139

niklasr22 reacted with thumbs up emoji
All reactions
  • 👍 1 reaction

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

This was referenced Aug 14, 2024
Bump libuv to 1.48 #616
Closed
vulnerability in libuv v1.46.0 #615
Closed
fantix
fantix reviewed Aug 14, 2024
View reviewed changes
Comment thread tests/test_dns.py Resolved

Uh oh!

There was an error while loading. Please reload this page.

fantix added 5 commits August 13, 2024 23:49
@fantix
Revert "Remove test cases"
7764b3f
This reverts commit 281dc2c.
@fantix
Merge remote-tracking branch 'origin/master' into upgrade-libuv
897fee7
@fantix
Fix for libuv 1.48
f3446e4
@fantix
Fix for macOS
bbe09cc
It seems getaddrinfo('', ...) on macOS is equivalent to nodename='localhost'.
This is inconsistent with libuv 1.48 which treats empty nodename as EINVAL.
@fantix
Add test
7768f08
Thanks to @tapple-cisco for the repro
@fantix
fantix requested review from 1st1 and elprans August 14, 2024 04:46
1st1
1st1 reviewed Aug 14, 2024
View reviewed changes
Comment thread uvloop/dns.pyx Resolved

Uh oh!

There was an error while loading. Please reload this page.

1st1
1st1 reviewed Aug 14, 2024
View reviewed changes
Comment thread uvloop/dns.pyx Resolved

Uh oh!

There was an error while loading. Please reload this page.

1st1
1st1 reviewed Aug 14, 2024
View reviewed changes
Comment thread tests/test_dns.py
a2 = self.loop.run_until_complete(
self.loop.getaddrinfo(*args, **kwargs))
except socket.gaierror as ex:
except (socket.gaierror, UnicodeError) as ex:

@1st1 1st1 Aug 14, 2024

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What input would trigger a UnicodeError?

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

@fantix fantix Aug 14, 2024

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is what @tapple-cisco mentioned with the vulnerability repro, as well as the BPO. A short example with CPython is like:

>>> payload = f'0x{"0"*246}7f000001.example.com'
>>> import socket; socket.getaddrinfo(payload, 0)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python3.12/socket.py", line 964, in getaddrinfo
    for res in _socket.getaddrinfo(host, port, family, type, proto, flags):
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/encodings/idna.py", line 173, in encode
    raise UnicodeError("label empty or too long")
UnicodeError: label empty or too long
encoding with 'idna' codec failed

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

@1st1 1st1 Aug 14, 2024

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is encoding with 'idna' codec failed a context exception?

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

@1st1 1st1 Aug 14, 2024

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i.e. .__context__.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

@fantix fantix Aug 14, 2024

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yeah it's a weird output, let me see

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

@fantix fantix Aug 15, 2024 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's __notes__, this particular note is added in 3.12 (changed from a __context__).

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

niklasr22 reacted with thumbs up emoji
All reactions
  • 👍 1 reaction
@fantix
crf: add comments and handle possible errors in error handling
c9ff4e4
1st1
1st1 approved these changes Aug 14, 2024
View reviewed changes
@fantix
fantix merged commit 7777852 into MagicStack:master Aug 15, 2024
@fantix fantix mentioned this pull request Aug 15, 2024
libuv 1.48.0: getaddrinfo('', 0): OSError: [Errno 22] Invalid argument #596
Closed
fantix added a commit that referenced this pull request Aug 15, 2024
@fantix
uvloop 0.20.0
2d35f10
Changes
=======

* Upgrade libuv to v1.48.0 (#600)
  (by @niklasr22 @fantix in 7777852 for #596 #615)

Fixes
=====

* Fix test_create_server_4 with Python 3.12.5 (#614)
  (by @shadchin in 62f9239)

* Use len(os.sched_getaffinity(0)) instead of os.cpu_count() (#591)
  (by @avkarenow in c8531c2 for #591)

* Inline _Py_RestoreSignals() from CPython (#604)
  (by @befeleme in 8511ba1 for #603)
@fantix fantix mentioned this pull request Aug 15, 2024
uvloop 0.20.0 #617
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@fantix fantix fantix left review comments
@1st1 1st1 1st1 approved these changes
@elprans elprans Awaiting requested review from elprans

Uh oh!

There was an error while loading. Please reload this page.

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Uh oh!

There was an error while loading. Please reload this page.

4 participants

@niklasr22 @fantix @tapple-cisco @1st1
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
You are visitor 01402503 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.