GeezerVison is showing github.com — `Path.from_uri()` doesn't work if the URI contains host component · Issue #123599 · python/cpython
Path.from_uri() doesn't work if the URI contains host component #123599
DescriptionBug report
Bug description:
Path.from_uri() introduced in Python 3.13 doesn't work properly if the URI contains a host component other than localhost. Following examples are run with Python 3.13 rc 1 on Linux with a machine having host name kone:
>>> print(Path().from_uri('file:///home/peke/test'))
/home/peke/test
>>> print(Path().from_uri('file://localhost/home/peke/test'))
/home/peke/test
>>> print(Path().from_uri(f'file://{socket.getfqdn()}/home/peke/test'))
//kone/home/peke/testAccording to RFC 8089 including the host component as a fully qualified name is fine so this looks like a bug to me.
CPython versions tested on:
3.13
Operating systems tested on:
Linux
Linked PRs
Activitypekkaklarck commented on Sep 2, 2024 Accepting a host component, other than localhost, raises a question about validity of the used host name. For example, browsers seem to totally ignore the host component and accept paths like file://whatever/home/peke/test in my case. I believe Python should be more strict, though, and raise a ValueError is the host component doesn't match the system where the code is run. Although the RFC mandates the host component to be fully qualified, I believe accepting only the host name should be fine too. If someone wants to parse file URIs with different host names, they can use urilib.parse.urlparse instead.
It might be that UNC Windows file paths even further. I tested this on Windows and there this usage makes sense:
>>> p = Path(r'\\host\path')
>>> print(p.as_uri())
file://host/path/
>>> p == Path.from_uri(p.as_uri())
True Perhaps from_uri behavior should depend on the operating system. barneygale commented on Sep 2, 2024 All URIs with non-empty, non-localhost authorities parse as Path objects that start with a double slash, so it should be straightforward to reject these paths:
path = Path.from_uri('file://server/share/foo.txt')
if path.as_posix().startswith('//'):
raise ValueError('Non-local file URI')We could add a local_authorities argument so that users can override the ['', 'localhost'] defaults pekkaklarck commented on Sep 2, 2024 Explicitly rejecting paths would certainly be better than returning invalid paths. Including socket.getfqdn() and possibly also socket.gethostname() in the list of local authorities could be more convenient than requiring users to pass them explicitly, though. If the list is made configurable, it probably should contain localhost and the empty string by default. pekkaklarck commented on Sep 2, 2024 This probably would anyway require special handling on Windows. On POSIX something like Path.from_uri('file://host/path') should yield Path('/path'), assuming that host is detected to be a local authority, but on Windows throwing the host part away would break UNC paths. barneygale commented on Sep 2, 2024 We should try to be consistent across OSs unless we really must diverge IMO.
Explicitly rejecting paths would certainly be better than returning invalid paths.
Paths starting with two slashes are valid on both Windows and POSIX. On Windows they're UNC paths, whereas on POSIX they're implementation-defined (ref). pekkaklarck commented on Sep 2, 2024 It can be hard to be totally consistent across OSes. On Windows Path(r'\\host\path').as_uri() yields file://host/path/, and it makes sense that round-trip works and Path.from_uri('file://host/path/') yields Path(r'\\host\path'). In other words, the host component is preserved on Windows. On the other hand, when Path.from_uri('file://host/path/') is used on POSIX, the host component can be validated but there's, AFAIK, no way to preserve it and the return value can only be Path('/path').
There's already other functionality in pathlib that's operating system dependent and I don't see why from_uri couldn't be as well. Someone needing, for example, Windows semantics on POSIX could then explicitly use PureWindowsPath.
I should have used "incorrect" instead of "invalid" in my earlier comment. Although Path('//hello/world') is valid, it certainly isn't the correct return value Path.from_uri('file://hello/world') on POSIX. barneygale commented on Oct 21, 2024 Upon further consideration, I've adjusted my PR so that:
- If the authority is empty or 'localhost', we suppress it.
- Otherwise, if we're on Windows, we return a UNC path
- Otherwise, if the authority resolves to a loopback address, we suppress it.
- Otherwise we raise
ValueError
This is basically the same logic as urllib.request.FileHandler.file_open(), and so in a future patch I'm planning to slim down the urllib code. serhiy-storchaka commented on Apr 14, 2025 socket.gethostbyname(socket.gethostname()) raises gaierror(8, 'Name does not resolve').
>>> socket.gethostbyname(socket.gethostname())
Traceback (most recent call last):
File "<python-input-1>", line 1, in <module>
socket.gethostbyname(socket.gethostname())
~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^
socket.gaierror: [Errno 8] Name does not resolveadded 2 commits that reference this issue on Apr 14, 2025GH-123599: Match `file:` URL hostname against machine hostname in url… barneygale commented on Apr 15, 2025 The above test failure was fixed in #132523 Reacted by Serhiy Storchaka barneygale commented on Apr 16, 2025 Re-opening to discuss this point from Serhiy on #126844:
url2pathname() now performs network requests (and hang for a time).
Having given it more thought, I reckon we should call gethostbyname() only when a new resolve_netloc keyword-only argument is set to true (default false). Existing users of this function probably don't expect it to perform network access, and it could have larger consequences in environments where DNS resolution is borked.
Thoughts / 👍 / 👎? serhiy-storchaka commented on Apr 16, 2025 You just read my mind. I was thinking about suggesting adding a keyword argument for resolving hostname, but then I thought that url2pathname() may be not much used in third-party code, it may be not worth a hassle.
But adding an option is safer. In future we can even change its default value, but the users will be able to request the old behavior. zooba commented on Apr 16, 2025 Also FYI, I'm seeing idna encoding errors on one of my private test machines from the address = socket.gethostbyname(authority) line. I'm not entirely sure what path is being passed in to cause it, but I suspect we ought to swallow UnicodeEncodeError in _is_local_authority anyway? added 2 commits that reference this issue on May 5, 2025GH-123599: `url2pathname()`: don't call `gethostbyname()` by default (#…
MetadataMetadataDevelopmentNo branches or pull requestsIssue actions
|