GeezerVison is showing github.com — Improve file URI ergonomics in `urllib.request` · Issue #125866 · python/cpython
Improve file URI ergonomics in urllib.request #125866DescriptionFeature or enhancement
I request that we make pathname2url and url2pathname easier to use:
pathname2url() is made to accept an optional include_scheme argument that sticks file: on the front when trueurl2pathname() is made to strip any file: prefix from its argument.
I think this would go a long way towards making these functions usable, and allow us to remove the scary "This does not accept/produce a complete URL" warnings from the docs.
Linked PRs
Activitybarneygale commented on Nov 25, 2024 ContributorAuthor More actions @serhiy-storchaka I've revised the description as follows:
I request that we make pathname2url and url2pathname easier to use:
pathname2url() is made to accept an optional include_scheme argument that sticks file: on the front when trueurl2pathname() is made to strip any file: prefix from its argument.
Does that seem like a good idea to you?
If we go for these changes, and the user provides a URL with a non-file: scheme to url2pathname(), what should happen in your opinion? (I'm erring towards raising URLError.)
Relatedly, what do you think should happen if the user provides a query or fragment in the URL? Currently it's returned as part of the file path. (This seems under-specified to me, but I'm leaning towards raising URLError here too.)
Thank you serhiy-storchaka commented on Dec 2, 2024 I wonder -- would not be better to add such functions in os.path? The implementation is clearly platform depending, and if we add support for platform with different path format (like old macpath), it will need a completely different implementation. In custom build for exotic platform it is easier to provide a separate os.path implementation than patch urllib.request. barneygale commented on Dec 3, 2024 ContributorAuthor More actions Good question! It's something that's come up before:
The current Windows + POSIX implementations don't have much in common, but I think that will change when I address this issue and #123599. To solve them, I'm planning to use urlsplit() to parse the URL, replacing parts of the janky implementation in url2pathname(). Here's a rough draft of where I want to end up:
def url2pathname(url):
scheme, authority, path, query, fragment = urlsplit(url, scheme='file')
if scheme != 'file':
raise URLError(f'URL uses non-file scheme: {url!r}')
if query:
raise URLError(f'file URL has query: {url!r}')
if fragment:
raise URLError(f'file URL has fragment: {url!r}')
if os.name == 'nt':
# Windows-specific file URI quirks.
if authority and authority != 'localhost':
# e.g. file://server/share/file.txt
path = '//' + authority + path
elif path[:3] == '///':
# e.g. file://///server/share/file.txt
path = path[1:]
else:
if path[:1] == '/' and path[2:3] in ':|':
# Skip past extra slash before DOS drive in URL path.
path = path[1:]
if path[1:2] == '|':
# Older URLs use a pipe after a drive letter
path = path.replace('|', ':', 1)
path = path.replace('/', '\\')
elif not _is_local_authority(authority):
# POSIX only: reject URL if authority doesn't resolve to localhost.
raise URLError(f'file URL has non-local authority: {url!r}')
encoding = sys.getfilesystemencoding()
errors = sys.getfilesystemencodeerrors()
return unquote(path, encoding=encoding, errors=errors)
def pathname2url(pathname, include_scheme=False):
if os.name == 'nt':
pathname = pathname.replace('\\', '/')
encoding = sys.getfilesystemencoding()
errors = sys.getfilesystemencodeerrors()
prefix = 'file:' if include_scheme else ''
drive, root, tail = os.path.splitroot(pathname)
if drive:
if drive[:4] == '//?/':
drive = drive[4:]
if drive[:4].upper() == 'UNC/':
drive = '//' + drive[4:]
if drive[1:] == ':':
prefix += '///'
drive = quote(drive, encoding=encoding, errors=errors, safe='/:')
elif root:
prefix += '//'
tail = quote(tail, encoding=encoding, errors=errors)
return prefix + drive + root + tailThere's clearly some OS-specific code in there, but there's also a fair amount of shared and URL-specific code which might feel out-of-place in os.path.
Besides, if we added functions in os.path we'd need to either live with near-duplicate functionality in urllib, or deprecate the urllib functions (which isn't much fun for their users). I'm not ruling that out, but it seems unnecessarily if there's a reasonable path towards making pathname2url() and url2pathname() a bit more usable. barneygale commented on Dec 3, 2024 ContributorAuthor More actions FWIW, if we go with the above merger, then we can deprecate the oddball nturl2path module.
MetadataMetadataDevelopmentNo branches or pull requestsIssue actions
|