GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — GH-125866: Support complete "file:" URLs in urllib by barneygale · Pull Request #132378 · python/cpython

Conversation

@barneygale

@barneygale barneygale commented Apr 10, 2025 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Contributor

Add optional add_scheme argument to urllib.request.pathname2url(); when set to true, a complete URL is returned. Likewise add optional require_scheme argument to url2pathname(); when set to true, a complete URL is accepted.

  • Issue: Improve file URI ergonomics in urllib.request #125866

📚 Documentation preview 📚: https://cpython-previews--132378.org.readthedocs.build/

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
@barneygale
pythonGH-125866: Support complete "file:" URLs in urllib
8c9fe62
Add optional *add_scheme* argument to `urllib.request.pathname2url()`; when
set to true, a complete URL is returned. Likewise add optional *has_scheme*
argument to `urllib.request.url2pathname()`; when set to true, a complete
URL is accepted.
@bedevere-app bedevere-app Bot mentioned this pull request Apr 10, 2025
Improve file URI ergonomics in urllib.request #125866
Closed
barneygale added 2 commits April 10, 2025 21:49
@barneygale
Fix lint
e5a94ce
@barneygale
Document that URLError is raised
3dff942
@barneygale
barneygale marked this pull request as ready for review April 10, 2025 21:44
@bedevere-app bedevere-app Bot added the awaiting core review label Apr 10, 2025
@barneygale
barneygale requested review from AA-Turner, picnixz and serhiy-storchaka April 10, 2025 21:45
@barneygale

barneygale commented Apr 10, 2025 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Contributor Author

Some Qs for reviewers:

  1. Do we like the "has_scheme" and "add_scheme" names? Perhaps the latter should be "include_scheme"?
  2. Should this wait until 3.15 given we've already made a few changes to url2pathname() and pathname2url() in 3.14?

Thanks in advance :]

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@barneygale
Doc[string] tweaks
4be48ab
@picnixz

picnixz commented Apr 12, 2025

Copy link
Copy Markdown
Member

Do we like the "has_scheme" and "add_scheme" names? Perhaps the latter should be "include_scheme"?

If I were to chose one, I would use with_scheme for both because it works in both cases ("should we include the scheme in the output?" and "is the scheme present in the input"?). I would however make it a keyword-only argument so that it's easier to change in 5 years for instance if something went wrong with our naming...

Should this wait until 3.15 given we've already made a few changes to url2pathname() and pathname2url() in 3.14?

I don't think we need to wait if we can ship them. Waiting means that users also wait for at least 2y until they can consider 3.15 as the stable release (and I guess many like having the latest stable release as their main version).

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

picnixz
picnixz reviewed Apr 12, 2025
View reviewed changes
Comment thread Doc/library/urllib.request.rst Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread Lib/test/test_urllib.py Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread Lib/urllib/request.py Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread Lib/urllib/request.py Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread Lib/urllib/request.py Resolved

Uh oh!

There was an error while loading. Please reload this page.

barneygale and others added 3 commits April 12, 2025 14:36
@barneygale @picnixz
Apply suggestions from code review
19d3a70
Co-authored-by: Bénédikt Tran <[email protected]>
@barneygale
Make keyword-only
2a744f8
@barneygale
Add a few more test cases
873b13c
@barneygale

barneygale commented Apr 12, 2025

Copy link
Copy Markdown
Contributor Author

Thank you for the review + feedback - much appreciated :)

I'm not sure how I feel about with_scheme -- is it a little imprecise? I wonder what Adam and Serhiy think about the arguments naming

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@barneygale
Fix windows tests
c9ba69c
@picnixz

picnixz commented Apr 12, 2025

Copy link
Copy Markdown
Member

is it a little imprecise

A bit, but at least users don't need to remember different names for the path->url and url->path functions.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

serhiy-storchaka
serhiy-storchaka approved these changes Apr 12, 2025
View reviewed changes

@serhiy-storchaka serhiy-storchaka left a comment

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. 👍

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

barneygale reacted with heart emoji
All reactions
  • ❤️ 1 reaction
@bedevere-app bedevere-app Bot added awaiting merge and removed awaiting core review labels Apr 12, 2025
@barneygale
Fix tests
5e7b71e
@barneygale

barneygale commented Apr 13, 2025

Copy link
Copy Markdown
Contributor Author

A bit, but at least users don't need to remember different names for the path->url and url->path functions.

Well, they didn't have to until we made the argument keyword-only 🙈

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@picnixz

picnixz commented Apr 13, 2025

Copy link
Copy Markdown
Member

Well, they didn't have to until we made the argument keyword-only

True! but on the other hand, if I see func(x, True) I would anyway need to find the docs to see what it's about I think. I'm suggesting a keyword-only to ease future changes (like adding other arguments afterwards). But I honestly don't mind having the names you suggested!

barneygale reacted with thumbs up emoji
All reactions
  • 👍 1 reaction

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@barneygale

barneygale commented Apr 13, 2025

Copy link
Copy Markdown
Contributor Author

I'm going to ask on discord, as I like with_scheme about equally to has_scheme / add_scheme and I'm struggling to decide.

picnixz reacted with thumbs up emoji
All reactions
  • 👍 1 reaction

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@ncoghlan

ncoghlan commented Apr 13, 2025

Copy link
Copy Markdown
Contributor

+1 for keyword-only (non-keyword boolean flags read poorly when called with boolean literals, and reducing the API surface by restricting the calling style generally makes future maintenance easier)

For the parameter naming though, is the flag on url2pathname actually necessary? Couldn't a file:// prefix (if present) be stripped unconditionally? Then if a flag were to be added, it would be require_scheme=True (to make it clear that the purpose of the flag is ensure that the scheme prefix being missing is considered an error rather than being accepted as a scheme-free partial URL).

I also think the ambiguity concern with a unified flag name is valid, so I'd be inclined to keep add_scheme on pathname2url either way (the scheme is added to the output, it isn't expected as part of the input)

barneygale and gpshead reacted with heart emoji
All reactions
  • ❤️ 2 reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@barneygale

barneygale commented Apr 13, 2025

Copy link
Copy Markdown
Contributor Author

For the parameter naming though, is the flag on url2pathname actually necessary? Couldn't a file:// prefix (if present) be stripped unconditionally? Then if a flag were to be added, it would be require_scheme=True (to make it clear that the purpose of the flag is ensure that the scheme prefix being missing is considered an error rather than being accepted as a scheme-free partial URL).

The prefix to remove would be file: rather than file://, because urllib supports expressing relative paths as URLs without authorities:

>>> pathname2url('hello')
'hello'
>>> url2pathname('foo:bar')
'foo:bar'
>>> url2pathname('file:bar')
'file:bar'

Though it's still very unlikely, I worry that file: could begin a legitimate relative path, and that we'd change behaviour here.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@serhiy-storchaka

serhiy-storchaka commented Apr 13, 2025

Copy link
Copy Markdown
Member

It was a pain to handle URLs like "file:123" where "file" was interpreted as a host name and "123" as a port number..

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@ncoghlan

ncoghlan commented Apr 13, 2025

Copy link
Copy Markdown
Contributor

That makes sense - the input ambiguity would be much worse than I initially thought it would be.

I'd suggest add_scheme/require_scheme as the bikeshed colours then. has_scheme and with_scheme don't really indicate "will throw an exception if the file: prefix is missing" for the URL input case to me.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@picnixz

picnixz commented Apr 13, 2025

Copy link
Copy Markdown
Member

has_scheme and with_scheme don't really indicate "will throw an exception if the file: prefix is missing" for the URL input case to me

I hadn't thought about the expected contract and the parameter name so require_scheme is a good one. In my projects, I usually use "strict" as the keyword argument meant to indicate that I want a strict parsing but I think strict is not explicit enough.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@barneygale
has_scheme --> require_scheme
7a4140c
picnixz
picnixz approved these changes Apr 13, 2025
View reviewed changes

@picnixz picnixz left a comment

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just making sure that we have kw-only at runtime as well and LGTM.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
Comment thread Lib/urllib/request.py Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread Lib/urllib/request.py Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread Lib/test/test_urllib.py Resolved

Uh oh!

There was an error while loading. Please reload this page.

@barneygale

barneygale commented Apr 13, 2025

Copy link
Copy Markdown
Contributor Author

I like require_scheme too! I've adjusted the PR to use that.

One last thought: we could maybe accept a str | None argument called scheme:

>>> url = 'file:///C:/Program%20Files'
>>> url2pathname(url, scheme='file')
'C:\\Program Files'
>>> path = 'C:\\Program Files'
>>> pathname2url(path, scheme='file')
'file:///C:/Program%20Files'

... but I can't really think of a use case for values other than None and "file", plus it may conflict with the meaning of the scheme argument to urlparse() and friends.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@barneygale @picnixz
Apply suggestions from code review
ec272ef
Co-authored-by: Bénédikt Tran <[email protected]>
@picnixz

picnixz commented Apr 13, 2025

Copy link
Copy Markdown
Member

... but I can't really think of a use case for values other than None and "file"

I also don't know of another scheme for local paths and for which url2pathname (the "pathname" part in the name is important) would make sense as well. If we need to support another URL scheme, then we should probably have another function (well maybe, the ftp scheme could make sense because it's related to files but it'd be weird to use url2pathname in this case IMO)

barneygale reacted with thumbs up emoji
All reactions
  • 👍 1 reaction

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@barneygale
Use subtests
c570e2a
@barneygale
barneygale requested a review from picnixz April 13, 2025 17:28
@barneygale
Merge branch 'main' into pythongh-125866-again
db2876d
picnixz
picnixz approved these changes Apr 13, 2025
View reviewed changes

@picnixz picnixz left a comment

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some nits but you can go ahead whenever you want.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
Comment thread Lib/test/test_urllib.py Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread Lib/test/test_urllib.py Resolved

Uh oh!

There was an error while loading. Please reload this page.

@barneygale @picnixz
Apply suggestions from code review
b5e2fb7
Co-authored-by: Bénédikt Tran <[email protected]>
@barneygale
barneygale merged commit ccad61e into python:main Apr 14, 2025
@bedevere-app bedevere-app Bot removed the awaiting merge label Apr 14, 2025
@barneygale barneygale mentioned this pull request Jul 19, 2025
Adjust path_to_url et al. to produce the same results on Python 3.14+ pypa/pip#13423
Merged
@coderabbitai coderabbitai Bot mentioned this pull request Jun 4, 2026
fix: green CI (frontend eslint) + gucci.com production scan (missing scanner deps) strikersam/autonomous-ai-agency#386
Merged
@coderabbitai coderabbitai Bot mentioned this pull request Jun 26, 2026
Add ComfyUI node compatibility checker Comfy-Org/workflow_templates#973
Merged
2 of 3 tasks
@coderabbitai coderabbitai Bot mentioned this pull request Jul 30, 2026
feat(i18n): add full Spanish (Español) translation for book, docs, and navigation bojieli/ai-agent-book#503
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@serhiy-storchaka serhiy-storchaka serhiy-storchaka approved these changes
@picnixz picnixz picnixz approved these changes
@AA-Turner AA-Turner Awaiting requested review from AA-Turner AA-Turner is a code owner

Uh oh!

There was an error while loading. Please reload this page.

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Uh oh!

There was an error while loading. Please reload this page.

4 participants

@barneygale @picnixz @ncoghlan @serhiy-storchaka
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
You are visitor 01402503 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.