Description

AndyQ

opened on May 3, 2023

Issue body actions

Relating to a Werkzueg issue (pallets/werkzeug#2691), when parsing an iOS App install url e.g.
itms-services:action=download-manifest&url=https://theacmeinc.com/abcdefeg, urlunpslit returns an invalid url.

e.g.

vals = urlparse( "itms-services://?action=download-manifest&url=https://theacmeinc.com/abcdefeg" )
print(vals)

newURL = urlunsplit((vals.scheme, vals.netloc, vals.path, vals.query, vals.params))
print( newURL )

prints:

ParseResult(scheme='itms-services', netloc='', path='', params='', query='action=download-manifest&url=https://theacmeinc.com/abcdefeg', fragment='')

itms-services:?action=download-manifest&url=https://theacmeinc.com/abcdefeg

Note the newURL is missing the // after the itms-services scheme.

This scheme is used to install ad-hoc and enterprise iOS apps.

Your environment

Tested on Apple M1 Max - 13.4 Beta (22F5049e)
Python: 3.10.10

For more details on the scheme here is a link to the Apple documentation (look for the "Use a website to distribute the app" section).
https://support.apple.com/en-gb/guide/deployment/depce7cefc4d/web

Linked PRs

• gh-104139: Add itms-services to uses_netloc urllib.parse. #104312

Activity

1. AndyQ

   added

   type-bugAn unexpected behavior, bug, or errorAn unexpected behavior, bug, or error

   on May 3, 2023
2. arhadthedev

   added

   stdlibStandard Library Python modules in the Lib/ directoryStandard Library Python modules in the Lib/ directory

   on May 3, 2023
3. davidism commented on May 3, 2023

   

   davidism

   on May 3, 2023

   More actions

   I don't think this scheme is standard. https://url.spec.whatwg.org/#url-serializing says "If url's host is non-null: Append "//" to output." This itms-services URL doesn't have a host, so it shouldn't require a //. I think there's some other special casing for schemes with the uses_netloc list, a similar "add //" could be added.

   cpython/Lib/urllib/parse.py

   Lines 44 to 74 in 9f9e001

   	# A classification of schemes.
   	# The empty string classifies URLs with no scheme specified,
   	# being the default value returned by “urlsplit” and “urlparse”.
   	uses_relative = ['', 'ftp', 'http', 'gopher', 'nntp', 'imap',
   	'wais', 'file', 'https', 'shttp', 'mms',
   	'prospero', 'rtsp', 'rtspu', 'sftp',
   	'svn', 'svn+ssh', 'ws', 'wss']
   	uses_netloc = ['', 'ftp', 'http', 'gopher', 'nntp', 'telnet',
   	'imap', 'wais', 'file', 'mms', 'https', 'shttp',
   	'snews', 'prospero', 'rtsp', 'rtspu', 'rsync',
   	'svn', 'svn+ssh', 'sftp', 'nfs', 'git', 'git+ssh',
   	'ws', 'wss']
   	uses_params = ['', 'ftp', 'hdl', 'prospero', 'http', 'imap',
   	'https', 'shttp', 'rtsp', 'rtspu', 'sip', 'sips',
   	'mms', 'sftp', 'tel']
   	# These are not actually used anymore, but should stay for backwards
   	# compatibility. (They are undocumented, but have a public-looking name.)
   	non_hierarchical = ['gopher', 'hdl', 'mailto', 'news',
   	'telnet', 'wais', 'imap', 'snews', 'sip', 'sips']
   	uses_query = ['', 'http', 'wais', 'imap', 'https', 'shttp', 'mms',
   	'gopher', 'rtsp', 'rtspu', 'sip', 'sips']
   	uses_fragment = ['', 'ftp', 'hdl', 'http', 'gopher', 'news',
   	'nntp', 'wais', 'https', 'shttp', 'snews',
   	'file', 'prospero']
4. davidism commented on May 8, 2023

   

   davidism

   on May 8, 2023

   More actions

   @gpshead I know you were looking at some other urllib issues recently, could you comment on this?
5. gpshead

   self-assigned this

   on May 8, 2023
6. gpshead commented on May 8, 2023

   

   gpshead

   on May 8, 2023

   Member

   More actions

   I agree with @davidism that the WhatWG URL spec does not require // when there is no pathname.

   Regardless, behavior wise this seems to match our existing uses_netloc special case so we can just add it to that list in Lib/urllib/parse.py and add test coverage in Lib/test/test_urlparse.py. I made a PR.

   Workaround for this to "work" on existing Pythons:

   if "itms-services" not in urllib.parse.uses_netloc:
       urllib.parse.uses_netloc.append("itms-services")

   I'm calling this a feature as code will have to deal with Python's that do not list it as such for a long time anyways via a code snippet like that, so backporting doesn't seem consistently helpful.

   Reacted by Dan Lenski
7. gpshead

   added

   type-featureA feature request or enhancementA feature request or enhancement

   3.12only security fixesonly security fixes

   on May 8, 2023
8. gpshead

   added a commit that references this issue on May 9, 2023

   gh-104139: Add itms-services to uses_netloc urllib.parse. (#104312)

   82f789b
9. gpshead commented on May 9, 2023

   

   gpshead

   on May 9, 2023

   Member

   More actions

   fixed in 3.12.
10. gpshead

    closed this as completedon May 9, 2023
11. carljm

    added 3 commits that reference this issue on May 9, 2023

    Merge branch 'main' into fixsendfamily

    e40071d

    Merge branch 'main' into supermon

    c48eee3

    Merge branch 'main' into fixloadsuperspec

    cf11908

Issue actions