Description

planetA

opened on Oct 7, 2022

Issue body actions

Subject

I parse an IP address with port to get a url, but ParseResult does not look right.

Environment

Describe your environment.
At least, paste here the output of:

OS Linux-5.10.0-17-amd64-x86_64-with-glibc2.31
>>> print("Python", platform.python_version())
Python 3.9.2

Steps to Reproduce

from urllib.parse import urlparse
urlparse('192.168.1.132:16992')

Expected Behavior

ParseResult(scheme='', netloc='192.168.1.132:16992', path='', params='', query='', fragment='')

Actual Behavior

ParseResult(scheme='192.168.1.132', netloc='', path='16992', params='', query='', fragment='')

I guess scheme should not be an IP address

This issue seems to be related: #38644

Reacted by Kcchouette

Activity

1. planetA

   added

   type-bugAn unexpected behavior, bug, or errorAn unexpected behavior, bug, or error

   on Oct 7, 2022
2. planetA commented on Oct 7, 2022

   

   planetA

   on Oct 7, 2022

   Author

   More actions

   Adding scheme does not seem to help:

   urlparse('1.2.3.4:80', 'http')

   ParseResult(scheme='1.2.3.4', netloc='', path='80', params='', query='', fragment='')
   
3. Jason-Y-Z commented on Oct 7, 2022

   

   Jason-Y-Z

   on Oct 7, 2022

   Contributor

   More actions

   I might be horribly wrong, but from the example given -

   >>> from urllib.parse import urlparse
   >>> urlparse("scheme://netloc/path;parameters?query#fragment")
   ParseResult(scheme='scheme', netloc='netloc', path='/path;parameters', params='',
               query='query', fragment='fragment')
   

   I think it's treating the IP address of your string as the scheme because it's the first part of the URL string (before the colon).
4. planetA commented on Oct 7, 2022

   

   planetA

   on Oct 7, 2022

   Author

   More actions

   According to RFC3986 scheme should start with an alpha character.

   Reacted by Konrad Sikorski and Taylor
5. kwsp commented on Oct 8, 2022

   

   kwsp

   on Oct 8, 2022

   Contributor

   More actions

   Quoting from the documentation of urlparse

   Following the syntax specifications in RFC 1808, urlparse recognizes a netloc only if it is properly introduced by ‘//’. Otherwise the input is presumed to be a relative URL and thus to start with a path component.

   To get your expected result, you need to prefix with // for urlparse to recognize the netloc:

   >>> urlparse('//192.168.1.132:16992')
   ParseResult(scheme='', netloc='192.168.1.132:16992', path='', params='', query='', fragment='')
   

   Since your input doesn't start with a scheme or '//', urlparse should treat your input as a "relative URL". However, your input also contains :, which doesn't make sense in a relative path, so I guess the result you see is actually undefined behavior?

   Reacted by Shantanu, Jason Zhang and Konrad Sikorski
6. hauntsaninja

   added

   pendingThe issue will be closed if no feedback is providedThe issue will be closed if no feedback is provided

   on Oct 9, 2022
7. Kcchouette commented on Oct 23, 2022

   

   Kcchouette

   on Oct 23, 2022

   More actions

   Hello
   I have the same problem in recent version of python

   >>> urlparse('google.com:443')
   ParseResult(scheme='google.com', netloc='', path='443', params='', query='', fragment='')
   

   Same if I specify the scheme in the function:

   >>> urlparse('google.com:443', scheme="https")
   ParseResult(scheme='google.com', netloc='', path='443', params='', query='', fragment='')
   
8. kwsp commented on Oct 23, 2022

   

   kwsp

   on Oct 23, 2022

   Contributor

   More actions

   I think the issue boils down to this:

   urllib.parse.urlparse follows the contents of RFC1808 (Relative URL) religiously. However, RFC1808 is a companion to RFC1738 (URL), and because of that, RFC1808 skipped the description of net_loc entirely (Section 2.1 of RFC1808), since it's describe in Section 3.1 of RFC1738. As a result, RFC1808 skips the description of port in the net_loc entirely, and doesn't consider the case where the net_loc is presented by itself, without the // prefix, which is undefined behaviour as this case is not defined by RFC1808 or RFC1738.

   I get the impression that nowadays, most people do not know that according to RFC1738, a net_loc must be prefixed by //, and using a netloc by itself (like 192.168.1.132:16992 presented in the original issue) is very commonplace, and we should just support it in urlparse.

   I will work on a patch for this
9. kwsp commented on Oct 24, 2022

   

   kwsp

   on Oct 24, 2022

   Contributor

   More actions

   On second thought and reading of the discussion on #38644, this is actually documented behaviour, as documented in the second paragraph of https://docs.python.org/3/library/urllib.parse.html#urllib.parse.urlparse.

   The only bug here then is as @planetA mentioned:

   According to RFC3986 scheme should start with an alpha character.

   I can look into a fix for this - maybe raise an error if the parsed scheme doesn't start with an alpha character?

   Adding scheme does not seem to help:

   urlparse('1.2.3.4:80', 'http')

   ParseResult(scheme='1.2.3.4', netloc='', path='80', params='', query='', fragment='')
   

   This is technically documented behaviour too:

   The scheme argument gives the default addressing scheme, to be used only if the URL does not specify one. It should be the same type (text or bytes) as urlstring, except that the default value '' is always allowed, and is automatically converted to b'' if appropriate.

   Since urlparse technically discovered a scheme, therefore it ignores the scheme argument.

   Reacted by Jason Zhang
10. arhadthedev commented on Mar 4, 2023

    

    arhadthedev

    on Mar 4, 2023

    Member

    More actions

    I can look into a fix for this

    ping
11. kwsp commented on Mar 9, 2023

    

    kwsp

    on Mar 9, 2023

    Contributor

    More actions

    @arhadthedev Sorry I forgot about this.

    Taking a look again, the behaviour and changes I described above has already been fixed in this issue (#99418) and PR (#99421).

    We can close this issue now.
12. sobolevn

    closed this as completedon Mar 11, 2023

Issue actions