GeezerVison is showing github.com — PurePosixPath no longer correctly parses PureWindowsPath · Issue #103631 · python/cpython
PurePosixPath no longer correctly parses PureWindowsPath #103631DescriptionBug report
In pathlib prior to Python 3.12, passing a PureWindowsPath to a PurePosixPath resulted in the Windows separator (\) being converted to the POSIX separator (/). However, in the current main branch the backslashes are preserved in the PurePosixPath object.
Here is an example which illustrates this:
Python 3.12.0a7 (tags/v3.12.0a7:b861ba4, Apr 6 2023, 16:09:18) [Clang 10.0.0 ] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import pathlib
>>> print(pathlib.PurePosixPath(pathlib.PureWindowsPath(r"a\b\c")))
a\b\c
>>> print(pathlib.PurePosixPath(pathlib.PureWindowsPath(r"a\b\c")).as_posix())
a\b\c Python 3.11.2 (tags/v3.11.2:878ead1, Mar 9 2023, 16:26:59) [Clang 10.0.0 ] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import pathlib
>>> print(pathlib.PurePosixPath(pathlib.PureWindowsPath(r"a\b\c")))
a/b/c
>>> print(pathlib.PurePosixPath(pathlib.PureWindowsPath(r"a\b\c")).as_posix())
a/b/c (The behaviour is the same if using PosixPath or WindowsPath on the relevant platform; it's not specific to the "pure" variants.)
Before the recent refactoring of the module, passing one Path or PurePath object to another resulted in the _parts attribute being inspected. This was a list of the individual path elements (e.g. ['a', 'b', 'c'] for the path a\b\c). The _parts attribute was removed in GH-102476 and replaced with _tail, but with slightly different semantics.
The current code replaces any os.altsep in the path with os.sep, which for WindowsPath replaces / with \ but for PosixPath does nothing as there is no alternative separator. However, the following will produce a correct result:
Python 3.12.0a7+ (heads/main:bd2ed06, Apr 19 2023, 15:27:47) [GCC 9.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import pathlib
>>> pathlib.PurePosixPath._flavour.altsep = "\\"
>>> pathlib.PurePosixPath(pathlib.PureWindowsPath(r"a\b\c")).as_posix()
'a/b/c' Thus I think the problem can be isolated to these lines here:
|
@classmethod |
|
def _parse_path(cls, path): |
|
if not path: |
|
return '', '', [] |
|
sep = cls._flavour.sep |
|
altsep = cls._flavour.altsep |
|
if altsep: |
|
path = path.replace(altsep, sep) |
|
drv, root, rel = cls._flavour.splitroot(path) |
Your environment
- CPython versions tested on: 3.8, 3.11 and 3.12 (alpha 7 and HEAD)
- Operating system and architecture: Ubuntu 20.04 and Windows 10
Linked PRs
Activityeryksun commented on Apr 19, 2023 If constructing a POSIX path directly from a Windows path is supported, it won't be implemented by adding backslash as an alternate path separator for POSIX paths. Backslash is a normal name character in POSIX, not a reserved path separator.
As a workaround, you can get a POSIX path before passing the path to the PurePosixPath constructor.
>>> pathlib.PurePosixPath(pathlib.PureWindowsPath(r"a\b\c").as_posix())
PurePosixPath('a/b/c')barneygale commented on Apr 19, 2023
In pathlib prior to Python 3.12, passing a PureWindowsPath to a PurePosixPath resulted in the Windows separator (\) being converted to the POSIX separator (/)
Only for relative paths. If you try to pass an absolute path, things quickly go south:
barney@acorn ~ $ python3.8
Python 3.8.10 (default, Mar 13 2023, 10:26:41)
[GCC 9.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import pathlib
>>> pathlib.PurePosixPath(pathlib.PureWindowsPath(r"c:\a\b\c"))
PurePosixPath('c:\\/a/b/c')
>>> pathlib.PurePosixPath(pathlib.PureWindowsPath(r"\\server\share\a\b\c"))
PurePosixPath('\\\\server\\share\\/a/b/c')barneygale commented on Apr 20, 2023 FYI, the behaviour was changed in #101667 and #102454 in order to fix this bug:
As a result, these all do the same thing:
>>> from pathlib import PurePosixPath, PureWindowsPath
>>> from os import fspath
>>> PurePosixPath(r"c:\a\b\c")
PurePosixPath('c:\\a\\b\\c')
>>> PurePosixPath(PureWindowsPath(r"c:\a\b\c"))
PurePosixPath('c:\\a\\b\\c')
>>> PurePosixPath(fspath(PureWindowsPath(r"c:\a\b\c")))
PurePosixPath('c:\\a\\b\\c')We tweaked the docs slightly to make clear that all os.PathLike objects are treated similarly:
diff --git a/Doc/library/pathlib.rst b/Doc/library/pathlib.rst
index c8a734ecad8e..8e91936680fa 100644
--- a/Doc/library/pathlib.rst
+++ b/Doc/library/pathlib.rst
@@ -105,8 +105,9 @@ we also call *flavours*:
PurePosixPath('setup.py')
Each element of *pathsegments* can be either a string representing a
- path segment, an object implementing the :class:`os.PathLike` interface
- which returns a string, or another path object::
+ path segment, or an object implementing the :class:`os.PathLike` interface
+ where the :meth:`~os.PathLike.__fspath__` method returns a string,
+ such as another path object::
>>> PurePath('foo', 'some/path', 'bar')
PurePosixPath('foo/some/path/bar')Eryk's suggestion looks ideal to me, and clearly signals intentions. mwichmann commented on May 22, 2023
As a workaround, you can get a POSIX path before passing the path to the PurePosixPath constructor.
>>> pathlib.PurePosixPath(pathlib.PureWindowsPath(r"a\b\c").as_posix())
PurePosixPath('a/b/c')
The problem with this is it means your code has to be system-aware. The case I've run across this, just figuring out it was broken now, uses a PureWindowsPath when processing a file of pathnames, so it can be read in an agnostic way (that is, either forward or backward slashes in the file work no matter what the platform), then converts it to concrete paths by calling Path() on the result, which used to work to produce suitable "native" paths. Now it's going to have to LBYL to figure out whether to as_posix or not. barneygale commented on May 22, 2023 Hm! Backslashes can appear in Posix filenames. How would I specify such a path in your file of pathnames? Or is that not a problem because you create all these paths from within your program? barneygale commented on May 22, 2023 For your use case, would this work?
pathlib.Path(pathlib.PureWindowsPath(yourpath).as_posix()) as_posix() returns a path with forward slashes as separators, which is valid for both WindowsPath and PosixPath.
Alternatively:
pathlib.Path(yourpath.replace('\\', '/'))mwichmann commented on May 22, 2023
For your use case, would this work?
pathlib.Path(pathlib.PureWindowsPath(r"a\b\c").as_posix()) as_posix() returns a path with forward slashes as separators, which is valid for both WindowsPath and PosixPath.
The usecase is more specifically a list of tests to run (cmdline or from a file), and we know we're not creating tests with particularly "odd" names, so it's a (somewhat) controlled namespace. We've wanted to display the names the way they were entered if they were on Windows, so if someone actually typed foo\bar it should look like foo\bar was being run, but that's just a nicety; normalizing all paths to "POSIX style" isn't unreasonable - somebody already tried to push that idea. barneygale commented on May 22, 2023 Gotcha.
Path() will switch the forward slashes back to backslashes on Windows, which sounds like what you want?
barneygale commented on May 25, 2023 Upon further consideration I don't think this is worth the backwards-compatibility break. PR up: #104949 added a commit that references this issue on May 26, 2023GH-103631: Fix `PurePosixPath(PureWindowsPath(...))` separator handli… barneygale commented on May 26, 2023 Thanks for reporting + discussing, all. This is fixed in:
3.12 beta 2 should contain the fix.
MetadataMetadataDevelopmentNo branches or pull requestsIssue actions
|