GeezerVison is showing github.com — bpo-43757: make pathlib use os.path.realpath() to resolve all symlinks in a path by barneygale · Pull Request #25264 · python/cpython
bpo-43757: make pathlib use os.path.realpath() to resolve all symlinks in a path
#25264
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and
privacy statement . We’ll occasionally send you account related emails.
Already on GitHub?
Sign in
to your account
Uh oh!
There was an error while loading. Please reload this page.
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
23 commits
Select commit
Hold shift + click to select a range
164dbfd
bpo-43757: make pathlib use os.path.realpath() to resolve all symlink…
barneygale Apr 7, 2021
2894e0a
Adjust `os.path.realpath` to match `pathlib.Path.resolve()` behaviour…
barneygale Apr 8, 2021
495bd8b
Call `os.stat()` to raise `OSError(errno.ELOOP, ...)`.
barneygale Apr 8, 2021
492c66b
Remove unused import
barneygale Apr 8, 2021
c8bac15
Stop ignoring ERROR_CANT_RESOLVE_FILENAME error in _getfinalpathname_…
barneygale Apr 8, 2021
fe69688
Fix symlink detection and `posixpath` tests.
barneygale Apr 8, 2021
4114be9
Raise RuntimeError in Path.resolve() if realpath() raises ERROR_CANT_…
barneygale Apr 8, 2021
2a594ce
Remove unused import
barneygale Apr 8, 2021
fb36a40
Make documentation a little more general to accurately cover Windows.
barneygale Apr 8, 2021
0634486
First pass on fixing `ntpath` tests
barneygale Apr 8, 2021
600fc9e
Second pass on fixing `ntpath` tests
barneygale Apr 8, 2021
047471c
Fix indentation
barneygale Apr 8, 2021
bf3a3f7
Copy description of 'strict' from pathlib.
barneygale Apr 8, 2021
f839db0
Add tests
barneygale Apr 8, 2021
2d30bb7
Add NEWS
barneygale Apr 8, 2021
68c6533
Fix ntpath tests (pass 1)
barneygale Apr 8, 2021
9fa60eb
Add some notes on OS differences
barneygale Apr 10, 2021
f979947
Split NEWS
barneygale Apr 10, 2021
a82bc18
Do not suppress initial ERROR_CANT_RESOLVE_FILENAME error in non-stri…
barneygale Apr 10, 2021
86c318c
realpath(): only raise OSError for symlink loops in strict mode
barneygale Apr 13, 2021
7362f7e
Fix test_ntpath (pass 1)
barneygale Apr 13, 2021
72147c5
Fix test_ntpath (pass 2)
barneygale Apr 13, 2021
df04357
Split up exception handling in resolve() for greater clarity.
barneygale Apr 13, 2021
File filter
Filter by extension
.py
(5)
.rst
(2)
All 2 file types selected
Viewed files
Conversations
Failed to load comments.
Loading
Uh oh!
There was an error while loading. Please reload this page.
Jump to
Jump to file
Failed to load files.
Loading
Uh oh!
There was an error while loading. Please reload this page.
Diff view
Hide whitespace
Diff view
Hide whitespace
There are no files selected for viewing
Original file line number
Diff line number
Diff line change
Expand Up
@@ -344,22 +344,34 @@ the :mod:`glob` module.)
Accepts a :term:`path-like object`.
.. function:: realpath(path)
.. function:: realpath(path, *, strict=False )
Return the canonical path of the specified filename, eliminating any symbolic
links encountered in the path (if they are supported by the operating
system).
If a path doesn't exist or a symlink loop is encountered, and *strict* is
``True``, :exc:`OSError` is raised. If *strict* is ``False``, the path is
resolved as far as possible and any remainder is appended without checking
whether it exists.
.. note::
When symbolic link cycles occur, the returned path will be one member of
the cycle, but no guarantee is made about which member that will be.
This function emulates the operating system's procedure for making a path
canonical, which differs slightly between Windows and UNIX with respect
to how links and subsequent path components interact.
Operating system APIs make paths canonical as needed, so it's not
normally necessary to call this function.
.. versionchanged:: 3.6
Accepts a :term:`path-like object`.
.. versionchanged:: 3.8
Symbolic links and junctions are now resolved on Windows.
.. versionchanged:: 3.10
The *strict* parameter was added.
.. function:: relpath(path, start=os.curdir)
Expand Down
Original file line number
Diff line number
Diff line change
Expand Up
@@ -635,7 +635,7 @@ def _getfinalpathname_nonstrict(path):
tail = join(name, tail) if tail else name
return tail
def realpath(path):
def realpath(path, *, strict=False ):
path = normpath(path)
if isinstance(path, bytes):
prefix = b'\\\\?\\'
Expand All
@@ -660,6 +660,8 @@ def realpath(path):
path = _getfinalpathname(path)
initial_winerror = 0
except OSError as ex:
if strict:
Comment thread
barneygale marked this conversation as resolved.
Resolved
Uh oh!
There was an error while loading. Please reload this page.
raise
initial_winerror = ex.winerror
path = _getfinalpathname_nonstrict(path)
# The path returned by _getfinalpathname will always start with \\?\ -
Expand Down
Original file line number
Diff line number
Diff line change
Expand Up
@@ -14,12 +14,6 @@
from urllib.parse import quote_from_bytes as urlquote_from_bytes
if os.name == 'nt':
from nt import _getfinalpathname
else:
_getfinalpathname = None
__all__ = [
"PurePath", "PurePosixPath", "PureWindowsPath",
"Path", "PosixPath", "WindowsPath",
Expand All
@@ -29,14 +23,17 @@
# Internals
#
_WINERROR_NOT_READY = 21 # drive exists but is not accessible
_WINERROR_INVALID_NAME = 123 # fix for bpo-35306
_WINERROR_CANT_RESOLVE_FILENAME = 1921 # broken symlink pointing to itself
# EBADF - guard against macOS `stat` throwing EBADF
_IGNORED_ERROS = (ENOENT, ENOTDIR, EBADF, ELOOP)
_IGNORED_WINERRORS = (
21, # ERROR_NOT_READY - drive exists but is not accessible
123, # ERROR_INVALID_NAME - fix for bpo-35306
1921, # ERROR_CANT_RESOLVE_FILENAME - fix for broken symlink pointing to itself
)
_WINERROR_NOT_READY,
_WINERROR_INVALID_NAME,
_WINERROR_CANT_RESOLVE_FILENAME)
def _ignore_error(exception):
return (getattr(exception, 'errno', None) in _IGNORED_ERROS or
Expand Down
Expand Up
@@ -186,30 +183,6 @@ def casefold_parts(self, parts):
def compile_pattern(self, pattern):
return re.compile(fnmatch.translate(pattern), re.IGNORECASE).fullmatch
def resolve(self, path, strict=False):
s = str(path)
if not s:
return path._accessor.getcwd()
previous_s = None
if _getfinalpathname is not None:
if strict:
return self._ext_to_normal(_getfinalpathname(s))
else:
tail_parts = [] # End of the path after the first one not found
while True:
try:
s = self._ext_to_normal(_getfinalpathname(s))
except FileNotFoundError:
previous_s = s
s, tail = os.path.split(s)
tail_parts.append(tail)
if previous_s == s:
return path
else:
return os.path.join(s, *reversed(tail_parts))
# Means fallback on absolute
return None
def _split_extended_path(self, s, ext_prefix=ext_namespace_prefix):
prefix = ''
if s.startswith(ext_prefix):
Expand All
@@ -220,10 +193,6 @@ def _split_extended_path(self, s, ext_prefix=ext_namespace_prefix):
s = '\\' + s[3:]
return prefix, s
def _ext_to_normal(self, s):
# Turn back an extended path into a normal DOS-like path
return self._split_extended_path(s)[1]
def is_reserved(self, parts):
# NOTE: the rules for reserved names seem somewhat complicated
# (e.g. r"..\NUL" is reserved but not r"foo\NUL").
Expand Down
Expand Up
@@ -281,54 +250,6 @@ def casefold_parts(self, parts):
def compile_pattern(self, pattern):
return re.compile(fnmatch.translate(pattern)).fullmatch
def resolve(self, path, strict=False):
sep = self.sep
accessor = path._accessor
seen = {}
def _resolve(path, rest):
if rest.startswith(sep):
path = ''
for name in rest.split(sep):
if not name or name == '.':
# current dir
continue
if name == '..':
# parent dir
path, _, _ = path.rpartition(sep)
continue
if path.endswith(sep):
newpath = path + name
else:
newpath = path + sep + name
if newpath in seen:
# Already seen this path
path = seen[newpath]
if path is not None:
# use cached value
continue
# The symlink is not resolved, so we must have a symlink loop.
raise RuntimeError("Symlink loop from %r" % newpath)
# Resolve the symbolic link
try:
target = accessor.readlink(newpath)
except OSError as e:
if e.errno != EINVAL and strict:
raise
# Not a symlink, or non-strict mode. We just leave the path
# untouched.
path = newpath
else:
seen[newpath] = None # not resolved symlink
path = _resolve(path, target)
seen[newpath] = path # resolved symlink
return path
# NOTE: according to POSIX, getcwd() cannot contain path components
# which are symlinks.
base = '' if path.is_absolute() else accessor.getcwd()
return _resolve(base, str(path)) or sep
def is_reserved(self, parts):
return False
Expand Down
Expand Up
@@ -424,6 +345,8 @@ def group(self, path):
expanduser = staticmethod(os.path.expanduser)
realpath = staticmethod(os.path.realpath)
Comment thread
barneygale marked this conversation as resolved.
Resolved
Uh oh!
There was an error while loading. Please reload this page.
_normal_accessor = _NormalAccessor()
Expand Down
Expand Up
@@ -1132,15 +1055,27 @@ def resolve(self, strict=False):
normalizing it (for example turning slashes into backslashes under
Windows).
"""
s = self._flavour.resolve(self, strict=strict)
if s is None:
# No symlink resolution => for consistency, raise an error if
# the path doesn't exist or is forbidden
self.stat()
s = str(self.absolute())
# Now we have no symlinks in the path, it's safe to normalize it.
normed = self._flavour.pathmod.normpath(s)
return self._from_parts((normed,))
def check_eloop(e):
winerror = getattr(e, 'winerror', 0)
if e.errno == ELOOP or winerror == _WINERROR_CANT_RESOLVE_FILENAME:
raise RuntimeError("Symlink loop from %r" % e.filename)
try:
s = self._accessor.realpath(self, strict=strict)
except OSError as e:
check_eloop(e)
raise
p = self._from_parts((s,))
# In non-strict mode, realpath() doesn't raise on symlink loops.
# Ensure we get an exception by calling stat()
if not strict:
try:
p.stat()
except OSError as e:
check_eloop(e)
return p
def stat(self, *, follow_symlinks=True):
"""
Expand Down
Original file line number
Diff line number
Diff line change
Expand Up
@@ -387,16 +387,16 @@ def abspath(path):
# Return a canonical path (i.e. the absolute location of a file on the
# filesystem).
def realpath(filename):
def realpath(filename, *, strict=False ):
"""Return the canonical path of the specified filename, eliminating any
symbolic links encountered in the path."""
filename = os.fspath(filename)
path, ok = _joinrealpath(filename[:0], filename, {})
path, ok = _joinrealpath(filename[:0], filename, strict, {})
return abspath(path)
# Join two paths, normalizing and eliminating any symbolic links
# encountered in the second path.
def _joinrealpath(path, rest, seen):
def _joinrealpath(path, rest, strict, seen):
if isinstance(path, bytes):
sep = b'/'
curdir = b'.'
Expand Down
Expand Up
@@ -425,7 +425,15 @@ def _joinrealpath(path, rest, seen):
path = pardir
continue
newpath = join(path, name)
if not islink(newpath):
try:
st = os.lstat(newpath)
except OSError:
if strict:
raise
is_link = False
else:
is_link = stat.S_ISLNK(st.st_mode)
if not is_link:
path = newpath
continue
# Resolve the symbolic link
Expand All
@@ -436,10 +444,14 @@ def _joinrealpath(path, rest, seen):
# use cached value
continue
# The symlink is not resolved, so we must have a symlink loop.
# Return already resolved part + rest of the path unchanged.
return join(newpath, rest), False
if strict:
# Raise OSError(errno.ELOOP)
os.stat(newpath)
else:
# Return already resolved part + rest of the path unchanged.
return join(newpath, rest), False
seen[newpath] = None # not resolved symlink
path, ok = _joinrealpath(path, os.readlink(newpath), seen)
path, ok = _joinrealpath(path, os.readlink(newpath), strict, seen)
if not ok:
return join(path, rest), False
seen[newpath] = path # resolved symlink
Expand Down
Original file line number
Diff line number
Diff line change
Expand Up
@@ -269,6 +269,17 @@ def test_realpath_basic(self):
self.assertPathEqual(ntpath.realpath(os.fsencode(ABSTFN + "1")),
os.fsencode(ABSTFN))
@os_helper.skip_unless_symlink
@unittest.skipUnless(HAVE_GETFINALPATHNAME, 'need _getfinalpathname')
def test_realpath_strict(self):
# Bug #43757: raise FileNotFoundError in strict mode if we encounter
# a path that does not exist.
ABSTFN = ntpath.abspath(os_helper.TESTFN)
os.symlink(ABSTFN + "1", ABSTFN)
self.addCleanup(os_helper.unlink, ABSTFN)
self.assertRaises(FileNotFoundError, ntpath.realpath, ABSTFN, strict=True)
self.assertRaises(FileNotFoundError, ntpath.realpath, ABSTFN + "2", strict=True)
@os_helper.skip_unless_symlink
@unittest.skipUnless(HAVE_GETFINALPATHNAME, 'need _getfinalpathname')
def test_realpath_relative(self):
Expand Down
Expand Up
@@ -340,8 +351,9 @@ def test_realpath_broken_symlinks(self):
@os_helper.skip_unless_symlink
@unittest.skipUnless(HAVE_GETFINALPATHNAME, 'need _getfinalpathname')
def test_realpath_symlink_loops(self):
# Symlink loops are non-deterministic as to which path is returned, but
# it will always be the fully resolved path of one member of the cycle
# Symlink loops in non-strict mode are non-deterministic as to which
# path is returned, but it will always be the fully resolved path of
# one member of the cycle
ABSTFN = ntpath.abspath(os_helper.TESTFN)
self.addCleanup(os_helper.unlink, ABSTFN)
self.addCleanup(os_helper.unlink, ABSTFN + "1")
Expand Down
Expand Up
@@ -383,6 +395,50 @@ def test_realpath_symlink_loops(self):
# Test using relative path as well.
self.assertPathEqual(ntpath.realpath(ntpath.basename(ABSTFN)), ABSTFN)
@os_helper.skip_unless_symlink
@unittest.skipUnless(HAVE_GETFINALPATHNAME, 'need _getfinalpathname')
def test_realpath_symlink_loops_strict(self):
# Symlink loops raise OSError in strict mode
ABSTFN = ntpath.abspath(os_helper.TESTFN)
self.addCleanup(os_helper.unlink, ABSTFN)
self.addCleanup(os_helper.unlink, ABSTFN + "1")
self.addCleanup(os_helper.unlink, ABSTFN + "2")
self.addCleanup(os_helper.unlink, ABSTFN + "y")
self.addCleanup(os_helper.unlink, ABSTFN + "c")
self.addCleanup(os_helper.unlink, ABSTFN + "a")
os.symlink(ABSTFN, ABSTFN)
self.assertRaises(OSError, ntpath.realpath, ABSTFN, strict=True)
os.symlink(ABSTFN + "1", ABSTFN + "2")
os.symlink(ABSTFN + "2", ABSTFN + "1")
self.assertRaises(OSError, ntpath.realpath, ABSTFN + "1", strict=True)
self.assertRaises(OSError, ntpath.realpath, ABSTFN + "2", strict=True)
self.assertRaises(OSError, ntpath.realpath, ABSTFN + "1\\x", strict=True)
# Windows eliminates '..' components before resolving links, so the
# following call is not expected to raise.
self.assertPathEqual(ntpath.realpath(ABSTFN + "1\\..", strict=True),
ntpath.dirname(ABSTFN))
self.assertRaises(OSError, ntpath.realpath, ABSTFN + "1\\..\\x", strict=True)
os.symlink(ABSTFN + "x", ABSTFN + "y")
self.assertRaises(OSError, ntpath.realpath, ABSTFN + "1\\..\\"
+ ntpath.basename(ABSTFN) + "y",
strict=True)
self.assertRaises(OSError, ntpath.realpath,
ABSTFN + "1\\..\\" + ntpath.basename(ABSTFN) + "1",
strict=True)
os.symlink(ntpath.basename(ABSTFN) + "a\\b", ABSTFN + "a")
self.assertRaises(OSError, ntpath.realpath, ABSTFN + "a", strict=True)
os.symlink("..\\" + ntpath.basename(ntpath.dirname(ABSTFN))
+ "\\" + ntpath.basename(ABSTFN) + "c", ABSTFN + "c")
self.assertRaises(OSError, ntpath.realpath, ABSTFN + "c", strict=True)
# Test using relative path as well.
self.assertRaises(OSError, ntpath.realpath, ntpath.basename(ABSTFN),
strict=True)
@os_helper.skip_unless_symlink
@unittest.skipUnless(HAVE_GETFINALPATHNAME, 'need _getfinalpathname')
def test_realpath_symlink_prefix(self):
Expand Down
Loading
Oops, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
Loading
Oops, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.