GeezerVison is showing github.com — relative symlinks in tarfile.extract broken (windows) · Issue #57911 · python/cpython
relative symlinks in tarfile.extract broken (windows) #57911Description
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
assignee = 'https://github.com/gustaebel'
closed_at = None
created_at = <Date 2012-01-03.16:42:43.116>
labels = ['type-bug', '3.8', '3.9', '3.10', 'library', 'OS-windows']
title = 'relative symlinks in tarfile.extract broken (windows)'
updated_at = <Date 2020-05-30.17:07:34.446>
user = 'https://bugs.python.org/PatrickvonReth' bugs.python.org fields:
activity = <Date 2020-05-30.17:07:34.446>
actor = 'eryksun'
assignee = 'lars.gustaebel'
closed = False
closed_date = None
closer = None
components = ['Library (Lib)', 'Windows']
creation = <Date 2012-01-03.16:42:43.116>
creator = 'Patrick.von.Reth'
dependencies = ['12926']
files = []
hgrepos = []
issue_num = 13702
keywords = []
message_count = 8.0
messages = ['150512', '150671', '150672', '150673', '217989', '218029', '218049', '370391']
nosy_count = 9.0
nosy_names = ['paul.moore', 'lars.gustaebel', 'tim.golden', 'brian.curtin', 'Patrick.von.Reth', 'zach.ware', 'eryksun', 'steve.dower', 'Andreas.G\xc3\xa4er']
pr_nums = []
priority = 'normal'
resolution = None
stage = 'needs patch'
status = 'open'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue13702'
versions = ['Python 3.8', 'Python 3.9', 'Python 3.10'] Linked PRs
ActivityPatrickvonReth commented on Jan 3, 2012 MannequinAuthor More actions when extracting http://www.openssl.org/source/openssl-1.0.0d.tar.gz with python3.2 on windows 7 extraction fails with
File "C:\python32\lib\tarfile.py", line 2175, in extract
set_attrs=set_attrs)
File "C:\python32\lib\tarfile.py", line 2259, in _extract_member
self.makelink(tarinfo, targetpath)
File "C:\python32\lib\tarfile.py", line 2359, in makelink
targetpath)
File "C:\python32\lib\tarfile.py", line 2251, in _extract_member
self.makefile(tarinfo, targetpath)
File "C:\python32\lib\tarfile.py", line 2292, in makefile
target = bltn_open(targetpath, "wb")
IOError: [Errno 22] Invalid argument: 'R:\\tmp\\os\\openssl-1.0.0d\\apps\\md4.c'
the reason is that the symlink is broken
R:\>dir R:\tmp\os\openssl-1.0.0d\apps\md4.c
Volume in drive R has no label.
Volume Serial Number is E8F0-7223
Directory of R:\tmp\os\openssl-1.0.0d\apps
02.01.2012 20:13 <SYMLINK> md4.c [../crypto/md4/md4.c]
it must be backslashes instead of front slashes and that's why python cant access the file the symlink is pointing to. PatrickvonReth changed the title [-]relative symlinks in tarfile.extract broken[/-] [+]relative symlinks in tarfile.extract broken (windows)[/+] on Jan 3, 2012gustaebel commented on Jan 5, 2012 You actually hit two bugs at the same time here: The target of the created symlink was not translated from unix to windows path delimiters and is therefore broken. The second bug is bpo-12926 which leads to the error in TarFile.makefile().
Brian, AFAIK all file-specific functions on windows accept forward slashes in pathnames, right? Has this been discussed in the course of the windows implementation of os.symlink()? I could certainly fix the slash translation in tarfile.py, but may be it's os.symlink() that should been fixed. PatrickvonReth commented on Jan 5, 2012 MannequinAuthor More actions to ignore the bug I also tried dereference=True, but it looks like python3 is ignoring it for extraction.
Is this the normal behavior or just another bug? gustaebel commented on Jan 5, 2012 The dereference option is only used for archive creation, so the contents of the file a symbolic link is pointing to is added instead of the symbolic link itself. AndreasGer commented on May 6, 2014 Is there any progress to the question if the problem should be fixed in os.symlink or in tarfile?
Because this currently seems to break installing source packages that contain symlinks with pip under Windows.
Try: "pip install networkx==1.8.1" for example eryksun commented on May 6, 2014 This should be fixed in os.symlink. The Windows CreateSymbolicLink function can't be relied on to translate slash to backslash. It only normalizes an absolute link, or a path that's relative to the current working directory on a drive (e.g. "R:../crypto") since that's stored as an absolute link.
For example:
>>> os.symlink('C:/Program Files/Python34', 'Python34')
>>> os.system('fsutil reparsepoint query Python34')
Reparse Tag Value : 0xa000000c
Tag value: Microsoft
Tag value: Name Surrogate
Tag value: Symbolic LinkReparse Data Length: 0x00000078
Reparse Data:
0000: 32 00 3a 00 00 00 32 00 00 00 00 00 43 00 3a 00 2.:...2.....C.:.
0010: 2f 00 50 00 72 00 6f 00 67 00 72 00 61 00 6d 00 /.P.r.o.g.r.a.m.
0020: 20 00 46 00 69 00 6c 00 65 00 73 00 2f 00 50 00 .F.i.l.e.s./.P.
0030: 79 00 74 00 68 00 6f 00 6e 00 33 00 34 00 5c 00 y.t.h.o.n.3.4.\.
0040: 3f 00 3f 00 5c 00 43 00 3a 00 5c 00 50 00 72 00 ?.?.\.C.:.\.P.r.
0050: 6f 00 67 00 72 00 61 00 6d 00 20 00 46 00 69 00 o.g.r.a.m. .F.i.
0060: 6c 00 65 00 73 00 5c 00 50 00 79 00 74 00 68 00 l.e.s.\.P.y.t.h.
0070: 6f 00 6e 00 33 00 34 00 o.n.3.4.
The print name uses forward slash, but the NT substitute name uses backslash. In this case, GetFinalPathNameByHandle works fine ("\??" is the NT DosDevices directory in which "C:" is a symbolic link to something like "\Device\HarddiskVolume1"):
>>> print(os.path._getfinalpathname('Python34'))
\\?\C:\Program Files\Python34OTOH, forward slashes aren't translated in a relative link:
>>> os.remove('Python34')
>>> os.symlink('/Program Files/Python34', 'Python34')
>>> os.system('fsutil reparsepoint query Python34')
Reparse Tag Value : 0xa000000c
Tag value: Microsoft
Tag value: Name Surrogate
Tag value: Symbolic LinkReparse Data Length: 0x00000068
Reparse Data:
0000: 2e 00 2e 00 00 00 2e 00 01 00 00 00 2f 00 50 00 ............/.P.
0010: 72 00 6f 00 67 00 72 00 61 00 6d 00 20 00 46 00 r.o.g.r.a.m. .F.
0020: 69 00 6c 00 65 00 73 00 2f 00 50 00 79 00 74 00 i.l.e.s./.P.y.t.
0030: 68 00 6f 00 6e 00 33 00 34 00 2f 00 50 00 72 00 h.o.n.3.4./.P.r.
0040: 6f 00 67 00 72 00 61 00 6d 00 20 00 46 00 69 00 o.g.r.a.m. .F.i.
0050: 6c 00 65 00 73 00 2f 00 50 00 79 00 74 00 68 00 l.e.s./.P.y.t.h.
0060: 6f 00 6e 00 33 00 34 00 o.n.3.4.
In this case GetFinalPathNameByHandle fails because the NT executive doesn't interpret forward slash as a path delimiter:
>>> os.path._getfinalpathname('Python34')
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
OSError: [WinError 123] The filename, directory name, or volume label
syntax is incorrect: 'Python34'I think this is a bug in CreateSymbolicLink, but os.symlink should work around it by first normalizing the target path to use os.sep. tjguk commented on May 7, 2014 eryksun: could you essay a patch? I'd be happy to review & apply it. eryksun commented on May 30, 2020 This is still a problem with WinAPI CreateSymbolicLinkW. It fails to replace slashes with backslashes in the substitute path if it's a relative path, which creates a broken link. As a workaround, os.symlink should replace slashes with backslashes in relative target paths. Except drive-relative targets such as "C:spam" can be ignored, since CreateSymbolicLinkW is forced to normalize them as fully-qualified paths.
Non-UNC rooted paths such as "/Program Files/Python38" are also relative paths. (ntpath.isabs incorrectly classifies them as absolute.) A relative target path gets resolved against the parsed, opened path of the symlink. For example, consider a symlink on a volume at r"Eggs\spam.txt" that targets r"\spam.txt". If the volume is mounted at "W:\\", then accessing r"W:\Eggs\spam.txt" resolves to r"W:\spam.txt". But if the volume is mounted at r"C:\Mount\Work", then accessing r"C:\Mount\Work\Eggs\spam.txt" resolves to r"C:\spam.txt". encukou commented on Jun 29, 2026 The fix introduced a regression; see discussion in #138309. encukou commented on Aug 5, 2026 The regression should be fixed in #151669.
MetadataMetadataStatusShow more project fields
DevelopmentNo branches or pull requestsIssue actions
|