GeezerVison is showing github.com — os.link(..., follow_symlinks=True) broken on Linux · Issue #81793 · python/cpython
os.link(..., follow_symlinks=True) broken on Linux #81793Description
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
assignee = None
closed_at = None
created_at = <Date 2019-07-17.23:04:55.849>
labels = ['extension-modules', 'type-bug', '3.8', '3.9', '3.10']
title = 'os.link(..., follow_symlinks=True) broken on Linux'
updated_at = <Date 2021-03-23.17:43:01.277>
user = 'https://github.com/jo-he' bugs.python.org fields:
activity = <Date 2021-03-23.17:43:01.277>
actor = 'takluyver'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = ['Extension Modules']
creation = <Date 2019-07-17.23:04:55.849>
creator = 'jo-he'
dependencies = []
files = []
hgrepos = []
issue_num = 37612
keywords = ['patch']
message_count = 4.0
messages = ['348086', '348103', '348110', '348111']
nosy_count = 5.0
nosy_names = ['larry', 'takluyver', 'serhiy.storchaka', 'eryksun', 'jo-he']
pr_nums = ['14843', '24997']
priority = 'normal'
resolution = None
stage = 'patch review'
status = 'open'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue37612'
versions = ['Python 3.8', 'Python 3.9', 'Python 3.10'] Linked PRs
Reacted by Dmitry Tantsur Activityjo-he commented on Jul 17, 2019 MannequinAuthor More actions Regarding link() POSIX states (https://pubs.opengroup.org/onlinepubs/9699919799/functions/link.html):
"If path1 names a symbolic link, it is implementation-defined whether link() follows the symbolic link, or creates a new link to the symbolic link itself."
In Linux, link() does _not_ follow symlinks (http://man7.org/linux/man-pages/man2/link.2.html):
"By default, linkat(), does not dereference oldpath if it is a symbolic link (like link())."
But Python 3 assumes the opposite to be always the case:
...which suits e.g. NetBSD (https://netbsd.gw.com/cgi-bin/man-cgi?link+2):
"When operating on a symlink, link() resolves the symlink and creates a hard link on the target."
Therefore, I recommend to always call linkat(), if the platform provides it. That's the modern superset of link() with clearly defined behavior.
Here are some commands to reproduce the issue on Linux (should hard link 'file' -> 'link', but tries '/tmp/symlink' -> 'link'):
~$ : >file
~$ ln -s "$PWD/file" /tmp/symlink
~$ strace -e link,linkat python -c 'import os; os.link("/tmp/symlink", "link", follow_symlinks=True)'
link("/tmp/symlink", "link") = -1 EXDEV (Cross-device link)
Traceback (most recent call last):
File "<string>", line 1, in <module>
OSError: [Errno 18] Cross-device link: '/tmp/symlink' -> 'link'
+++ exited with 1 +++For comparison, calling linkat() without AT_SYMLINK_FOLLOW results in the same error:
~$ strace -e link,linkat python -c 'import os; os.link("/tmp/symlink", "link", follow_symlinks=False)'
linkat(AT_FDCWD, "/tmp/symlink", AT_FDCWD, "link", 0) = -1 EXDEV (Cross-device link)
Traceback (most recent call last):
File "<string>", line 1, in <module>
OSError: [Errno 18] Cross-device link: '/tmp/symlink' -> 'link'
+++ exited with 1 +++Currently, the only way to call linkat() with AT_SYMLINK_FOLLOW from Python, is to provide a directory file descriptor != AT_FDCWD:
~$ strace -e link,linkat python -c 'import os; d=os.open(".", 0); os.link("/tmp/symlink", "link", dst_dir_fd=d, follow_symlinks=True)'
linkat(AT_FDCWD, "/tmp/symlink", 3, "link", AT_SYMLINK_FOLLOW) = 0
+++ exited with 0 +++ serhiy-storchaka commented on Jul 18, 2019 Not always linkat() can be used instead of link(). But on Windows (where there is no linkat()) os.link() creates a new link to the symbolic link itself. This is yet one argument for making follow_symlinks=False by default and changing the default behavior of os.link() on NetBSD. eryksun commented on Jul 18, 2019
on Windows (where there is no linkat()) os.link() creates a new link
to the symbolic link itself.
Yes, CreateHardLinkW opens the source file by calling NtOpenFile with the option FILE_OPEN_REPARSE_POINT. So the behavior is follow_symlinks=False.
Note, however, that Windows has distinct file and directory reparse points, so we can't hardlink to a directory symlink, or any other type of directory reparse point such as a junction mountpoint. In Unix, follow_symlinks=False (if implemented) allows creating a hardlink to a symlink that targets a directory.
Also, I noticed that I can pass follow_symlinks=False in Windows, but this should raise NotImplementedError. It's supposed to be checked via follow_symlinks_specified(). jo-he commented on Jul 18, 2019 MannequinAuthor More actions The problem that POSIX does not define the behavior of link() regarding symlinks (and that Unix implementations differ indeed), is independent from Python's os.link() defaults.
Since it makes no sense to call link(), when linkat() is available, I propose this change:
--- a/Modules/posixmodule.c
+++ b/Modules/posixmodule.c
@@ -3512,15 +3512,11 @@ os_link_impl(PyObject *module, path_t *src, path_t *dst, int src_dir_fd,
#else
Py_BEGIN_ALLOW_THREADS
#ifdef HAVE_LINKAT
- if ((src_dir_fd != DEFAULT_DIR_FD) ||
- (dst_dir_fd != DEFAULT_DIR_FD) ||
- (!follow_symlinks))
- result = linkat(src_dir_fd, src->narrow,
- dst_dir_fd, dst->narrow,
- follow_symlinks ? AT_SYMLINK_FOLLOW : 0);
- else
+ result = linkat(src_dir_fd, src->narrow, dst_dir_fd, dst->narrow,
+ follow_symlinks ? AT_SYMLINK_FOLLOW : 0);
+#else
+ result = link(src->narrow, dst->narrow);
#endif /* HAVE_LINKAT */
- result = link(src->narrow, dst->narrow);
Py_END_ALLOW_THREADSThis fix also simplifies the code, and should be safe for back-porting.
Whether Python's defaults should be changed regarding Windows and those (mostly obsolete) Unix platforms that do not provide linkat(), is another question.
MetadataMetadataDevelopmentNo branches or pull requestsIssue actions
|