GeezerVisonThe World Wide Web, restored to 800 pixels of glory. | ||||||||||||||||||||||||||||||||||||
| Home|About|Gaming cabinet|Archives|How it works|Starter sites | ||||||||||||||||||||||||||||||||||||
GeezerVison is showing github.com — gh-99726: Add 'fast' argument to os.[l]stat for faster calculation by zooba · Pull Request #99727 · python/cpython Conversation
|
|
I'm not yet sold on the feature. We already have a faster stat combined with directory scanning. |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
| object. | ||
|
|
||
| As of Python 3.3, this is equivalent to ``os.stat(fd)``. | ||
| As of Python 3.3, this is equivalent to ``os.stat(fd, fast=fast)``. |
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
| As of Python 3.3, this is equivalent to ``os.stat(fd, fast=fast)``. | |
| As of Python 3.3, this is equivalent to ``os.stat(fd, fast=False)``. |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
😆 whoops
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
Wait, no, this is correct. There's a fast argument to this function, which gets passed through (it just doesn't happen to make any difference right now, but that's why fast doesn't guarantee it'll leave out any information. Sometimes we have to take the slow path regardless, and that's the case here.)
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
oh, right, I missed that :) Never mind then
Sorry, something went wrong.
There was an error while loading. Please reload this page.
|
I'm wondering if something in the spirit of |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
| if (PathCchSkipRoot(path, &rootEnd) || | ||
| wcsncpy_s(rootBuffer, MAX_PATH, path, (rootEnd - path))) { | ||
| /* No root for the path, so let it use the current volume */ | ||
| root = NULL; | ||
| } |
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
This is problematic. Instead, open a handle via CreateFileW() and call GetVolumeInformationByHandleW(). It's no more expensive than calling GetVolumeInformationW(), which also has to open a path, and it's reliable. That said, we're only here because it isn't a 'fast' stat, so we may as well take the slow path from the start.
PathCchSkipRoot() only supports backslash as the path separator. That's an irrelevant problem, however, considering the fundamental problem that the path may contain mount points, symlinks, and other reparse points. In this case the drive/device of the opened path is probably irrelevant to the file.
GetVolumeInformationW() will open any path that has a trailing slash. However, after opening the path it queries the FileNameInformation to verify that the final path ends in a backslash, i.e. that a filesystem root directory was opened. The problem is finding the relevant root directory in the path without resolving the final path. GetVolumePathNameW() is supposed to handle this, but I find that it's buggy with substitute drives, for which it returns ridiculous results. It also returns the wrong result for a symlink to a UNC path. And it's more work than simply opening the path.
Actually, the requirement for a root directory is irrelevant to the underlying system call that queries the FileFsVolumeInformation (volume name and serial number) and FileFsAttributeInformation (filesystem name, flags, and max component length). Any open file on the volume is sufficient. Hence, just open the path and call GetVolumeInformationByHandleW().
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
Yes, I know all of this. I'm still having a private discussion with the Windows team about optimising GetVolumeInformation or adding VSN into the stat structure, either of which would let us bypass this extra step.
Right now, if someone passes fast=False, we could always just go back to the original slow path. But in the future, this slow path could get drastically faster without us having to change anything, which means automatic "backports". Right now it's about the same.
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
Yes, I know all of this.
Okay, I just felt it was worth discussing since the result of win32_xstat_get_st_dev() as written would often be incorrect.
optimising GetVolumeInformation
They really should fix GetVolumeInformationW() to work consistently with GetVolumeInformationByHandleW(). All of the loops it jumps through to require a root directory are pointless and frustrating.
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
That's an irrelevant problem, however, considering the fundamental problem that the path may contain mount points, symlinks, and other reparse points.
I think we can probably define this as an edge case: "lstat() on Windows when passed a path including junctions or non-symlink reparse points that direct to another drive are ignored when getting st_dev. The device matching the start of the path will be returned, even if the reparse points mean the file is on another drive."
It's also possible that if the VSN gets added into the stat data before Windows releases, that we won't need this path at all.
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was an error while loading. Please reload this page.
Co-authored-by: Eryk Sun <[email protected]>
There was an error while loading. Please reload this page.
Yeah, this is the reason I went with that API first (should be in a previous commit). I found it felt clunky compared to Guido is right that we already have a fast stat (for |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
Ah, now we're on to something. A new function that returns a custom struct seems better than a flag to |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
The new function is |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was an error while loading. Please reload this page.
Successfully merging this pull request may close these issues.
There was an error while loading. Please reload this page.