GeezerVison is showing github.com — shutil.which() can return non-executable path starting with 3.12 on Windows · Issue #127001 · python/cpython
shutil.which() can return non-executable path starting with 3.12 on Windows #127001DescriptionBug report
Bug description:
Starting with Python 3.12 if a file is in PATH that does not end with PATHEXT, but happens to be named the same as the searched command it is returned instead of the real command later in PATH which includes PATHEXT.
# Assume:
# PATH=C:\foo;C:\WINDOWS\system32
# "C:\foo\cmd" exists
# "C:\WINDOWS\system32\cmd.exe" exists
import shutil
print(shutil.which('cmd'))
# Actual: C:\foo\cmd
# Expected: C:\WINDOWS\system32\cmd.exe (3.11 result)I have verified that this change was introduced with #103179 and reverting it fixes the issue.
@csm10495 ^
Downstream context: mesonbuild/meson#13886
CPython versions tested on:
3.12
Operating systems tested on:
Windows
Linked PRs
Activityzooba commented on Nov 19, 2024 Reverting the entire change is clearly not going to happen - the logic was incorrect before, and apparently is still incorrect, so let's figure out what correct logic for handling PATHEXT would be. Reacted by Christoph Reiter serhiy-storchaka commented on Nov 19, 2024 serhiy-storchaka commented on Nov 19, 2024 Ah, it still checks for file without extension. lazka commented on Nov 19, 2024 ContributorAuthor More actions
Should not it be fixed by #109590?
That only handles the "in the same directory" case, and not in different directories, from what I see.
But, that issue mentioned Get-Command cmd in powershell, which I now tried, and that also returns the wrong path as "CommandType=Application" here, even though the docs say that it only returns files ending with PATHEXT (https://learn.microsoft.com/en-us/powershell/module/microsoft.powershell.core/get-command?view=powershell-7.4#-commandtype) so I don't know what's "right" anymore :)
PS C:\Users\user> echo $env:PATH
C:\msys64\usr\bin;C:\Windows\System32;C:\Windows
PS C:\Users\user> echo $env:PATHEXT
.COM;.EXE;.BAT;.CMD;.VBS;.VBE;.JS;.JSE;.WSF;.WSH;.MSC;.CPL
PS C:\Users\user> Get-Command cmd
CommandType Name Version Source
----------- ---- ------- ------
Application cmd 0.0.0.0 C:\msys64\usr\bin\cmd
PS C:\Users\user> Get-Command -All cmd
CommandType Name Version Source
----------- ---- ------- ------
Application cmd 0.0.0.0 C:\msys64\usr\bin\cmd
Application cmd.exe 10.0.2610… C:\Windows\System32\cmd.exe zooba commented on Nov 19, 2024 The right behaviour ought to be "whatever CreateProcess("cmd") would run". Terminals all layer their own logic on top, but Python tries to generally stick to the closest public API that implements the functionality.
(One explicit exception we make to CreateProcess behaviour is that we don't search the application directory first with which, because if we did, it wouldn't be useful. By skipping that somewhat strange edge case, it is possible to use which to override the application directory. This mainly only matters for python.exe - CreateProcess("python") will always launch sys._base_executable due to how the API works, but CreateProcess(shutil.which("python")) follows PATH, which is likely what was intended if a user provided the command name.) lazka commented on Nov 19, 2024 ContributorAuthor More actions
The right behaviour ought to be "whatever CreateProcess("cmd") would run". Terminals all layer their own logic on top, but Python tries to generally stick to the closest public API that implements the functionality.
Also CreateProcess prefers "C:\Windows\System32" over PATH, like curl/tar/bash that are in there, which is another can of worms :(
Since my initial presented scenario occurs in MSYS2, and it's not quite clear why we have that "cmd" bash script in PATH in the first place, I'm trying to get rid of it instead now :/ csm10495 commented on Nov 21, 2024 Ouch. Sorry for the regression folks. I wish this was less complicated than it is. I see @serhiy-storchaka has already started on a fix. Thanks all. serhiy-storchaka commented on Nov 22, 2024 There was other bug introduced in #109995: full match did not work for files with multcomponent extension in PATHEXT, (e.g. .foo.bar).
Other corner case was only discussed in comments: a dot (.) in PATHEXT. lazka commented on Nov 22, 2024 ContributorAuthor More actions I can confirm that this fixes things for my use case. Thanks everyone! eabase commented on Dec 14, 2024 This got to be the best issue number in all of internet!
127.0.0.1 Reacted by Christoph Reiter and Kagami Sascha Rosylight
MetadataMetadataDevelopmentNo branches or pull requestsIssue actions
|