GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — gh-109590 - Update shutil.which on win32 to only give back PATHEXT matches by default by csm10495 · Pull Request #109995 · python/cpython

gh-109590 - Update shutil.which on win32 to only give back PATHEXT matches by default #109995

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
zooba merged 4 commits into python:main from csm10495:shutil_which_no_extv3
Oct 2, 2023

Uh oh!

There was an error while loading. Please reload this page.
Merged

gh-109590 - Update shutil.which on win32 to only give back PATHEXT matches by default #109995

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
e13eefe
gh-109590 - Update shutil.which on win32 to only give back PATHEXT ma…
csm10495 Sep 28, 2023
f5a736b
Apply suggestions from code review
csm10495 Sep 28, 2023
d93e51f
Remove casefold since it doesn't exist on bytes (and the code needs t…
csm10495 Sep 29, 2023
4bed27c
Behavior updated from pr comments
csm10495 Sep 30, 2023
File filter
Filter by extension
Clear filters
Conversations
Failed to load comments.
Loading

Uh oh!

There was an error while loading. Please reload this page.
Jump to
Jump to file
Failed to load files.
Loading

Uh oh!

There was an error while loading. Please reload this page.
Diff view
Diff view

There are no files selected for viewing

6 changes: 6 additions & 0 deletions 6 Doc/library/shutil.rst
Show comments
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -476,6 +476,12 @@ Directory and files operations
or ends with an extension that is in ``PATHEXT``; and filenames that
have no extension can now be found.

.. versionchanged:: 3.12.1
On Windows, if *mode* includes ``os.X_OK``, executables with an
extension in ``PATHEXT`` will be preferred over executables without a
matching extension.
This brings behavior closer to that of Python 3.11.

.. exception:: Error

This exception collects exceptions that are raised during a multi-file
Expand Down
12 changes: 10 additions & 2 deletions 12 Lib/shutil.py
Show comments
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1554,8 +1554,16 @@ def which(cmd, mode=os.F_OK | os.X_OK, path=None):
if use_bytes:
pathext = [os.fsencode(ext) for ext in pathext]

# Always try checking the originally given cmd, if it doesn't match, try pathext
files = [cmd] + [cmd + ext for ext in pathext]
files = ([cmd] + [cmd + ext for ext in pathext])

# gh-109590. If we are looking for an executable, we need to look
# for a PATHEXT match. The first cmd is the direct match
# (e.g. python.exe instead of python)
# Check that direct match first if and only if the extension is in PATHEXT
# Otherwise check it last
suffix = os.path.splitext(files[0])[1].upper()
if mode & os.X_OK and not any(suffix == ext.upper() for ext in pathext):
files.append(files.pop(0))
else:
# On other platforms you don't have things like PATHEXT to tell you
# what file suffixes are executable, so just pass on cmd as-is.
Expand Down
82 changes: 72 additions & 10 deletions 82 Lib/test/test_shutil.py
Show comments
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2067,6 +2067,14 @@ def setUp(self):
self.curdir = os.curdir
self.ext = ".EXE"

def to_text_type(self, s):
'''
In this class we're testing with str, so convert s to a str
'''
if isinstance(s, bytes):
return s.decode()
return s

def test_basic(self):
# Given an EXE in a directory, it should be returned.
rv = shutil.which(self.file, path=self.dir)
Expand Down Expand Up @@ -2254,9 +2262,9 @@ def test_empty_path_no_PATH(self):

@unittest.skipUnless(sys.platform == "win32", 'test specific to Windows')
def test_pathext(self):
ext = ".xyz"
ext = self.to_text_type(".xyz")
temp_filexyz = tempfile.NamedTemporaryFile(dir=self.temp_dir,
prefix="Tmp2", suffix=ext)
prefix=self.to_text_type("Tmp2"), suffix=ext)
os.chmod(temp_filexyz.name, stat.S_IXUSR)
self.addCleanup(temp_filexyz.close)

Expand All @@ -2265,38 +2273,39 @@ def test_pathext(self):
program = os.path.splitext(program)[0]

with os_helper.EnvironmentVarGuard() as env:
env['PATHEXT'] = ext
env['PATHEXT'] = ext if isinstance(ext, str) else ext.decode()
rv = shutil.which(program, path=self.temp_dir)
self.assertEqual(rv, temp_filexyz.name)

# Issue 40592: See https://bugs.python.org/issue40592
@unittest.skipUnless(sys.platform == "win32", 'test specific to Windows')
def test_pathext_with_empty_str(self):
ext = ".xyz"
ext = self.to_text_type(".xyz")
temp_filexyz = tempfile.NamedTemporaryFile(dir=self.temp_dir,
prefix="Tmp2", suffix=ext)
prefix=self.to_text_type("Tmp2"), suffix=ext)
self.addCleanup(temp_filexyz.close)

# strip path and extension
program = os.path.basename(temp_filexyz.name)
program = os.path.splitext(program)[0]

with os_helper.EnvironmentVarGuard() as env:
env['PATHEXT'] = f"{ext};" # note the ;
env['PATHEXT'] = f"{ext if isinstance(ext, str) else ext.decode()};" # note the ;
rv = shutil.which(program, path=self.temp_dir)
self.assertEqual(rv, temp_filexyz.name)

# See GH-75586
@unittest.skipUnless(sys.platform == "win32", 'test specific to Windows')
def test_pathext_applied_on_files_in_path(self):
with os_helper.EnvironmentVarGuard() as env:
env["PATH"] = self.temp_dir
env["PATH"] = self.temp_dir if isinstance(self.temp_dir, str) else self.temp_dir.decode()
env["PATHEXT"] = ".test"

test_path = pathlib.Path(self.temp_dir) / "test_program.test"
test_path.touch(mode=0o755)
test_path = os.path.join(self.temp_dir, self.to_text_type("test_program.test"))
open(test_path, 'w').close()
os.chmod(test_path, 0o755)

self.assertEqual(shutil.which("test_program"), str(test_path))
self.assertEqual(shutil.which(self.to_text_type("test_program")), test_path)

# See GH-75586
@unittest.skipUnless(sys.platform == "win32", 'test specific to Windows')
Expand All @@ -2312,16 +2321,69 @@ def test_win_path_needs_curdir(self):
self.assertFalse(shutil._win_path_needs_curdir('dontcare', os.X_OK))
need_curdir_mock.assert_called_once_with('dontcare')

# See GH-109590
@unittest.skipUnless(sys.platform == "win32", 'test specific to Windows')
def test_pathext_preferred_for_execute(self):
with os_helper.EnvironmentVarGuard() as env:
env["PATH"] = self.temp_dir if isinstance(self.temp_dir, str) else self.temp_dir.decode()
env["PATHEXT"] = ".test"

exe = os.path.join(self.temp_dir, self.to_text_type("test.exe"))
open(exe, 'w').close()
os.chmod(exe, 0o755)

# default behavior allows a direct match if nothing in PATHEXT matches
self.assertEqual(shutil.which(self.to_text_type("test.exe")), exe)

dot_test = os.path.join(self.temp_dir, self.to_text_type("test.exe.test"))
open(dot_test, 'w').close()
os.chmod(dot_test, 0o755)

# now we have a PATHEXT match, so it take precedence
self.assertEqual(shutil.which(self.to_text_type("test.exe")), dot_test)

# but if we don't use os.X_OK we don't change the order based off PATHEXT
# and therefore get the direct match.
self.assertEqual(shutil.which(self.to_text_type("test.exe"), mode=os.F_OK), exe)

# See GH-109590
@unittest.skipUnless(sys.platform == "win32", 'test specific to Windows')
def test_pathext_given_extension_preferred(self):
with os_helper.EnvironmentVarGuard() as env:
env["PATH"] = self.temp_dir if isinstance(self.temp_dir, str) else self.temp_dir.decode()
env["PATHEXT"] = ".exe2;.exe"

exe = os.path.join(self.temp_dir, self.to_text_type("test.exe"))
open(exe, 'w').close()
os.chmod(exe, 0o755)

exe2 = os.path.join(self.temp_dir, self.to_text_type("test.exe2"))
open(exe2, 'w').close()
os.chmod(exe2, 0o755)

# even though .exe2 is preferred in PATHEXT, we matched directly to test.exe
self.assertEqual(shutil.which(self.to_text_type("test.exe")), exe)
self.assertEqual(shutil.which(self.to_text_type("test")), exe2)


class TestWhichBytes(TestWhich):
def setUp(self):
TestWhich.setUp(self)
self.dir = os.fsencode(self.dir)
self.file = os.fsencode(self.file)
self.temp_file.name = os.fsencode(self.temp_file.name)
self.temp_dir = os.fsencode(self.temp_dir)
self.curdir = os.fsencode(self.curdir)
self.ext = os.fsencode(self.ext)

def to_text_type(self, s):
'''
In this class we're testing with bytes, so convert s to a bytes
'''
if isinstance(s, str):
return s.encode()
return s


class TestMove(BaseTest, unittest.TestCase):

Expand Down
3 changes: 3 additions & 0 deletions 3 Misc/NEWS.d/next/Library/2023-09-24-06-04-14.gh-issue-109590.9EMofC.rst
Show comments
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
:func:`shutil.which` will prefer files with an extension in ``PATHEXT`` if the given mode includes ``os.X_OK`` on win32.
If no ``PATHEXT`` match is found, a file without an extension in ``PATHEXT`` can be returned.
This change will have :func:`shutil.which` act more similarly to previous behavior in Python 3.11.
Add this suggestion to a batch that can be applied as a single commit. This suggestion is invalid because no changes were made to the code. Suggestions cannot be applied while the pull request is closed. Suggestions cannot be applied while viewing a subset of changes. Only one suggestion per line can be applied in a batch. Add this suggestion to a batch that can be applied as a single commit. Applying suggestions on deleted lines is not supported. You must change the existing code in this line in order to create a valid suggestion. Outdated suggestions cannot be applied. This suggestion has been applied or marked resolved. Suggestions cannot be applied from pending reviews. Suggestions cannot be applied on multi-line comments. Suggestions cannot be applied while the pull request is queued to merge. Suggestion cannot be applied right now. Please check back later.
You are visitor 01402503 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.