GeezerVison is showing github.com — Add case_sensitive argument to `pathlib.PurePath.match()` · Issue #104484 · python/cpython
Add case_sensitive argument to pathlib.PurePath.match() #104484DescriptionIn #81079 we added a case_sensitive argument to pathlib.Path.glob() and rglob(). It would be good to have this functionality available in the closely-related PurePath.match() method. The rationale is much the same: this argument is useful when dealing with case-sensitive filesystems on Windows, and case-insensitive filesystems on Posix. Also, it would allow this method to be used from glob() to implement recursive globbing efficiently, which is important for #102613.
Linked PRs
Activitybarneygale commented on May 14, 2023 ContributorAuthor More actions I'd like to land #101398 before I start work on this. thirumurugan-git commented on May 16, 2023 Hi @barneygale,
I believe it's a relatively simple problem that I can help fix. May I have the opportunity to work on it? thirumurugan-git commented on May 16, 2023 barneygale commented on May 18, 2023 ContributorAuthor More actions eryksun commented on May 18, 2023 It should be noted in the docs that Path.match() has changed in 3.12 to use re.IGNORECASE instead of normalizing the path and pattern with os.path.normcase(). On Windows, os.path.normcase() is based on the system's lowercase mapping, while re.IGNORECASE is based on Python's lowercase mapping. The two differ for 200 BMP characters and 260 non-BMP characters. Thus some patterns may match a path in 3.11 but not in 3.12, and vice versa. barneygale commented on May 19, 2023 ContributorAuthor More actions It used str.lower() in 3.11:
|
def casefold_parts(self, parts): |
|
return [p.lower() for p in parts] |
eryksun commented on May 19, 2023 I lost track of changes in 3.12. I thought it always used normcase() for case folding in match() as well as the hash and comparison methods. Instead, this issue restores the use of str.lower() in match(). However, the hash and comparison methods will be inconsistent on Windows because they still use _str_normcase based on os.path.normcase(). Thus match() could be true while __eq__() is false, or vice versa, given component names that contain any of the 460 characters for which str.lower() and ntpath.normcase() are different. barneygale commented on May 24, 2023 ContributorAuthor More actions Here's a table showing how __eq__(), match() and glob() are implemented in 3.11 and 3.12:
|
3.11 |
3.12 |
__eq__() |
str.lower() |
os.path.normcase() |
match() |
str.lower() |
re.IGNORECASE |
glob() |
re.IGNORECASE |
re.IGNORECASE |
You can see that things were already inconsistent in 3.11. For 3.12, we've:
- Brought
__eq__() more in line with Windows' filename comparisons
- Brought
match() in line with its sister method glob()
I wouldn't mind changing __eq__() and friends back to using lower() if you thought that best, @eryksun! eryksun commented on May 24, 2023 @barneygale
AFAIK, str.lower() and re.IGNORECASE are consistent. The inconsistency is with ntpath.normcase() on Windows. It's probably for the best if the pure comparison methods revert to using str.lower() for the sake of consistency, not only with glob() and match(), but also with using PureWindowsPath on POSIX. Maybe platform-dependent comparisons could be implemented on Path.
On Windows, ntpath.normcase() is based on LCMapStringEx(). It turns out that this function implements a case mapping for some non-BMP characters. WinAPI CompareStringOrdinal(), on the other hand, has no case mapping for non-BMP characters, which is consistent with Microsoft's filesystems. Thus I'd prefer for a platform-dependent comparison to use CompareStringOrdinal() instead of LCMapStringEx().
Given that any tree in a Linux ext4 filesystem can be made case insensitive, and any tree in a Windows NTFS filesystem can be made case sensitive (even a non-empty directory if the registry setting allows it), I'd prefer for a platform-dependent comparison to support partially realized behavior in this regard. Maybe the Path constructor could allow overriding the default case sensitivity for all of a path or a subtree of a path. barneygale commented on May 25, 2023 ContributorAuthor More actions Thank you Eryk. I've logged a new issue here: #104947. Will get it fixed pronto!
AFAIK, str.lower() and re.IGNORECASE are consistent.
Seemingly not in all circumstances:
>>> s = 'ı'
>>> t = 'i'
>>> s.lower() == t.lower()
False
>>> import re
>>> re.match(s, t, flags=re.IGNORECASE)
<re.Match object; span=(0, 1), match='i'>
Given that any tree in a Linux ext4 filesystem can be made case insensitive, and any tree in a Windows NTFS filesystem can be made case sensitive (even a non-empty directory if the registry setting allows it), I'd prefer for a platform-dependent comparison to support partially realized behavior in this regard. Maybe the Path constructor could allow overriding the default case sensitivity for all of a path or a subtree of a path.
This is on my to-do list :) barneygale commented on May 25, 2023 ContributorAuthor More actions Re-resolving this ticket per the above.
MetadataMetadataDevelopmentNo branches or pull requestsIssue actions
|