GeezerVison is showing github.com — '*' matches entire path in fnmatch · Issue #72904 · python/cpython
'*' matches entire path in fnmatch #72904Description
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
assignee = None
closed_at = None
created_at = <Date 2016-11-16.20:12:07.121>
labels = ['library']
title = "'*' matches entire path in fnmatch"
updated_at = <Date 2019-03-31.12:59:09.371>
user = 'https://github.com/decibel' bugs.python.org fields:
activity = <Date 2019-03-31.12:59:09.371>
actor = 'Toon Verstraelen'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = ['Library (Lib)']
creation = <Date 2016-11-16.20:12:07.121>
creator = 'Jim Nasby'
dependencies = []
files = []
hgrepos = []
issue_num = 28718
keywords = []
message_count = 8.0
messages = ['280985', '281017', '281018', '288608', '290624', '307867', '339054', '339256']
nosy_count = 6.0
nosy_names = ['serhiy.storchaka', 'josh.r', 'Jim Nasby', 'Alberto Galera', 'aaron-whitehouse', 'Toon Verstraelen']
pr_nums = []
priority = 'normal'
resolution = None
stage = None
status = 'open'
superseder = None
type = None
url = 'https://bugs.python.org/issue28718'
versions = [] Linked PRs
Activitydecibel commented on Nov 16, 2016 MannequinAuthor More actions A '*' in fnmatch.translate is converted into '.*', which will greedily match directory separators. This doesn't match shell behavior, which is that * will only match file names:
decibel@decina:[14:07]~$ls \~/tmp/*/1|head
ls: /Users/decibel/tmp/*/1: No such file or directory
decibel@decina:[14:07]~$ls \~/tmp/d*/base/1|head
112
From a posix standpoint, this would easily be fixed by using '[^/]*' instead of '.*'. I'm not sure how to make this work cross-platform though.
It's worth noting that some programs (rsync, git) support **, which would correctly translate to '.*'. MojoVampire commented on Nov 17, 2016 Presumably something like:
r'(?:' + r'|'.join({re.escape(os.path.sep), re.escape(os.path.altsep)}) + r')'
would cover it completely. I switched to using non-capturing groups over a character class both to deal with the fact that escaping doesn't work the same way for character classes and to cover the possibility (no idea here) that some terrible OS might have a multicharacter path separator. MojoVampire commented on Nov 17, 2016 Oops, altsep is None, not the empty string when there is only one separator. And I didn't handle inverting the match. Sigh. You get the idea. Hooloovoo commented on Feb 26, 2017 Note that somebody has forked the standard library to implement this:
https://github.com/kianxineki/python-wildcard
This shows that the actual changes would be pretty small (though pywildcard is based on 2.x code and does not handle the cross-platform slashes you have been discussing).
It is also worth noting that the glob standard library:
https://docs.python.org/3.7/library/glob.html
implements a "recursive" option that has similar behaviour (* does not span path separators whereas ** does) and essentially builds this on top of fnmatch for the actual filename matching.
I do not think we can change the default behaviour of fnmatch at this point, but I would like to see this behaviour triggered by an optional argument to the various functions, e.g.:
fnmatch.fnmatch(filename, pattern, glob_asterisks=False)
fnmatch.fnmatchcase(filename, pattern, glob_asterisks=False)
fnmatch.filter(names, pattern, glob_asterisks=False)
fnmatch.translate(pattern, glob_asterisks=False)
In each case, if glob_asterisks (or whatever other name we came up with) is true, the behaviour would match the pywildcard behaviour, i.e.:
** matches everything
* matches in one path level
I look after the glob matching code in duplicity and would like to start using the standard library to do filename matching for us, but we need the above behaviour. I am happy to do the patching if there is a realistic chance of it being accepted. Hooloovoo changed the title [-]'*' matches entire path in fnmatch.translate[/-] [+]'*' matches entire path in fnmatch[/+] on Feb 26, 2017Hooloovoo commented on Mar 27, 2017 AlbertoGalera commented on Dec 8, 2017 I see that they have commented on the lib that I made a few years ago (python-wildcard).
The reason for the creation of that little fork started in this issue:
https://bugs.python.org/issue25734 tovrstra commented on Mar 28, 2019 For consistency with the corresponding feature in the glob function since Python 3.5, I would suggest to add an extra optional argument 'recursive' instead of 'glob_asterisks'. With the default recursive=False, one gets the old behavior, with recursive=True, it can handle the '**' and '*' as in pywildcard.
I realize that with recursive=False, the behavior is not exactly consistent with glob, but I'd still prefer the same name for the optional argument. It is the common terminology for this type of feature. See https://en.wikipedia.org/wiki/Matching_wildcards tovrstra commented on Mar 31, 2019 Just for reference, here are a few more implementations of the same idea, next to pywildcard, sometimes combined with other useful features:
The last one is rather active, with regular releases, last one on March 24, 2019. Reacted by Wim Jeantine-Glenn barneygale commented on Mar 15, 2023 I have an implementation of this for pathlib:
It exploits a simple trick: swapping path separators and newlines, and then matching without setting re.DOTALL. This causes * (and other wildcards) to not match directory separators.
If folks thought it was a good idea, we could instead put the implementation in an fnmatch.globmatch() or glob.match() function. barneygale commented on Jul 2, 2023 Some other differences between fnmatch and glob:
**/ has a special meaning in glob.glob(). You might think it means "any number of characters, followed by one slash". In fact it means "any number of characters, as long as we terminate after a slash". Importantly, this means it can match nothing at all, and so a pattern like /etc/**/**/**/hosts will match /etc/hosts.* as a standalone path segment also has a special meaning. You might think it means "any number of any non-slash characters". In fact it means "at least one non-slash character", i.e. it always consumes precisely one path component. barneygale commented on Aug 13, 2023 wimglenn commented on Sep 25, 2023 barneygale commented on Nov 13, 2023 Addressed in Python 3.13 / cf67ebf / #106703 -- we've added a new glob.translate() function that converts a glob expression into a regular expression. hauntsaninja commented on May 17, 2024 @barneygale minor thing, I noticed that the pat argument is documented as pathname. Should we make the arg positional-only or rename the arg to match the documentation?
MetadataMetadataStatusShow more project fields
DevelopmentNo branches or pull requestsIssue actions
|