GeezerVison is showing github.com — Proposal: add re.fullmatch() method · Issue #60407 · python/cpython
Proposal: add re.fullmatch() method #60407Description
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
assignee = 'https://github.com/serhiy-storchaka'
closed_at = <Date 2013-11-23.22:46:48.282>
created_at = <Date 2012-10-11.21:30:41.951>
labels = ['expert-regex', 'type-feature']
title = 'Proposal: add re.fullmatch() method'
updated_at = <Date 2013-11-23.22:46:48.282>
user = 'https://github.com/gvanrossum' bugs.python.org fields:
activity = <Date 2013-11-23.22:46:48.282>
actor = 'serhiy.storchaka'
assignee = 'serhiy.storchaka'
closed = True
closed_date = <Date 2013-11-23.22:46:48.282>
closer = 'serhiy.storchaka'
components = ['Regular Expressions']
creation = <Date 2012-10-11.21:30:41.951>
creator = 'gvanrossum'
dependencies = []
files = ['28140', '28955', '32076', '32370']
hgrepos = []
issue_num = 16203
keywords = ['patch']
message_count = 36.0
messages = ['172695', '172696', '172697', '172698', '172704', '172775', '172796', '172816', '172819', '172822', '172844', '172845', '173110', '173113', '173114', '173116', '173121', '173122', '173206', '176498', '177553', '181395', '181451', '181468', '181496', '181552', '181565', '199620', '199661', '199663', '199664', '201340', '203059', '203074', '204106', '204107']
nosy_count = 9.0
nosy_names = ['gvanrossum', 'tim.peters', 'georg.brandl', 'pitrou', 'ezio.melotti', 'mrabarnett', 'python-dev', 'ezberch', 'serhiy.storchaka']
pr_nums = []
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue16203'
versions = ['Python 3.4'] Activitygvanrossum commented on Oct 11, 2012 I've noticed a subtle bug in some of our internal code. Someone wants to ensure that a certain string (e.g. a URL path) matches a certain pattern in its entirety. They use re.match() with a regex ending in $. Fine. Now someone else comes along and modifies the pattern. Somehow the $ gets lost, or the pattern develops a set of toplevel choices that don't all end in $. And now things that have a valid *prefix* suddenly (and unintentionally) start matching.
I think this is a common enough issue and propose a new API: a fullmatch() function and method that work just like the existing match() function and method but also check that the whole input string matches. This can be implemented slightly awkwardly as follows in user code:
def fullmatch(regex, input, flags=0):
m = re.match(regex, input, flags)
if m is not None and m.end() == len(input):
return m
return None(The corresponding method will have to be somewhat more complex because the underlying match() method takes optional pos and endpos arguments.) tim-one commented on Oct 11, 2012 +1. Note that this really can't be done in user-level code. For example, consider matching the pattern
a|ab
against the string
ab
Without being _forced_ to consider the "ab" branch, the regexp will match just the "a" branch. So, e.g., the example code you posted will say "nope, it didn't match (the whole thing)". serhiy-storchaka commented on Oct 11, 2012 What will be with non-greedy qualifiers? Should '.*?' full match any string?
>>> re.match('.*?$', 'abc').group()
'abc'
>>> re.match('.*?', 'abc').group()
''pitrou commented on Oct 11, 2012
Note that this really can't be done in user-level code.
Well, how about:
def fullmatch(regex, input, flags=0):
return re.match("(:?" + regex + ")$", input, flags)tim-one commented on Oct 11, 2012 Antoine, that's certainly the conceptual intent here. Can't say whether your attempt works in all cases. The docs don't guarantee it. For example, if the original regexp started with (?x), the docs explicitly say the effect of (?x) is undefined "if there are non-whitespace characters before the [inline (?x)] flag".
Sure, you could parse the regexp is user code too, and move an initial (?...x...) before your non-capturing group. For that matter, you could write your own regexp engine in user code too ;-)
The point is that it should be easy for the regexp engine to implement the desired functionality - and user attempts to "fake it" have pitfalls (even Guido didn't get it right - LOL ;-) ). mrabarnett commented on Oct 12, 2012 '$' will match at the end of the string or just before the final '\n':
>>> re.match(r'abc$', 'abc\n')
<_sre.SRE_Match object at 0x00F15448> So shouldn't you be using r'\Z' instead?
>>> re.match(r'abc\Z', 'abc')
<_sre.SRE_Match object at 0x00F15410>
>>> re.match(r'abc\Z', 'abc\n')
>>> And what happens if the MULTILINE flag is turned on?
>>> re.match(r'abc$', 'abc\ndef', flags=re.MULTILINE)
<_sre.SRE_Match object at 0x00F15448>
>>> re.match(r'abc\Z', 'abc\ndef', flags=re.MULTILINE)
>>> tim-one commented on Oct 13, 2012 Matthew, Guido wrote "check that the whole input string matches" (or slice if pos and (possibly also) endpos is/are given). So, yes, \Z is more to the point than $ if people want to continue wasting time trying to implement this as a Python-level function ;-)
I don't understand what you're asking about MULTILINE. What's the issue there? Focus on Guido's "whole input string matches", not on his motivational talk about "a regex ending in $". $ and/or \Z aren't the point here; "whole input string matches" is the point. mrabarnett commented on Oct 13, 2012 Tim, my point is that if the MULTILINE flag happens to be turned on, '$' won't just match at the end of the string (or slice), it'll also match at a newline, so wrapping the pattern in (?:...)$ in that case could give the wrong answer, but wrapping it in (?:...)\Z would give the right answer. pitrou commented on Oct 13, 2012
Tim, my point is that if the MULTILINE flag happens to be turned on,
'$' won't just match at the end of the string (or slice), it'll also
match at a newline, so wrapping the pattern in (?:...)$ in that case
could give the wrong answer, but wrapping it in (?:...)\Z would give
the right answer.
This means Tim and Guido are right that a dedicated fullmatch() method
is desireable. serhiy-storchaka commented on Oct 13, 2012 Definitely this is not easy issue. tim-one commented on Oct 14, 2012 Serhiy, I expect this is easy to implement _inside_ the regexp engine. The complications come from trying to do it outside the engine. But even there, wrapping the original regexp <re> in
(?:<re>)\Z
is at worst very close. The only insecurity with that I've thought of concerns the doc's warnings about what can appear before an inline re.VERBOSE flag. It probably works fine even if <re> does begin with (?...x....). mrabarnett commented on Oct 14, 2012 It certainly appears to ignore the whitespace, even if the "(?x)" is at the end of the pattern or in the middle of a group.
Another point we need to consider is that the user might want to use a pre-compiled pattern. mrabarnett commented on Oct 16, 2012 I'm about to add this to my regex implementation and, naturally, I want it to have the same name for compatibility.
However, I'm not that keen on "fullmatch" and would prefer "matchall" instead.
What do you think? pitrou commented on Oct 12, 2013 Serhiy, sorry to ping you, but do you think you're gonna look at this? birkenfeld commented on Oct 13, 2013 I updated the patch to current tip, fixed three issues from the review, and added documentation updates. python-dev commented on Oct 13, 2013 birkenfeld commented on Oct 13, 2013 Sorry, accidental push, already reverted. serhiy-storchaka commented on Oct 26, 2013 Patch updated to current tip. I have added some changes from the review and have added some tests.
Matthew, why change for SRE_OP_REPEAT_ONE is needed? Tests are passed without it. serhiy-storchaka commented on Nov 16, 2013 Matthew, could you please answer my question? mrabarnett commented on Nov 16, 2013 I don't know that it's not needed. python-dev commented on Nov 23, 2013 serhiy-storchaka commented on Nov 23, 2013 Committed with additional test (re.fullmatch('a+', 'ab')) which proves that change for SRE_OP_REPEAT_ONE are needed.
Thank you Matthew for your contribution.
MetadataMetadataDevelopmentNo branches or pull requestsIssue actions
|