GeezerVisonThe World Wide Web, restored to 800 pixels of glory. | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Home|About|Gaming cabinet|Archives|How it works|Starter sites | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
GeezerVison is showing github.com — gh-102988: Detect email address parsing errors and return empty tuple to indicate the parsing error (old API) by tdwyer · Pull Request #108250 · python/cpython Conversation
|
|
Note, I just have the regex looking for both Double Quotes and Single Quotes but I think only Double-Quotes are allowed to specify a Real Name part. Let me know if Single Quotes can NOT be used to wrap the Real Name e.g. I'll remove that from the regex if so Additionally, I'm also stripping out escaped commas before counting e.g. |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
| accepted_values = [] | ||
| for v in email_header_fields: | ||
| s = v.replace('\\(', '').replace('\\)', '') | ||
| if s.count('(') != s.count(')'): |
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
This does not check )( or any other combination when ) proceeds (. Probably it could be better to iterate through the string and manually increment and decrement verifying that the counter never goes below 0 and it is exactly 0 at the end
| if s.count('(') != s.count(')'): | |
| counter = 0 | |
| for c in s: | |
| if c == '(': | |
| counter += 1 | |
| elif c == ')': | |
| counter -= 1 | |
| if counter < 0: | |
| break | |
| if counter != 0: |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
Looks like this is not necessary. The fix already accounts for this and none of those are an issue.
My current solution works, because instead of trying to actually parse things I just detect an error has occurred by making sure that the resulting number of address 2-Tuples matches the number of Headers that were parsed. Which is why I went this direction. It's super hard to fix the actual parsing logic of RFC 2822 headers :P
./python
Python 3.13.0a0 (heads/blah_1:a1cc74c4ee, Aug 21 2023, 18:40:57) [GCC 7.5.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>>
>>> from email.utils import parseaddr, getaddresses
>>>
>>>
>>> parseaddr('[email protected]])(<[email protected]>')
('', '')
>>>
>>> parseaddr('[email protected])(<[email protected]>')
('', '')
>>>
>>> parseaddr('[email protected] <[email protected]>')
('', '')
>>> parseaddr('[email protected]..<[email protected]>')
('', '')
>>>
>>> getaddresses(['[email protected]..<[email protected]>'])
[('', '')]
>>>
>>> getaddresses(['[email protected]])(<[email protected]>'])
[('', '')]
>>>
>>> getaddresses(['[email protected])(<[email protected]>'])
[('', '')]
With the un-patched python your attack results in multiple Tuples being returned
>>> getaddresses(['[email protected]])(<[email protected]>'])
[('', '[email protected]'), ('', ''), ('', '')]
A cleaner way to say it is this. There are countless ways to trigger this parsing error but there is only one error e.g. the parser will return an abnormal number of output tuples. So, rather than trying to detect every possible input which could trigger the bug, I just detect the one error that they all result in.
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
After looking at this again, it is interesting that my solution works here… why do I check for matching parentheses at all ?
[email protected])(<bob&@example.com
I have ideas, but I’ll look into it.
Sorry, something went wrong.
There was an error while loading. Please reload this page.
|
|
||
| # When a comma is used in the Real Name part it is not a deliminator | ||
| # So strip those out before counting the commas | ||
| pattern = r'"[^"]*,[^"]*"|\'[^\']*,[^\']\'*|\\,' |
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
Does this regex support multiple commas inside quotes?
| pattern = r'"[^"]*,[^"]*"|\'[^\']*,[^\']\'*|\\,' | |
| pattern = r'''"[^"]*,[^"]*"|'[^']*,[^']'*|\\,''' |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
The regex works for multiple commas because the Not-In-List [^"] allows for other commas. There just simply has to be at least one comma in between the Quotes.
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
I'm just curious. The regex proposed by me simply ignores whatever is in single or double quotes and seems to be much simpler than this one. Is there any benefit in ignoring only the quoted parts containing at least one comma? I think it has no effect on the comma-counting logic.
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
@frenzymadness I don't see your suggested regex. In any case, perhaps what you are suggesting would be fine. I'll think on it. However, from a security perspective I'd like to be as specific as possible, because things get super tricky and I'd not want to implement a fix that is latter proven to be able to be bypassed. It's only the commas in quotes that are an issue here and should be accounted for.
In ether case, I plan on at least implementing the Triple Single-Quote syntax so I don't need to escape the single-quotes.
Additionally, I want looks some more to make sure that single-quotes are valid chars to isolate the Real Name part of the header. It's looking like only Double-Quotes to that, if so then only commas in Double-Quotes should be stripped.
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
The regex has been proposed in this comment: #102988 (comment)
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
Okay, I'll consider that. I've been busy with work but I plan on adding the extra test_email.py and updating the regex over this weekend.
Sorry, something went wrong.
There was an error while loading. Please reload this page.
| eq(utils.getaddresses(['[email protected]]<[email protected]>']), | ||
| [('', '')]) |
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
| eq(utils.getaddresses(['[email protected]]<[email protected]>']), | |
| [('', '')]) | |
| eq(utils.getaddresses(['[email protected]]<[email protected]>']), | |
| [('', '')]) | |
| eq(utils.getaddresses(['[email protected]])(<[email protected]>']), | |
| [('', '')]) |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
However, not a bad idea to add extra tests. That way if someone else changes this code in the future these other corner cases will be tested for.
Sorry, something went wrong.
There was an error while loading. Please reload this page.
|
Any updates on that? |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was an error while loading. Please reload this page.
|
After some investigation, I confirmed that I was being too carful before. Only commas in double-quotes should be accounted for not commas in single-quotes nor commas with prefixed with a backslash A comma in single-quotes and a comma prefixed by a backslash trigger the bug. However, a comma in double-quotes is parsed correctly. This is how all of those cases behave in: Python 2.7.17 Additionally, somehow the fix for CVE-2019-16056 regarding Python 2.7.17 So, this PR fixes both CVE-2019-16056 and CVE-2023-27043 |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
|
What is necessary here to move this PR forward? |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
|
My colleague Lumir Balhar @frenzymadness ran an impact check on this PR on Fedora: in short, there is no impact, the test suite of all Python packages (in Fedora) pass with the change. While there were some build errors, they were unrelated to the email issue. For details, see https://copr.fedorainfracloud.org/coprs/lbalhar/email-CVE/builds/ COPR which as more than 4300 builds. I created a copy of this PR to add a strict parameter: PR #111116. |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
|
Closing this in favor of Victor's updated version. |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was an error while loading. Please reload this page.
Successfully merging this pull request may close these issues.
There was an error while loading. Please reload this page.