GeezerVison is showing github.com — email: multiline encoded headers · Issue #91957 · python/cpython
email: multiline encoded headers #91957DescriptionBug report
There seems to be a problem with multiline encoded headers, for example subject headers.
from email import message_from_string, policy
import base64
e = """From: <[email protected]>
To: <[email protected]>
Subject: =?UTF-8?B?0JfQsNGB0YLRgNCw0YXQvtCy0LrQsCAi0JDQu9C40LDQvdGGINCX0LTRgNCw0LLQ?=
=?UTF-8?B?tSIg4oCTINCY0L3QtNC40LLQuNC00YPQsNC70LXQvSDQvdC+0LzQtdGAINC90LAg?=
=?UTF-8?B?0YPQstC10LTQvtC80LvQtdC90LjQtQ==?=
test"""
e = message_from_string(e, policy=policy.default)
s1 = e["Subject"]
try:
e.as_bytes()
except Exception:
print("error")
else:
print("ok")
s = (base64.b64decode("0JfQsNGB0YLRgNCw0YXQvtCy0LrQsCAi0JDQu9C40LDQvdGGINCX0LTRgNCw0LLQ") + base64.b64decode("tSIg4oCTINCY0L3QtNC40LLQuNC00YPQsNC70LXQvSDQvdC+0LzQtdGAINC90LAg")
+ base64.b64decode("0YPQstC10LTQvtC80LvQtdC90LjQtQ==")).decode("utf-8")
del e["Subject"]
e["Subject"] = s
s2 = e["Subject"]
try:
e.as_bytes()
except Exception:
print("error")
else:
print("ok")
print(s1 == s2)
e = """From: <[email protected]>
To: <[email protected]>
Subject: =?UTF-8?B?0JfQsNGB0YLRgNCw0YXQvtCy0LrQsCAi0JDQu9C40LDQvdGGINCX0LTRgNCw0LLQtSIg4oCTINCY0L3QtNC40LLQuNC00YPQsNC70LXQvSDQvdC+0LzQtdGAINC90LAg0YPQstC10LTQvtC80LvQtdC90LjQtQ==?=
test"""
e = message_from_string(e, policy=policy.default)
s3 = e["Subject"]
try:
e.as_bytes()
except Exception:
print("error")
else:
print("ok")
print(s2 == s3)
Your environment
Tested on Python 3.10.4 and 3.11.0a7 on Arch Linux Activitywbolster commented on May 9, 2022 it would help if you described what you believe the problem is, @mdierksen (without forcing others to reverse engineer your example code to try to understand) mdierksen commented on May 10, 2022 The first e.as_bytes() causes this exception:
Traceback (most recent call last):
File "/home/dierksen/test4.py", line 16, in <module>
e.as_bytes()
^^^^^^^^^^^^
File "/usr/local/lib/python3.11/email/message.py", line 178, in as_bytes
g.flatten(self, unixfrom=unixfrom)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/email/generator.py", line 116, in flatten
self._write(msg)
^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/email/generator.py", line 199, in _write
self._write_headers(msg)
^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/email/generator.py", line 422, in _write_headers
self._fp.write(self.policy.fold_binary(h, v))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/email/policy.py", line 200, in fold_binary
folded = self._fold(name, value, refold_binary=self.cte_type=='7bit')
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/email/policy.py", line 214, in _fold
return self.header_factory(name, ''.join(lines)).fold(policy=self)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/email/headerregistry.py", line 253, in fold
return header.fold(policy=policy)
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/email/_header_value_parser.py", line 156, in fold
return _refold_parse_tree(self, policy=policy)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/email/_header_value_parser.py", line 2825, in _refold_parse_tree
last_ew = _fold_as_ew(tstr, lines, maxlen, last_ew,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/email/_header_value_parser.py", line 2913, in _fold_as_ew
encoded_word = _ew.encode(to_encode_word, charset=encode_as)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/email/_encoded_words.py", line 222, in encode
bstring = string.encode('ascii', 'surrogateescape')
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
UnicodeEncodeError: 'ascii' codec can't encode characters in position 0-10: ordinal not in range(128)
The problem is with the subject header. Extracting the subject as a string works however.
Manually decoding the subject header and replacing it in the email object fixes the problem. Extracting the subject as a string yields the same string as before.
The problem does not occur for encoded headers in a single line, but only for multiline headers that are split up in a certain way.
MetadataMetadataDevelopmentNo branches or pull requestsIssue actions
|