GeezerVison is showing github.com — email.message.Message.add_header behavior differs from Message[header]=v · Issue #151454 · python/cpython
email.message.Message.add_header behavior differs from Message[header]=v #151454DescriptionBug report
Bug description:
Attempting to copy headers from an existing message to a new (to-be-cleaned copy of the original) message, I iterated over the old message, using something like
for h, v in oldmsg.items():
newmsg.add_header(h, v)which works for many of the test-messages in my test Maildir folder, but errors out on certain messages with
newmsg.add_header(h, v)
~~~~~~~~~~~~~~~~~~~~~~^^^^^^
File "/usr/local/lib/python3.13/email/message.py", line 587, in add_header
self[_name] = SEMISPACE.join(parts)
~~~~~~~~~~~~~~^^^^^^^
TypeError: sequence item 0: expected str instance, Header found
It appears that elements in msg.items() can produce a mixed-content list containing both (str, str) and (str, Header) elements; but add_header doesn't like receiving a Header object as a value, only str.
If I change it to use the .__setitem__() method function:
for h, v in oldmsg.items():
newmsg[h] = vit appears to work fine, appending the header that .items() gave me.
According to the .add_header() documentation, it should behave similar to the .__setitem__() method:
This method is similar to __setitem__() except that additional header parameters can be provided as keyword arguments. _name is the header field to add and _value is the primary value for the header.
The fix would involve making add_header accept the same sorts of Header values as __setitem__, or at least align the output of msg.items() with the types that msg.set_header() accepts. If not aligned, at a bare minimum, the documentation should reflect the difference in accepted value types (str vs Header)
This may also have tie-ins with similar Issues here:
(manifested on 3.13 on OpenBSD, but it's standard-library stuff that would likely manifest on any OS)
CPython versions tested on:
3.13
Operating systems tested on:
Other
Linked PRs
ActivityGumnos commented on Jun 13, 2026 the offending header in question was a multi-line X-Ham-Report header. Dropping a breakpoint() in a try/except TypeError block:
(Pdb) print(h)
X-Ham-Report
(Pdb) type(v)
<class 'email.header.Header'>
(Pdb) str(v)
'Spam detection software, running on the system "example.com", has\n identified this incoming email as possible spam. The original message\n has been attached to this so you can view it (if it isn\'t spam) or label\n similar future email. If you have any questions, see\n root\\@localhost for details.\n \n Content preview: Are you still calling people around you in the workplace with\n worn-out words? Terms such as colleague, employee, boss, and intern are so\n pass�. Here\'s a fresh supply of words to bring some variety at work. [...]\n \n \n Content analysis details: (-1.7 points, 5.0 required)\n \n pts rule name description\n ---- ---------------------- --------------------------------------------------\n -0.0 SPF_HELO_PASS SPF: HELO matches SPF record\n -0.0 SPF_PASS SPF: sender matches SPF record\n 1.1 DATE_IN_PAST_03_06 Date: is 3 to 6 hours before Received: date\n -2.8 RP_MATCHES_RCVD Envelope sender domain matches handover relay domain\n 0.0 T_TVD_MIME_EPI BODY: T_TVD_MIME_EPI\n 0.0 HTML_MESSAGE BODY: HTML included in message'
(underlying message is from my Word.A.Day mailing-list archive, FWIW) picnixz commented on Jun 27, 2026 I'm honestly unsure whether this is deliberate or not. The documentation also says:
In a model generated from bytes, any header values that (in contravention of the RFCs) contain non-ASCII bytes will, when retrieved through this interface, be represented as Header objects with a charset of unknown-8bit.
However, in all examples, values are not header instances. Now, the interface in __setitem__ used for storing the value (whatever its type) is through the policy object and header_store_parse, which becomes a pass-through if we are given a headere instance (because the purpose of the parser is to normalize the inputs and always work with Header instances).
OTOH, I honestly don't know whether add_header should really accept a Header instance because it gives a different semantic of _params, namely it does two different jobs. As such, keyword arguments become irrelevant (and somehow conflicting) if we pass a Header instance as a value and that's for me a no-go. So I would rather explicitly document this. picnixz commented on Jun 27, 2026 Gumnos commented on Jun 27, 2026 Based on the descriptions, I'd expect parity between add_header() and things returned from .items(). If add_header() rejects Header objects, then if feels like .items() should at least consistently return (str, str) pairs rather than a mix of (str, str) and (str, Header). Alternatively, if .items() is going to return both types of tuple, then add_header() should accept both. picnixz commented on Jun 27, 2026 I think I actually misread the interface. In the Header description it says:
If you want to include non-ASCII characters in your email headers, say in the Subject or To fields, you should use the Header class and assign the field in the Message object to an instance of Header instead of using a string for the header value
So, Headere instance sare expected to be used when the header value is non-ASCII. I incorrectly assumed that **_params in this case would be for such cases. So, sorry for this and you're right in asking for accepting such objects as otherwise it's impossible to combine header with non-ASCII characters and with specific parameters. Gumnos commented on Jun 27, 2026 I suspect that, because a Header (should?) likely know how to encode non-7-bit ASCII, the add_header() method might just have to check whether it's a 7-bit ASCII string and do what it already does; and if it's not a 7-bit ASCII string, call some sort of "encode this Header object using the appropriate encoding method" function on the Header and then fall back to the previous case. Or just directly assign the Header value and then have the serialization of the message do that encoding/conversion. picnixz commented on Jun 27, 2026 I still wonder what to do with the additional parameters though... it feels weird for them to be ignored (as it is the case in the given PR) Gumnos commented on Jun 27, 2026 My gut says that both __setitem__() and add_headers() should be the same, i.e., __setitem__() just calls add_headers() with no extra **_params
The Header object doesn't seem well-/officially-documented in detail, but if it contains extra metadata that would translate to **_params info, then using __setitem__(Header) could extract the metadata from the Header into the **_params as part of that process. bitdancer commented on Jun 28, 2026 The behavior here is a bit wonky for historical reasons: in the transition to python3 we first tried to handle bytes via this unknow-8bit header dodge, but that was a band-aid. Really fixing bytes handling required a whole new API. Returning Header objects from items for dirty 8bit data was quite possibly not the best decision, even at the time, but it was the best I could come up with.
Header is pretty dumb, and somewhat broken. If you want to handle non-ascii, you have to build tuples, call make_header, and then pass the resulting string to add_header, using the params arguments as appropriate if you also have parameters...and you need triples there if you have non-ascii. The new API automates all of this stuff. If you are using a non-compat32 policy you just pass the string with the unicode in it, and it handles any needed encoding (which might be none if policy.utf8=True).
What we need here is a documentation overhaul to emphasize to the right way to use add_header with the new API. Which, to be honest, I have to go back and review before I could write it, though I think the answer is "just pass it a string, which can include unicode". We also need to deprecate compat32 and make 'default' the actual default policy...but that's a job I'm not quite ready to tackle yet, though I'm getting closer ;)
As for the immediate question of whether add_header should accept a Header getitem produces in compat32...maybe? You'd have to disallow the specification of params in that case. It would be a band aid on top of a band aid, though.
In the meantime, if you use a new API policy, it will probably simplify your code...and if it doesn't let me know what the problems are, please. harjothkhara commented on Jul 10, 2026 Right now the PR only covers the simple case: a Header with no params works. If you pass a Header and a param together, you still get an error, just like on main:
>>> msg.add_header('Content-Disposition', hdr, filename='x.ppt')
TypeError: sequence item 0: expected str instance, Header found
So the open question is what that combination should do. Two options:
- Keep it narrow: allow a
Header only when there are no params, and raise a clear error instead of the confusing one above.
- Make the combination work, so a value with non-ASCII characters can carry params too.
I'm fine doing either. And if a docs change is the better fix here, I can close #151458 and write that up instead. harjothkhara commented on Aug 2, 2026 picnixz commented on Aug 2, 2026 @harjothkhara Avoid LLM generated answers. This wastes reviewer's time. Gumnos commented on Aug 2, 2026 @harjothkhara wrote:
Right now the PR only covers the simple case: a Header with no params works. If you pass a Header and a param together, you still get an error
In my original example, I used .add_header() with just a Header object, no additional parameters to conflict, but that's part of what caused the "TypeError: sequence item 0: expected str instance, Header found" error.
I'm fine with the limitation that add_header() reject additional parameters when passing a Header object. And I'm fine with add_header() not taking a Header, but would then expect the documentation to remove the "works like __additem__" portion, or give a stern note detailing that, while __additem__ accepts strings and Header objects, add_header values must be strings, not Header objects. harjothkhara commented on Aug 15, 2026 Fair point. I'll keep this to a concrete decision instead of listing options.
Keeping the narrow fix: add_header accepts a Header when there are no extra params, same as what's in the PR now. Passing params with a Header value still raises a TypeError, but I made it a clear one about mixing a Header with params instead of the confusing "sequence item 0: expected str instance, Header found" message.
This fixes the bug reported here (copying headers from an existing message raises TypeError) without touching the parameterized case. A docs overhaul on the new email API is worth doing but it's a bigger, separate job. harjothkhara commented on Oct 21, 2026 @bitdancer, checking in on this. The PR lets add_header store a Header when no params are passed, but passing params still raises a TypeError. You said that might work as a stopgap. Should I keep the PR as it is, or switch to a docs-only change showing how to pass strings with the newer policies?
MetadataMetadataDevelopmentNo branches or pull requestsIssue actions
|