Bug report

Bug description:

email.header.Header.encode() builds each output line in the private _Accumulator list, whose __len__() adds up the lengths of all its parts on every call:

_ValueFormatter._append_chunk() calls len() after every word it appends (L514), so the cost of a line grows with the square of the number of words on it. With folding disabled the whole value is a single line. That is the default for Message.as_string() (maxheaderlen=0) with the default compat32 policy, so a header with many words, such as a long References: or To: header on a mailing list message, makes as_string() quadratic:

import email, time

for n in (1_000, 2_000, 4_000, 8_000):
    msg = email.message_from_string('Subject: ' + 'word ' * n + '\n\nbody\n')
    start = time.perf_counter()
    msg.as_string()
    print(f'{n:>5,}: {(time.perf_counter() - start) * 1000:.1f} ms')

Header(value).encode(maxlinelen=0) behaves the same way. (macOS arm64, release build of main at 7e3290d, best of 3.)

This is a different code path from gh-160675 and gh-136063, which are about email._header_value_parser, used by the newer policies.

Proposed fix

Keep the total length up to date in push(), pop(), pop_from() and reset(), which are the only places the accumulator changes, and return it from __len__(). The output is unchanged. Ordinary messages get a little faster too, because len() is also called once per word when lines are folded at 78 characters.

CPython versions tested on:

CPython main branch

Operating systems tested on:

macOS

Linked PRs

• gh-161284: Fix quadratic time in email.header when folding long headers #161285