GeezerVison is showing github.com — binascii base64 & base32 decode ignore excess padding bits in their input · Issue #146311 · python/cpython
binascii base64 & base32 decode ignore excess padding bits in their input #146311DescriptionBug description:
CPython has had this behavior forever on these, but base64 and base32 decode both ignore excess bits in their input data. Per https://datatracker.ietf.org/doc/html/rfc4648.html#section-3.5 "decoders MAY chose to reject an encoding if the pad bits have not been set to zero".
a2b_base64() has a strict_mode= option, so doing this in that mode makes sense.
base32 is documented as always being strict so just fixing its glitch there makes sense to me. We could add a strict_mode= parameter to a2b_base32() but that would be a little sad. I doubt anyone depends on a behavior like this. It is unnatural to have excess bits in encoded data because encoders MUST (per the rfc) set those bits to 0.
This is a follow on to these recent changes:
CPython versions tested on:
3.15alpha
Linked PRs
Activitygpshead commented on Mar 22, 2026 @serhiy-storchaka thoughts on what to do about base32 here? it is an observable behavior change, but I don't intend to backport this. Should it gain a strict_mode= parameter like base64 has so that anyone depending on the behavior has an easy way out? serhiy-storchaka commented on Mar 23, 2026 This option is the least important of all RFC 4648 variations, because it is off by default. It is "decoders MAY chose to reject", not "decoders MAY chose to ignore". On other hand, all other strictness checks should be enabled by default. I think that strict_mode should be True by default, strict_mode=False should only be supported for backward compatibility, but there should be other options to disable different strictness checks. For example, ignorechars covers some of such options (ignoring newlines/whitespaces/non-alphabetic characters/"=" not at the end).
I think that:
- it is not a big deal if we continue to ignore the lowest bits in the last quad for a while
- if we add an option, it should be independent from
strict_mode.
There are still several issues with higher priority (#73613, #71986/#76672/#64331). This issue can wait. gpshead commented on Mar 23, 2026 Given the RFC section is "Canonical Encoding" naming a kwarg for this
canonical= has some appeal to it.
…
On Sun, Mar 22, 2026 at 5:43 PM Serhiy Storchaka ***@***.***> wrote:
*serhiy-storchaka* left a comment ( python/cpython#146311)
< #146311 (comment)>
This option is the least important of all RFC 4648 variations, because it
is off by default. It is "decoders MAY chose to reject", not "decoders MAY
chose to ignore". On other hand, all other strictness checks should be
enabled by default. I think that strict_mode should be True by default,
strict_mode=False should only be supported for backward compatibility,
but there should be other options to disable different strictness checks.
For example, ignorechars covers some of such options (ignoring
newlines/whitespaces/non-alphabetic characters/"=" not at the end).
I think that:
- it is not a big deal if we continue to ignore the lowest bits in the
last quad for a while
- if we add an option, it should be independent from strict_mode.
There are still several issues with higher priority ( #73613
< #73613>, #71986
< #71986>/ #76672
< #76672>/ #64331
< #64331>). This issue can wait.
—
Reply to this email directly, view it on GitHub
< #146311?email_source=notifications&email_token=AAAQXC6NTJFKDB5KB2JNAR34SCCCNA5CNFSNUABFM5UWIORPF5TWS5BNNB2WEL2JONZXKZKDN5WW2ZLOOQXTIMJQG4ZTIOBVGQ22M4TFMFZW63VGMFZXG2LHN2SWK5TFNZ2KYZTPN52GK4S7MNWGSY3L#issuecomment-4107348545>,
or unsubscribe
< https://github.com/notifications/unsubscribe-auth/AAAQXC2DLJ4LJ5YB3R7OBED4SCCCNAVCNFSM6AAAAACW27G4ZGVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHM2DCMBXGM2DQNJUGU>
.
You are receiving this because you were assigned.Message ID:
***@***.***>
serhiy-storchaka commented on Mar 26, 2026 Yes, that was the first thing that came to my mind. We should keep this in mind, but the final name will be chosen after adding options for other features. serhiy-storchaka commented on Apr 4, 2026 This is relevant also for Base85. It cannot be formulated in terms of bits, but decoding different inputs can produce the same output.
>>> binascii.a2b_base85(b'VE')
b'a'
>>> binascii.a2b_base85(b'VF')
b'a' Even more, Base85 does not produce encoded data of length 1 modulo 5. But the decoder accepts it and silently ignores that redundant character.
>>> binascii.a2b_base85(b'')
b''
>>> binascii.a2b_base85(b'V')
b'' Base32 and Base64 decoders are slightly more strict. gpshead commented on Apr 4, 2026 True, being consistent would be good. draft PR updated. serhiy-storchaka commented on Apr 5, 2026 This is not so simple for Ascii85.
- The document that specifies Ascii85 (https://www.adobe.com/jp/print/postscript/pdfs/PLRM.pdf, section 3.13.3) does not have restriction on unambiguousity of representation of the last incomplete group.
- There may be ambiguous representation in the middle -- a group of 4 zeroes should be represented as 'z', but we do not reject '00000' which is decoded to the same value.
- Finally, any whitespaces are ignored, so we can encode a lot of hidden information by inserting whitespaces.
I think it would be nice to add such option, but we should be careful with its description.
Base85 is less fluid by default, so we can speak about "canonical" representation, except that it is not officially defined, it is just projection from Base32 and Base64. Maybe use "canonical" in quotes? Reacted by Gregory P. Smith added 2 commits that reference this issue on Apr 25, 2026gh-146311: Reject non-canonical padding bits in base32, 64, & 85 deco… gpshead commented on Apr 25, 2026 merged, thanks for all the discussion and reviews (and other binascii/base64 improvements!).
MetadataMetadataDevelopmentNo branches or pull requestsIssue actions
|