GeezerVisonThe World Wide Web, restored to 800 pixels of glory. | |||
| Home|About|Gaming cabinet|Archives|How it works|Starter sites | |||
GeezerVison is showing github.com — gh-146311: Reject non-canonical padding bits in base32, 64, & 85 decoding by gpshead · Pull Request #146312 · python/cpython Conversation
|
|
discussing if base32 needs strict_mode on the issue. not adding a NEWS entry until that is decided. |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
…ro-padding-bits
Gate non-zero padding bits rejection behind a new canonical= keyword argument independent of strict_mode, per discussion on pythongh-146311. Per RFC 4648 section 3.5 ("Canonical Encoding"), decoders MAY reject encodings where pad bits are not zero. The new canonical=True flag enables this check for a2b_base64, a2b_base32, a2b_base85, and a2b_ascii85. For base85/ascii85, the canonical check also rejects single-character final groups (never produced by a conforming encoder) and verifies that partial group encodings match what the encoder would produce. Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
The _Py_ID(canonical) identifier used by the clinic-generated argument parsing code needs to be registered in the global strings. Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
RFC 4648 only covers base16, base32, and base64. The canonical encoding concept applies to base85 but is not defined by that RFC. Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
Replace the re-encode-and-compare loops with a quotient comparison: two divisions by 85**n_pad tell us whether the decoded uint32 and the zero-padded output bytes share the same leading base-85 digits. Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
- Test non-canonical rejection for all partial group sizes (2/3/4 chars) - Test digit-0 1-char group for ascii85 (exercises chunk_len==0 guard) - Test boundary byte values (\x00, \xff) at each group size Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
- Round-trip tests: encoder always produces canonical output (base64, base32, base85, ascii85) - Uniqueness tests: for base85/ascii85 partial groups, sweep all 85 last-digit values and verify exactly one decodes to the original payload with canonical=True Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
LGTM technically, but I have two notes:
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was an error while loading. Please reload this page.
There was an error while loading. Please reload this page.
Per the PLRM spec (section 3.13.3), a final partial 5-tuple containing only one character is an encoding violation. Move this check outside the `canonical=` guard so it is always enforced. Also change chunk_len and i from Py_ssize_t to int per review feedback. Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
When canonical=True, reject '!!!!!' (five zero digits) in favor of the 'z' abbreviation. The PLRM spec defines 'z' as the representation for all-zero groups, so '!!!!!' is a non-canonical encoding. Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
- Document that single-character final groups are always rejected - Add versionchanged:: next markers for the behavioral change - Update canonical= description for ascii85 to mention z-abbreviation - Update canonical= description for base85 Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
There was an error while loading. Please reload this page.
There was an error while loading. Please reload this page.
There was an error while loading. Please reload this page.
It was decided to use canonical, right? Please update also What's New. |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
…ro-padding-bits # Conflicts: # Doc/library/base64.rst
…atsnew - Reorder versionchanged param lists alphabetically in base64.rst and binascii.rst (canonical, ignorechars, padded). - Restore "two to five" in a2b_ascii85/a2b_base85 partial-group text; partial finals are 2-4 chars but a complete final group is 5. - Drop redundant versionchanged blocks from a2b_ascii85/a2b_base85 since both functions are new in 3.15. - Add Misc/NEWS.d entry and Doc/whatsnew/3.15.rst bullets covering the new canonical kwarg and the unconditional 1-char rejection.
The PLRM (Adobe Ascii85) and the various base85 alphabets do not define a "canonical" form. Where the underlying spec doesn't ground the term, the doc text now says explicitly that "canonical" here means "the encoding the corresponding b2a_* function would produce." Base16/32/64 entries are unchanged: RFC 4648 grounds the term there.
yeah given our discussion on the issue that seemed like the best name. I also updated the *85 API docs to cover what "canonical" specifically means to us given that isn't defined by any of the looser specs for those. |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
LGTM. 👍
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was an error while loading. Please reload this page.
There was an error while loading. Please reload this page.
Successfully merging this pull request may close these issues.
There was an error while loading. Please reload this page.