Description

picnixz

opened on Jun 15, 2025

Issue body actions

I recently rewrote the _blake2 module. I want to the same for other modules where I can align the naming of functions across the different modules. The reason why I'm doing this is because it's becoming harder to make "similar" changes everywhere. In general, if I need to change something in MD5, then I also need to change it in SHA1/SHA2/SHA3, and it's easier if the code looks similar elsewhere.

There are some places that need to be updated because they are dead code, e.g.:

/*[clinic input]
module _sha2
class SHA256Type "SHA256object *" "&PyType_Type"
class SHA512Type "SHA512object *" "&PyType_Type"
[clinic start generated code]*/

should be

/*[clinic input]
module _sha2
class SHA256Type "SHA256object *" "clinic_state()->sha256_type"
class SHA512Type "SHA512object *" "clinic_state()->sha512_type"
[clinic start generated code]*/

Linked PRs

• gh-135532: use _Py_strhex in _md5.md5.hexdigest #135742
• gh-135532: optimize calls to PyMem_Malloc in SHAKE digest computation #135744
• gh-135532: cleanup clinic module directives for cryptographic modules #135822
• gh-135532: use defining_class for copying BLAKE-2 and SHA-3 objects #135838
• gh-135532: fortify DEBUG checks when fetching HACL*-based module state #135844

Bug fixes (3.14+)

• gh-135532: update python_hacl_namespaces.h #135741
• [3.14] gh-135532: update Modules/_hacl/python_hacl_namespaces.h (GH-135741) #135745

Backported PRs (HMAC-only)

• gh-135532: simplify handling of HACL* errors in _hmac #135740
• [3.14] gh-135532: simplify handling of HACL* errors in _hmac (GH-135740) #135743

Abandoned

• gh-135532: module-wide cleanup for HACL*-based extension modules #135536
• gh-135532: rewrite SHA-2 with macros #135821
• gh-135532: rewrite SHA-3 with macros picnixz/cpython#2

Activity

1. picnixz

   self-assigned this

   on Jun 15, 2025
2. picnixz

   added

   extension-modulesC modules in the Modules dirC modules in the Modules dir

   type-refactorCode refactoring (with no changes in behavior)Code refactoring (with no changes in behavior)

   on Jun 15, 2025
3. picnixz commented on Jun 19, 2025

   

   picnixz

   on Jun 19, 2025

   MemberAuthor

   More actions

   I've rewritten SHA-2 and SHA-3 modules with generic macros but I'm torn between the now cleaner modules and the fact that I entirely rewrote them. SHA-2 is smaller but SHA-3 is larger.
4. picnixz commented on Jun 19, 2025

   

   picnixz

   on Jun 19, 2025

   MemberAuthor

   More actions

   Offline I found other stuff that need to be updated independently:

   • SHA-3 types in Argument Clinic code.
   • MD-5 digests computations don't use _Py_strhex.
   • Useless memory allocation for 0-length SHAKE hashes (we can switch conditions checks)
   • Usage of unsigned char where we could use uint8_t: HACL* code expects an uint8_t so we can consistently use it.
   • Inconsistentcies in naming classes: we have _sha2.SHA256Type vs _sha3.sha3_224. Note SHA-2 doesn't expose a __new__ method but SHA-3 does. MD-5/SHA-1/SHA-1 don't have a tp_new slot but SHA-3 and BLAKE-2 do. I don't know which one is the best, but I don't think we should have both.
5. picnixz

   added a commit that references this issue on Jun 20, 2025

   gh-135532: simplify handling of HACL* errors in `_hmac` (#135740)

   2dbada1
6. miss-islington

   added a commit that references this issue on Jun 20, 2025

   pythongh-135532: simplify handling of HACL* errors in `_hmac` (python…

   fdad74d
7. picnixz

   added 2 commits that reference this issue on Jun 20, 2025

   [3.14] gh-135532: simplify handling of HACL* errors in `_hmac` (GH-13…

   d057cb4

   gh-135532: update `Modules/_hacl/python_hacl_namespaces.h` (#135741)

   57dba7c
8. miss-islington

   added a commit that references this issue on Jun 20, 2025

   pythongh-135532: update `Modules/_hacl/python_hacl_namespaces.h` (pyt…

   adeb79f
9. picnixz

   added 5 commits that reference this issue on Jun 20, 2025

   gh-135532: use `_Py_strhex` in HACL-MD5's `hexdigest` (#135742)

   eec7a8f

   [3.14] gh-135532: update `Modules/_hacl/python_hacl_namespaces.h` (GH…

   28fc24e

   gh-135532: optimize calls to `PyMem_Malloc` in SHAKE digest computati…

   d08b4b2

   gh-135532: cleanup clinic `module` directives for cryptographic modul…

   621a8bd

   gh-135532: use `defining_class` for copying BLAKE-2 and SHA-3 objects (…

   ef4fc86
10. 7 remaining items
11. Pranjal095

    added a commit that references this issue on Jul 12, 2025

    pythongh-135532: use `defining_class` for copying BLAKE-2 and SHA-3 o…

    9b530a9
12. picnixz commented on Jul 13, 2025

    

    picnixz

    on Jul 13, 2025

    MemberAuthor

    More actions

    I think I've done enough cleanup. The PR that is currently a draft for the module's state is stale and I won't merge it because it's not that important IMO. Other modules also suffer from similar constructions, so the best plan is to add some internal helpers for correctly fetching the module's def/state from a final heap type.
13. picnixz

    closed this as completedon Jul 13, 2025
14. taegyunkim

    added 6 commits that reference this issue on Aug 4, 2025

    pythongh-135532: simplify handling of HACL* errors in `_hmac` (python…

    0a9b1bb

    pythongh-135532: update `Modules/_hacl/python_hacl_namespaces.h` (pyt…

    d962c88

    pythongh-135532: use `_Py_strhex` in HACL-MD5's `hexdigest` (python#1…

    f262490

    pythongh-135532: optimize calls to `PyMem_Malloc` in SHAKE digest com…

    c5183d7

    pythongh-135532: cleanup clinic `module` directives for cryptographic…

    6a4d724

    pythongh-135532: use `defining_class` for copying BLAKE-2 and SHA-3 o…

    6c18125
15. Agent-Hellboy

    added 6 commits that reference this issue on Aug 19, 2025

    pythongh-135532: simplify handling of HACL* errors in `_hmac` (python…

    100306b

    pythongh-135532: update `Modules/_hacl/python_hacl_namespaces.h` (pyt…

    2b72f68

    pythongh-135532: use `_Py_strhex` in HACL-MD5's `hexdigest` (python#1…

    e682bcb

    pythongh-135532: optimize calls to `PyMem_Malloc` in SHAKE digest com…

    b6f87f1

    pythongh-135532: cleanup clinic `module` directives for cryptographic…

    e356a1c

    pythongh-135532: use `defining_class` for copying BLAKE-2 and SHA-3 o…

    4dc62a2

Issue actions