GeezerVison is showing github.com — Deprecate Py_HASH_EXTERNAL as it cannot be used for `--with-hash-algorithm` · Issue #141226 · python/cpython
Deprecate Py_HASH_EXTERNAL as it cannot be used for --with-hash-algorithm #141226DescriptionBug report
Bug description:
In PEP-456, we introduced the notion of interchangeable hash functions for strings, bytes and memoryviews. Historically, Python used FNV as its hash algorithm, switched to SipHash2-4 in 3.4 before making SipHash1-3 the default in 3.11. It is however possible for embedders to implement their own hashing algorithms but this requires Python to be configured with --with-hash-algorithm=external and linking Python with the appropriate additional library.
Unfortunately, it's not possible to choose external and the only workaround would be to pass -DPy_HASH_ALGORITHM 0 but this is neither documented nor tested (it may not even work if the macro is still defined afterwards).
CPython versions tested on:
CPython main branch
Operating systems tested on:
No response
Linked PRs
Activitychanged the title [-]Py_HASH_EXTERNAL cannot be configure[/-] [+]Py_HASH_EXTERNAL cannot be used[/+] on Nov 8, 2025changed the title [-]Py_HASH_EXTERNAL cannot be used[/-] [+]Py_HASH_EXTERNAL cannot be used for `--with-hash-algorithm`[/+] on Nov 8, 2025vstinner commented on Nov 8, 2025 I'm not convinced that this feature is needed/useful. Was it asked by users embedding Python? picnixz commented on Nov 8, 2025 vstinner commented on Nov 8, 2025 It happens that a PEP is not fully implemented, and that's ok. https://peps.python.org/pep-3118/ is an example of that:
"Not all features proposed here were implemented. Specifically:"
PyObject_CopyToObject was not added.
[Additions to the struct string-syntax](https://peps.python.org/pep-3118/#struct-additions) were not added, except for ? (_Bool).
PyObject_GetMemoryView is named [PyMemoryView_FromObject](https://docs.python.org/3/c-api/memoryview.html#c.PyMemoryView_FromObject).
I suggest updating the PEP to document that external hashing was not implemented. picnixz commented on Nov 8, 2025
I suggest updating the PEP to document that external hashing was not implemented.
Unlike PEP-3118, the code path already and defining Py_HASH_ALGORITHM Py_HASH_EXTERNAL at compile time is possible for anyone building Python from source. It is just not easily exposed through the configure script (and it was really a matter of a single option). I cannot say it's not used but it's definitely possible to use it.
One might want to provide an implementation of SipHash-1-3 using SIMD when possible or maybe there are private infrastructures that would do that when the build Python from scratch. I believe it's more work than just patching the sources directly but I cannot rule out this possibility.
Now, it would ease my life if we were to drop this constant but it's being declared in PyO3 (but doesn't seem to be used) so I don't know if it's possible to do anything in Rust as well. I can also say in the docs that it should not be used. vstinner commented on Nov 8, 2025 I suggest to deprecate this feature rather than fixing its implementation. picnixz commented on Nov 8, 2025 Ok, I will prepare a PR for the deprecation as well. changed the title [-]Py_HASH_EXTERNAL cannot be used for `--with-hash-algorithm`[/-] [+]Deprecate Py_HASH_EXTERNAL as it cannot be used for `--with-hash-algorithm`[/+] on Nov 8, 2025encukou commented on Nov 10, 2025 AFAIK, the way to do this is configure with CFLAGS='-DPy_HASH_ALGORITHM=Py_HASH_EXTERNAL', and then link with an implementation of PyHash_Func.
For example you can make a very slow but relatively functional interpreter with:
static Py_hash_t zerohash(const void *src, Py_ssize_t src_sz) { return 0; }
PyHash_FuncDef PyHash_Func = {zerohash, "zerohash", 0, 0};Looks like the only thing from the PEP that's not implemented is the configure integration? picnixz commented on Nov 10, 2025 picnixz commented on Feb 21, 2026 Removal has been scheduled for Python 3.19. If I hear complaints, I will revert the deprecation and ressurect the PR that fully implements the support via configure.
MetadataMetadataDevelopmentNo branches or pull requestsIssue actions
|