GeezerVison is showing github.com — Undefined Behavior in _PyUnicodeWriter_WriteASCIIString: NULL pointer passed to memcpy when len is 0 · Issue #146196 · python/cpython
Undefined Behavior in _PyUnicodeWriter_WriteASCIIString: NULL pointer passed to memcpy when len is 0 #146196Labels interpreter-core(Objects, Python, Grammar, and Parser dirs)(Objects, Python, Grammar, and Parser dirs)type-bugAn unexpected behavior, bug, or errorAn unexpected behavior, bug, or error DescriptionBug report
Bug description:
The function _PyUnicodeWriter_WriteASCIIString in Objects/unicodeobject.c (or Objects/unicode_writer.c in some versions) contains a potential Undefined Behavior. When len is 0 and ascii is NULL, it calls memcpy with a NULL source pointer.
According to the C standard, passing NULL to memcpy is undefined even if the count is zero.
Proof of Concept
Clang's UndefinedBehaviorSanitizer (UBSan) reports:
../Objects/unicode_writer.c:494:36: runtime error: null pointer passed as argument 2, which is declared to never be null
This happens in the following block:
case PyUnicode_1BYTE_KIND:
{
const Py_UCS1 *str = (const Py_UCS1 *)ascii;
Py_UCS1 *data = writer->data;
memcpy(data + writer->pos, str, len); // <--- UB if str is NULL and len is 0
break;
}Mitigation
The function should return early if len == 0. This is a common pattern in CPython to avoid unnecessary work and prevent UB with memory functions.
if (len == -1)
len = strlen(ascii);
if (len == 0)
return 0;CPython versions tested on:
CPython main branch
Operating systems tested on:
Linux
Linked PRs
Activitysergey-miryanov commented on Mar 20, 2026 Could you check what versions affected and add MRE? ashm-dev commented on Mar 20, 2026 ContributorAuthor More actions This is only in the main branch, and I'll push a PR with the fix myself shortly ronaldoussoren commented on Mar 20, 2026 ashm-dev commented on Mar 20, 2026 ContributorAuthor More actions vstinner commented on Mar 20, 2026
Could you check what versions affected and add MRE?
@serhiy-storchaka recently added writer.write_ascii(NULL, 0) test to Lib/test/test_capi/test_unicode.py. It should trigger the undefined behavior. vstinner commented on Mar 20, 2026 Ah yes, I confirm that the newly added test triggers the undefined behavior:
$ ./configure --with-pydebug --with-undefined-behavior-sanitizer
$ make
$ ./python -m test -v test_capi.test_unicode -m test_ascii
(...)
test_ascii (test.test_capi.test_unicode.PyUnicodeWriterTest.test_ascii) ...
Objects/unicode_writer.c:494:9: runtime error: null pointer passed as argument 2, which is declared to never be null
(...)
So there is no need to add another test.
The good news is that other PyUnicodeWriter functions don't emit undefined behavior (according to existing tests). serhiy-storchaka commented on Mar 20, 2026 The test was backported to 3.14.
Note that we get a crash for non-zero size. The case for zero size was not explicitly supported, the test was only added to prevent future regressions.
We should ether make the behavior well defined by adding the explicit check for 0 size, or comment out the test. We can also explicitly raise SystemError for NULL. Either is fine. But since there is no use case for passing NULL, commenting out the test may be preferable -- it adds zero overhead. added 3 commits that reference this issue on Mar 20, 2026gh-146196: Fix Undefined Behavior in _PyUnicodeWriter_WriteASCIIStrin… vstinner commented on Mar 20, 2026 I close the issue, it has been fixed.
We should ether make the behavior well defined by adding the explicit check for 0 size, or comment out the test.
I merged the PR #146201 which fixes the undefined behavior by adding if (len == 0) test.
The test was backported to 3.14.
I also backported the change fixing the undefined behavior to 3.14, so we should be good.
MetadataMetadatainterpreter-core(Objects, Python, Grammar, and Parser dirs)(Objects, Python, Grammar, and Parser dirs)type-bugAn unexpected behavior, bug, or errorAn unexpected behavior, bug, or error DevelopmentNo branches or pull requestsIssue actions
|