GeezerVison

The World Wide Web, restored to 800 pixels of glory.

GeezerVison is showing github.com — Use-after-free in `memoryview` comparison via re-entrant `struct.Struct.unpack_from` · Issue #142663 · python/cpython

Use-after-free in `memoryview` comparison via re-entrant `struct.Struct.unpack_from` #142663

New issue

Copy link

New issue

Copy link

Open

Use-after-free in memoryview comparison via re-entrant struct.Struct.unpack_from#142663

Copy link

Assignees

Labels

interpreter-core(Objects, Python, Grammar, and Parser dirs)(Objects, Python, Grammar, and Parser dirs)type-crashA hard crash of the interpreter, possibly with a core dumpA hard crash of the interpreter, possibly with a core dump

Description

jackfromeast

opened

on Dec 13, 2025

Issue body actions

What happened?

Comparing two 1-D memoryviews iterates element-by-element. For each element, struct_unpack_single() first memcpys from the left view’s saved pointer, then calls the type’s unpack_from. A subclassed unpack_from can mv.release() and resize the exporting array('d'), reallocating and freeing the original buffer. The loop then advances to the next element using that stale pointer, so the next memcpy dereferences freed memory, causing a heap use-after-free.

Proof of Concept:

from array import array
import struct

a = array("d", [1.0, 2.0])
b = array("l", [1, 2])
mv = memoryview(a)
mv2 = memoryview(b)

class EvilStruct(struct.Struct):
    def unpack_from(self, buf, /, offset=0):
        mv.release()
        a.append(3.14)  # resizes array (reallocates & frees old data)
        return (1,)

struct.Struct = EvilStruct
print(mv == mv2)

Affected Versions:

Details

Python Version	Status	Exit Code
`Python 3.9.24+ (heads/3.9:9c4638d, Oct 17 2025, 11:19:30)`	ASAN	1
`Python 3.10.19+ (heads/3.10:0142619, Oct 17 2025, 11:20:05) [GCC 13.3.0]`	ASAN	1
`Python 3.11.14+ (heads/3.11:88f3f5b, Oct 17 2025, 11:20:44) [GCC 13.3.0]`	ASAN	1
`Python 3.12.12+ (heads/3.12:8cb2092, Oct 17 2025, 11:21:35) [GCC 13.3.0]`	ASAN	1
`Python 3.13.9+ (heads/3.13:0760a57, Oct 17 2025, 11:22:25) [GCC 13.3.0]`	ASAN	1
`Python 3.14.0+ (heads/3.14:889e918, Oct 17 2025, 11:23:02) [GCC 13.3.0]`	ASAN	1
`Python 3.15.0a1+ (heads/main:fbf0843, Oct 17 2025, 11:23:37) [GCC 13.3.0]`	ASAN	1

Related Code Snippet

Details

static PyObject *
memory_richcompare(PyObject *v, PyObject *w, int op)
{
    PyObject *res;
    Py_buffer wbuf, *vv;
    Py_buffer *ww = NULL;
    struct unpacker *unpack_v = NULL;
    struct unpacker *unpack_w = NULL;
    char vfmt, wfmt;
    int equal = MV_COMPARE_NOT_IMPL;

    ...
    else if (vv->ndim == 1) {
        equal = cmp_base(vv->buf, ww->buf, vv->shape,
                         vv->strides, vv->suboffsets,
                         ww->strides, ww->suboffsets,
                         vfmt, unpack_v, unpack_w);
    }
}

static int
cmp_base(const char *p, const char *q, const Py_ssize_t *shape,
         const Py_ssize_t *pstrides, const Py_ssize_t *psuboffsets,
         const Py_ssize_t *qstrides, const Py_ssize_t *qsuboffsets,
         char fmt, struct unpacker *unpack_p, struct unpacker *unpack_q)
{
    Py_ssize_t i;
    int equal;

    for (i = 0; i < shape[0]; p+=pstrides[0], q+=qstrides[0], i++) {
        const char *xp = ADJUST_PTR(p, psuboffsets, 0);
        const char *xq = ADJUST_PTR(q, qsuboffsets, 0);
        equal = unpack_cmp(xp, xq, fmt, unpack_p, unpack_q);
        if (equal <= 0)
            return equal;
    }

    return 1;
}

/* unpack a single item */
static PyObject *
struct_unpack_single(const char *ptr, struct unpacker *x)
{
    PyObject *v;
		
	// Bug: The second time comparsion will trigger the crash as the first unpack free the old buffer
    memcpy(x->item, ptr, x->itemsize);
    
    // Override Struct.unpack_from has been called on each element.
    v = PyObject_CallOneArg(x->unpack_from, x->mview);
    if (v == NULL)
        return NULL;

    if (PyTuple_GET_SIZE(v) == 1) {
        PyObject *res = Py_NewRef(PyTuple_GET_ITEM(v, 0));
        Py_DECREF(v);
        return res;
    }

    return v;
}

Sanitizer Output

Details

=================================================================
==1537956==ERROR: AddressSanitizer: heap-use-after-free on address 0x504000005de8 at pc 0x78dc8acfb42e bp 0x7fffff77b8b0 sp 0x7fffff77b058
READ of size 8 at 0x504000005de8 thread T0
    #0 0x78dc8acfb42d in memcpy ../../../../src/libsanitizer/sanitizer_common/sanitizer_common_interceptors_memintrinsics.inc:115
    #1 0x63635b462d2e in memcpy /usr/include/x86_64-linux-gnu/bits/string_fortified.h:29
    #2 0x63635b462d2e in struct_unpack_single Objects/memoryobject.c:2136
    #3 0x63635b462e55 in struct_unpack_cmp Objects/memoryobject.c:2940
    #4 0x63635b463c8f in unpack_cmp Objects/memoryobject.c:3030
    #5 0x63635b463dc5 in cmp_base Objects/memoryobject.c:3052
    #6 0x63635b4681c1 in memory_richcompare Objects/memoryobject.c:3168
    #7 0x63635b477687 in do_richcompare Objects/object.c:1059
    #8 0x63635b47793d in PyObject_RichCompare Objects/object.c:1108
    #9 0x63635b64865f in _PyEval_EvalFrameDefault Python/generated_cases.c.h:4683
    #10 0x63635b679e54 in _PyEval_EvalFrame Include/internal/pycore_ceval.h:121
    #11 0x63635b67a148 in _PyEval_Vector Python/ceval.c:2001
    #12 0x63635b67a3f8 in PyEval_EvalCode Python/ceval.c:884
    #13 0x63635b771507 in run_eval_code_obj Python/pythonrun.c:1365
    #14 0x63635b771723 in run_mod Python/pythonrun.c:1459
    #15 0x63635b77257a in pyrun_file Python/pythonrun.c:1293
    #16 0x63635b775220 in _PyRun_SimpleFileObject Python/pythonrun.c:521
    #17 0x63635b7754f6 in _PyRun_AnyFileObject Python/pythonrun.c:81
    #18 0x63635b7c674d in pymain_run_file_obj Modules/main.c:410
    #19 0x63635b7c69b4 in pymain_run_file Modules/main.c:429
    #20 0x63635b7c81b2 in pymain_run_python Modules/main.c:691
    #21 0x63635b7c8842 in Py_RunMain Modules/main.c:772
    #22 0x63635b7c8a2e in pymain_main Modules/main.c:802
    #23 0x63635b7c8db3 in Py_BytesMain Modules/main.c:826
    #24 0x63635b24c645 in main Programs/python.c:15
    #25 0x78dc8a82a1c9 in __libc_start_call_main ../sysdeps/nptl/libc_start_call_main.h:58
    #26 0x78dc8a82a28a in __libc_start_main_impl ../csu/libc-start.c:360
    #27 0x63635b24c574 in _start (/home/jackfromeast/Desktop/entropy/tasks/grammar-afl++-latest/targets/cpython/python+0x2dd574) (BuildId: ff3dc40ea460bd4beb2c3a72283cca525b319bf0)

0x504000005de8 is located 24 bytes inside of 40-byte region [0x504000005dd0,0x504000005df8)
freed by thread T0 here:
    #0 0x78dc8acfc778 in realloc ../../../../src/libsanitizer/asan/asan_malloc_linux.cpp:85
    #1 0x63635b47f2d2 in _PyMem_RawRealloc Objects/obmalloc.c:85
    #2 0x63635b4814e4 in _PyMem_DebugRawRealloc Objects/obmalloc.c:3010
    #3 0x63635b481823 in _PyMem_DebugRealloc Objects/obmalloc.c:3108
    #4 0x63635b4a82e7 in PyMem_Realloc Objects/obmalloc.c:1063
    #5 0x78dc89f39b78 in array_resize Modules/arraymodule.c:183
    #6 0x78dc89f39dae in ins1 Modules/arraymodule.c:698
    #7 0x78dc89f39f06 in ins Modules/arraymodule.c:1153
    #8 0x78dc89f39f45 in array_array_append_impl Modules/arraymodule.c:1418
    #9 0x78dc89f39f57 in array_array_append Modules/clinic/arraymodule.c.h:327
    #10 0x63635b3d7473 in method_vectorcall_O Objects/descrobject.c:476
    #11 0x63635b3b7e7f in _PyObject_VectorcallTstate Include/internal/pycore_call.h:169
    #12 0x63635b3b7f72 in PyObject_Vectorcall Objects/call.c:327
    #13 0x63635b636056 in _PyEval_EvalFrameDefault Python/generated_cases.c.h:1620
    #14 0x63635b679e54 in _PyEval_EvalFrame Include/internal/pycore_ceval.h:121
    #15 0x63635b67a148 in _PyEval_Vector Python/ceval.c:2001
    #16 0x63635b3b79b8 in _PyFunction_Vectorcall Objects/call.c:413
    #17 0x63635b3bde05 in _PyObject_VectorcallTstate Include/internal/pycore_call.h:169
    #18 0x63635b3be1ed in method_vectorcall Objects/classobject.c:65
    #19 0x63635b3b7e7f in _PyObject_VectorcallTstate Include/internal/pycore_call.h:169
    #20 0x63635b3b803f in PyObject_CallOneArg Objects/call.c:395
    #21 0x63635b462d5e in struct_unpack_single Objects/memoryobject.c:2137
    #22 0x63635b462e55 in struct_unpack_cmp Objects/memoryobject.c:2940
    #23 0x63635b463c8f in unpack_cmp Objects/memoryobject.c:3030
    #24 0x63635b463dc5 in cmp_base Objects/memoryobject.c:3052
    #25 0x63635b4681c1 in memory_richcompare Objects/memoryobject.c:3168
    #26 0x63635b477687 in do_richcompare Objects/object.c:1059
    #27 0x63635b47793d in PyObject_RichCompare Objects/object.c:1108
    #28 0x63635b64865f in _PyEval_EvalFrameDefault Python/generated_cases.c.h:4683
    #29 0x63635b679e54 in _PyEval_EvalFrame Include/internal/pycore_ceval.h:121

previously allocated by thread T0 here:
    #0 0x78dc8acfd9c7 in malloc ../../../../src/libsanitizer/asan/asan_malloc_linux.cpp:69
    #1 0x63635b47f284 in _PyMem_RawMalloc Objects/obmalloc.c:63
    #2 0x63635b47e655 in _PyMem_DebugRawAlloc Objects/obmalloc.c:2887
    #3 0x63635b47e6bd in _PyMem_DebugRawMalloc Objects/obmalloc.c:2920
    #4 0x63635b47ff3b in _PyMem_DebugMalloc Objects/obmalloc.c:3085
    #5 0x63635b4a8204 in PyMem_Malloc Objects/obmalloc.c:1041
    #6 0x78dc89f39a2c in newarrayobject Modules/arraymodule.c:663
    #7 0x78dc89f3d4a4 in array_new Modules/arraymodule.c:2788
    #8 0x63635b4de346 in type_call Objects/typeobject.c:2448
    #9 0x63635b3b7c71 in _PyObject_MakeTpCall Objects/call.c:242
    #10 0x63635b3b7f19 in _PyObject_VectorcallTstate Include/internal/pycore_call.h:167
    #11 0x63635b3b7f72 in PyObject_Vectorcall Objects/call.c:327
    #12 0x63635b636056 in _PyEval_EvalFrameDefault Python/generated_cases.c.h:1620
    #13 0x63635b679e54 in _PyEval_EvalFrame Include/internal/pycore_ceval.h:121
    #14 0x63635b67a148 in _PyEval_Vector Python/ceval.c:2001
    #15 0x63635b67a3f8 in PyEval_EvalCode Python/ceval.c:884
    #16 0x63635b771507 in run_eval_code_obj Python/pythonrun.c:1365
    #17 0x63635b771723 in run_mod Python/pythonrun.c:1459
    #18 0x63635b77257a in pyrun_file Python/pythonrun.c:1293
    #19 0x63635b775220 in _PyRun_SimpleFileObject Python/pythonrun.c:521
    #20 0x63635b7754f6 in _PyRun_AnyFileObject Python/pythonrun.c:81
    #21 0x63635b7c674d in pymain_run_file_obj Modules/main.c:410
    #22 0x63635b7c69b4 in pymain_run_file Modules/main.c:429
    #23 0x63635b7c81b2 in pymain_run_python Modules/main.c:691
    #24 0x63635b7c8842 in Py_RunMain Modules/main.c:772
    #25 0x63635b7c8a2e in pymain_main Modules/main.c:802
    #26 0x63635b7c8db3 in Py_BytesMain Modules/main.c:826
    #27 0x63635b24c645 in main Programs/python.c:15
    #28 0x78dc8a82a1c9 in __libc_start_call_main ../sysdeps/nptl/libc_start_call_main.h:58
    #29 0x78dc8a82a28a in __libc_start_main_impl ../csu/libc-start.c:360

SUMMARY: AddressSanitizer: heap-use-after-free ../../../../src/libsanitizer/sanitizer_common/sanitizer_common_interceptors_memintrinsics.inc:115 in memcpy
Shadow bytes around the buggy address:
  0x504000005b00: fa fa fd fd fd fd fd fd fa fa 00 00 00 00 00 02
  0x504000005b80: fa fa 00 00 00 00 00 00 fa fa fd fd fd fd fd fd
  0x504000005c00: fa fa 00 00 00 00 00 fa fa fa fd fd fd fd fd fd
  0x504000005c80: fa fa fd fd fd fd fd fa fa fa fd fd fd fd fd fd
  0x504000005d00: fa fa 00 00 00 00 07 fa fa fa 00 00 00 00 00 00
=>0x504000005d80: fa fa fd fd fd fd fd fa fa fa fd fd fd[fd]fd fa
  0x504000005e00: fa fa fd fd fd fd fd fa fa fa 00 00 00 00 00 fa
  0x504000005e80: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x504000005f00: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x504000005f80: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x504000006000: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
Shadow byte legend (one shadow byte represents 8 application bytes):
  Addressable:           00
  Partially addressable: 01 02 03 04 05 06 07 
  Heap left redzone:       fa
  Freed heap region:       fd
  Stack left redzone:      f1
  Stack mid redzone:       f2
  Stack right redzone:     f3
  Stack after return:      f5
  Stack use after scope:   f8
  Global redzone:          f9
  Global init order:       f6
  Poisoned by user:        f7
  Container overflow:      fc
  Array cookie:            ac
  Intra object redzone:    bb
  ASan internal:           fe
  Left alloca redzone:     ca
  Right alloca redzone:    cb
==1537956==ABORTIN

Linked PRs

gh-142663: Fix use-after-free in memoryview comparison #143305

Activity

aisk
added
type-crashA hard crash of the interpreter, possibly with a core dumpA hard crash of the interpreter, possibly with a core dump

interpreter-core(Objects, Python, Grammar, and Parser dirs)(Objects, Python, Grammar, and Parser dirs)
on Dec 13, 2025
superboy-zjc
added a commit that references this issue on Dec 31, 2025
pythongh-142663: Fix use-after-free in memoryview comparison
edd566c
superboy-zjc commented on Dec 31, 2025
superboy-zjc
on Dec 31, 2025
Contributor
More actions
When comparing two memoryview objects with different formats, memory_richcompare uses the struct module to unpack elements. A custom struct.Struct.unpack_from implementation could releases and resizes underlying buffer, which invalidates the buffer pointer, during iteration. This leads to a use-after-free when the comparison loop continued accessing the freed memory.

The proposed fix: #143305 increments the exports count of the memoryview objects before performing the comparison, effectively locking the buffers. This mirrors the protection already provided for non-memoryview objects via PyObject_GetBuffer.
Reacted by jackfromeast
picnixz
assigned
superboy-zjc
on Jan 1, 2026
superboy-zjc
added a commit that references this issue on Jan 3, 2026
pythongh-142663: resolving comments by refactoring unit tests and cac…
06846a9
serhiy-storchaka
added this to Struct, memoryview and array issues 🏗️on Jul 3, 2026
github-project-automation
moved this to Todo in Struct, memoryview and array issues 🏗️on Jul 3, 2026
serhiy-storchaka
moved this from Todo to In Progress in Struct, memoryview and array issues 🏗️on Jul 13, 2026

ngoldbaum commented on Oct 21, 2026

ngoldbaum

on Oct 21, 2026

Contributor

More actions

I found a similar issue that #143305 doesn't fix:

from threading import Event, Thread


data = bytearray(b"a" * 65536)
view = memoryview(data)
started = Event()
resume = Event()


class Other:
    def __buffer__(self, flags):
        started.set()
        resume.wait()
        return memoryview(data)


def replace_data():
    started.wait()
    try:
        view.release()
        data.clear()
        data.extend(b"a" * 65536)
        print("data replaced", flush=True)
    except BufferError:
        print("resize blocked", flush=True)
    finally:
        resume.set()


thread = Thread(target=replace_data)
thread.start()
print("equal:", view == Other())
thread.join()

This triggers a heap user-after-free on the GIL-enabled build.

I think this indicates that we need to expand the free-threaded-only fixes in #156248 (ping @nascheme) to also cover the GIL-enabled build. Specifically, I think we need the memoryview_pin and memoryview_unpin APIs Neil's PR adds to manage the lifetime issues we're running into.

Metadata

Assignees

superboy-zjc

Labels

Projects

Struct, memoryview and array issues 🏗️
Status
In Progress
Show more project fields

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions

You are visitor 01402520 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.

GeezerVison

Use-after-free in memoryview comparison via re-entrant struct.Struct.unpack_from #142663

Description

What happened?

Linked PRs

Activity

superboy-zjc commented on Dec 31, 2025

ngoldbaum commented on Oct 21, 2026

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions

Use-after-free in `memoryview` comparison via re-entrant `struct.Struct.unpack_from` #142663