GeezerVison

The World Wide Web, restored to 800 pixels of glory.

GeezerVison is showing github.com — Double `Py_DECREF` on parent parser in `pyexpat.xmlparser.ExternalEntityParserCreate` error paths · Issue #145989 · python/cpython

Double `Py_DECREF` on parent parser in `pyexpat.xmlparser.ExternalEntityParserCreate` error paths #145989

New issue

Copy link

New issue

Copy link

Closed as duplicate

Double Py_DECREF on parent parser in pyexpat.xmlparser.ExternalEntityParserCreate error paths#145989

Copy link

Labels

extension-modulesC modules in the Modules dirC modules in the Modules dirtopic-XMLtype-crashA hard crash of the interpreter, possibly with a core dumpA hard crash of the interpreter, possibly with a core dump

Description

0xCyberstan

opened

on Mar 15, 2026

Issue body actions

Bug report

Bug description:

crash_report.txt

Bug Description

ExternalEntityParserCreate_impl() in Modules/pyexpat.c increments the parent parser's refcount early, stores it in new_parser->parent, then on three separate allocation failure paths does:

Py_DECREF(new_parser);
Py_DECREF(self);

But Py_DECREF(new_parser) triggers xmlparse_dealloc, which already calls Py_CLEAR(self->parent) — decrementing self's refcount. The explicit Py_DECREF(self) after it is therefore a double decrement on the parent parser.

Additionally, xmlparse_dealloc calls XML_ParserFree() on the child's XML_Parser handle. Because the child was created by XML_ExternalEntityParserCreate, it shares DTD hash table state with the parent. Freeing the child while the parent's XML_Parse is mid-execution corrupts the parent's internal Expat structures, causing an immediate segfault.

Affected Code

Three error paths in ExternalEntityParserCreate_impl() ([source](https://github.com/python/cpython/blob/main/Modules/pyexpat.c#L1097)):

Path A — buffer malloc failure (line ~1099):

if (new_parser->buffer == NULL) {
    Py_DECREF(new_parser);    // → dealloc → Py_CLEAR(parent) → DECREF self
    Py_DECREF(self);          // ← double DECREF
    return PyErr_NoMemory();
}

Path B — XML_ExternalEntityParserCreate failure (line ~1105):

if (!new_parser->itself) {
    Py_DECREF(new_parser);    // → dealloc → Py_CLEAR(parent) → DECREF self
    Py_DECREF(self);          // ← double DECREF
    return PyErr_NoMemory();
}

Path C — handlers malloc failure (line ~1119):

if (!new_parser->handlers) {
    Py_DECREF(new_parser);    // → dealloc → Py_CLEAR(parent) → DECREF self
    Py_DECREF(self);          // ← double DECREF
    return PyErr_NoMemory();
}

The root cause is that Py_INCREF(self) is called early (line ~1082), new_parser->parent is set to self (line ~1093), and xmlparse_dealloc already handles decrementing the parent reference via Py_CLEAR(self->parent) (line ~1567). The explicit Py_DECREF(self) on each error path is redundant and incorrect.

PoC

Triggers Path A by setting a large buffer_size under a constrained address space:

"""Minimal PoC: Double Py_DECREF in pyexpat ExternalEntityParserCreate"""
import resource, xml.parsers.expat

p = xml.parsers.expat.ParserCreate()
p.SetParamEntityParsing(xml.parsers.expat.XML_PARAM_ENTITY_PARSING_ALWAYS)
p.buffer_text = True
p.buffer_size = 2**31 - 1

def handler(context, base, sysid, pubid):
    p.ExternalEntityParserCreate(context)
    return 1

p.ExternalEntityRefHandler = handler
resource.setrlimit(resource.RLIMIT_AS, (1 << 30, 1 << 30))

p.Parse(b'<?xml version="1.0"?><!DOCTYPE f[<!ENTITY e SYSTEM "x">]><f>&e;</f>', True)

Expected: MemoryError raised cleanly.

Actual: Segmentation fault (exit code 139).

$ python3 poc.py
Segmentation fault (core dumped)

GDB Backtrace (Python 3.14.2)

#0  clear_handlers (initial=0x0, self=0x7ffff733f530) at ./Modules/pyexpat.c:2398
#1  xmlparse_clear (op=0x7ffff733f530) at ./Modules/pyexpat.c:1456
#2  xmlparse_dealloc (op=0x7ffff733f530) at ./Modules/pyexpat.c:1470
#3  _Py_Dealloc (op=0x7ffff733f530) at Objects/object.c:3072
#4  Py_DECREF (op=0x7ffff733f530) at ./Include/refcount.h:421
#5  pyexpat_xmlparser_ExternalEntityParserCreate_impl (...) at ./Modules/pyexpat.c:1113
#6  pyexpat_xmlparser_ExternalEntityParserCreate (...) at ./Modules/clinic/pyexpat.c.h:313
#7  _PyObject_VectorcallTstate (...) at ./Include/internal/pycore_call.h:169
#8  PyObject_Vectorcall (...) at Objects/call.c:327
#9  _PyEval_EvalFrameDefault (...) at Python/generated_cases.c.h:1619
#10 _PyEval_EvalFrame (...) at ./Include/internal/pycore_ceval.h:121
#11 _PyEval_Vector (...) at Python/ceval.c:2083
#12 call_with_frame (..., funcname="ExternalEntityRef") at ./Modules/pyexpat.c:300
#13 my_ExternalEntityRefHandler (...) at ./Modules/pyexpat.c:746
#14 doContent (parser=0x555555675058, ...) at ./Modules/expat/xmlparse.c:3453
#15 contentProcessor (...) at ./Modules/expat/xmlparse.c:3174
#16 doProlog (...) at ./Modules/expat/xmlparse.c:5480
#17 prologProcessor (...) at ./Modules/expat/xmlparse.c:5183
#18 prologInitProcessor (...) at ./Modules/expat/xmlparse.c:4985
#19 prologInitProcessor (...) at ./Modules/expat/xmlparse.c:4979
#20 callProcessor (...) at ./Modules/expat/xmlparse.c:1291
#21 PyExpat_XML_ParseBuffer (...) at ./Modules/expat/xmlparse.c:2489
#22 PyExpat_XML_Parse (...) at ./Modules/expat/xmlparse.c:2443
#23 pyexpat_xmlparser_Parse_impl (...) at ./Modules/pyexpat.c:880

The crash is at frame #5 (pyexpat.c:1113) — the redundant Py_DECREF(self) after Py_DECREF(new_parser). The dealloc chain (frames #0–#4) shows new_parser's dealloc calling clear_handlers on an already-corrupted parser object.

Suggested Fix

Remove the three Py_DECREF(self) lines from the error paths. The dealloc of new_parser already handles the parent reference. Optionally, move the Py_INCREF(self) to right before the assignment and use Py_NewRef for clarity:

-    Py_INCREF(self);
     ...
-    new_parser->parent = (PyObject *)self;
+    new_parser->parent = Py_NewRef(self);
     ...
     if (new_parser->buffer == NULL) {
         Py_DECREF(new_parser);
-        Py_DECREF(self);
         return PyErr_NoMemory();
     }
     if (!new_parser->itself) {
         Py_DECREF(new_parser);
-        Py_DECREF(self);
         return PyErr_NoMemory();
     }
     ...
     if (!new_parser->handlers) {
         Py_DECREF(new_parser);
-        Py_DECREF(self);
         return PyErr_NoMemory();
     }

Versions

Segfault reproduced on all tested versions:

Python 3.12.12
Python 3.13.1
Python 3.14.2
Linux x86_64

Ive attached the full gdb backtrace of the crash below.

crash_report.txt

CPython versions tested on:

3.14

Operating systems tested on:

Linux

Linked PRs

gh-145989: Fix double Py_DECREF in pyexpat ExternalEntityParserCreate #146017

Activity

0xCyberstan
added
type-bugAn unexpected behavior, bug, or errorAn unexpected behavior, bug, or error
on Mar 15, 2026
StanFromIreland
added
extension-modulesC modules in the Modules dirC modules in the Modules dir

topic-XML

type-crashA hard crash of the interpreter, possibly with a core dumpA hard crash of the interpreter, possibly with a core dump
and removed
type-bugAn unexpected behavior, bug, or errorAn unexpected behavior, bug, or error
on Mar 15, 2026
VanshAgarwal24036 commented on Mar 16, 2026
VanshAgarwal24036
on Mar 16, 2026
Contributor
More actions
Hello, I’m interested in working on this issue if it is not already being worked on.
Please let me know if I can take this up.
Thanks!
encukou commented on Mar 16, 2026
encukou
on Mar 16, 2026
Member
More actions
@stanly363, do you want to send a PR for this?

The Py_NewRef does look more clear to me.
picnixz commented on Mar 16, 2026
picnixz
on Mar 16, 2026
Member
More actions
Wait, wasn't it already fixed?
picnixz commented on Mar 16, 2026
picnixz
on Mar 16, 2026
Member
More actions
This is a dup of #144984
Reacted by Petr Viktorin
picnixz
closed this as a duplicateof NULL deref + Double Py_DECREF in pyexpat ExternalEntityParserCreate() #144984 on Mar 16, 2026

Metadata

Assignees

No one assigned

Labels

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions

You are visitor 01402520 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.

GeezerVison

Double Py_DECREF on parent parser in pyexpat.xmlparser.ExternalEntityParserCreate error paths #145989

Description

Bug report

Bug description:

Bug Description

Affected Code

PoC

GDB Backtrace (Python 3.14.2)

Suggested Fix

Versions

CPython versions tested on:

Operating systems tested on:

Linked PRs

Activity

VanshAgarwal24036 commented on Mar 16, 2026

encukou commented on Mar 16, 2026

picnixz commented on Mar 16, 2026

picnixz commented on Mar 16, 2026

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions

Double `Py_DECREF` on parent parser in `pyexpat.xmlparser.ExternalEntityParserCreate` error paths #145989