When the bug occurs, a (at least one?) Python dumper() thread is calling pickle.dumps(shared):
- _pickle_dumps() -> _pickle_dumps_impl() -> dump() -> save() -> save_dict() -> batch_dict_exact()
The problem is that batch_dict_exact() is stuck in a loop, it keeps writing into memory and grows its buffer in an unlimited loop.
The loop in gdb:
(gdb) next
3489 if (dict_size - total == 1) {
(gdb) next
3518 if (_Pickler_Write(self, &mark_op, 1) < 0)
(gdb) next
3522 Py_BEGIN_CRITICAL_SECTION(obj);
(gdb) next
3523 next = PyDict_Next(obj, &ppos, &key, &value);
(gdb) next
3524 if (next) {
(gdb) next
3528 Py_END_CRITICAL_SECTION();
(gdb) next
3529 if (!next) {
(gdb) next
3545 if (_Pickler_Write(self, &setitems_op, 1) < 0)
(gdb) next
3547 if (PyDict_GET_SIZE(obj) != dict_size) {
(gdb) next
3554 } while (total < dict_size);
(gdb) next
(...)
At each iteration, next is false: PyDict_Next() exhausted the dictionary iterator, but the loop continue since the (total < dict_size) condition is false.
Variables:
- ppos = 114
- total = 47
- dict_size = 50
The code makes the assumption that PyDict_Next() iterates exactly 50 times if a dictionary has 50 entries.
Problem: Py_BEGIN_CRITICAL_SECTION(obj); and Py_END_CRITICAL_SECTION(); is put around a single PyDict_Next() call, whereas the PyDict_Next() documentation recommends putting the whole loop in the critical section.