GeezerVison is showing github.com — Fastcall uses more C stack · Issue #73044 · python/cpython
Fastcall uses more C stack #73044Description
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
assignee = None
closed_at = <Date 2016-12-05.17:45:34.448>
created_at = <Date 2016-12-02.07:54:27.620>
labels = ['3.7']
title = 'Fastcall uses more C stack'
updated_at = <Date 2016-12-05.17:45:34.447>
user = 'https://github.com/vstinner' bugs.python.org fields:
activity = <Date 2016-12-05.17:45:34.447>
actor = 'vstinner'
assignee = 'none'
closed = True
closed_date = <Date 2016-12-05.17:45:34.448>
closer = 'vstinner'
components = []
creation = <Date 2016-12-02.07:54:27.620>
creator = 'vstinner'
dependencies = []
files = ['45731', '45757']
hgrepos = []
issue_num = 28858
keywords = []
message_count = 10.0
messages = ['282228', '282247', '282252', '282370', '282371', '282373', '282377', '282385', '282426', '282447']
nosy_count = 3.0
nosy_names = ['vstinner', 'python-dev', 'serhiy.storchaka']
pr_nums = []
priority = 'normal'
resolution = 'fixed'
stage = None
status = 'closed'
superseder = None
type = None
url = 'https://bugs.python.org/issue28858'
versions = ['Python 3.7'] Activityvstinner commented on Dec 2, 2016 Serhiy Storchaka reported that Python 3.6 crashs earlier than Python 3.5 on calling json.dumps() when sys.setrecursionlimit() is increased.
I tested the script he wrote. Results on Python built in release mode:
Python 3.7:
...
58100 116204
Segmentation fault (core dumped)
Python 3.6:
...
74800 149604
Segmentation fault (core dumped)
Python 3.5:
...
74700 149404
Segmentation fault (core dumped)
Oh, it seems like Python 3.7 does crash earlier.
But to be clear, it's hard to control the usage of the C stack. vstinner commented on Dec 2, 2016 Oh, I didn't understand that the regression was introduced by the revision b9c9691c72c5. The purpose of this revision was to *reduce* the memory usage of the C stack!?
It seems like _PyObject_CallArg1() uses more stack memory than PyObject_CallFunctionObjArgs(). PyObject_CallFunctionObjArgs() allocates 4O bytes (5*sizeof(PyObject*)) on the stack.
At least, I can say that when the crash occurs, _PyObject_FastCallDict() is not the gdb backtrace. serhiy-storchaka commented on Dec 2, 2016 Yes, that is why I asked you to revert your changes.
In additional, they introduced compiler warnings. python-dev commented on Dec 4, 2016 vstinner commented on Dec 4, 2016
Serhiy Storchaka reported that Python 3.6 crashs earlier than Python 3.5 on calling json.dumps() when sys.setrecursionlimit() is increased.
Reference: http://bugs.python.org/issue23507#msg282190 (issue bpo-23507).
Serhiy Storchaka: "Yes, that is why I asked you to revert your changes."
Sorry, I misunderstood your comments. So yes, my change b9c9691c72c5 introduced a regression. Sorry, I didn't have time before now to revert my change. I just pushed the change d35fc6e58a70 which reverts b9c9691c72c5.
The question is how replacing PyObject_CallFunctionObjArgs() with _PyObject_CallArg1() increases the usage of the C stack. I wrote my change to reduce the usage of the C stack.
PyObject_CallFunctionObjArgs() allocates 5 "PyObject *", so 40 bytes, on the C stack. Maybe using _PyObject_CallArg1() increases the usage of C stack in the *caller*.
In additional, they introduced compiler warnings.
This one was fixed by Benjamin Peterson in the issue bpo-28855 (change 96245d4af0ca). serhiy-storchaka commented on Dec 4, 2016 Thanks Victor.
Following script includes several examples of achieving a stack overflow (most are real world examples or can be used in real world examples). It measures maximal deep for every type of recursion. vstinner commented on Dec 4, 2016 When I wrote the _PyObject_CallArg1(), it looks as a cool hack:
#define _PyObject_CallArg1(func, arg) \
_PyObject_FastCall((func), (PyObject **)&(arg), 1)It hacks the declaration of an explicit "stack" like:
PyObject *stack[1];
stack[0] = arg;
res = _PyObject_FastCall(func, stack, 1); And I expected that the C compiler magically computes the memory address of the argument. But it seems like requesting the memory address of an argument allocates something on the C stack.
On x86_64, first function arguments are passed with CPU registers. Maybe requesting the memory address of an argument requires to allocate a local variable, copy the register into the variable, to get the address of the local variable?
So, I suggest to *remove* the _PyObject_CallArg1() macro, and use existing functions like PyObject_CallFunctionObjArgs().
What do you think Serhiy? serhiy-storchaka commented on Dec 5, 2016 That was my initial preference. Mainly because this doesn't add code churn.
But I don't understand how PyObject_CallFunctionObjArgs() that uses _PyObject_CallArg1() and has many local variables can consume less stack than _PyObject_CallArg1() itself. python-dev commented on Dec 5, 2016 vstinner commented on Dec 5, 2016 The changeset 4171cc26272c "Remove _PyObject_CallArg1() macro" fixed the initial bug report, so I now close the issue.
Serhiy: If you see further enhancements, please open a new issue. Your second stack_overflow.py script is interesting, but I don't see any obvious possible changes to enhance results.
Thanks Serhiy for digging into this issue ;-)
MetadataMetadataDevelopmentNo branches or pull requestsIssue actions
|