GeezerVison is showing github.com — gh-139653: Add PyUnstable_ThreadState_SetStackProtection() by vstinner · Pull Request #139668 · python/cpython
gh-139653: Add PyUnstable_ThreadState_SetStackProtection()
#139668
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and
privacy statement . We’ll occasionally send you account related emails.
Already on GitHub?
Sign in
to your account
Uh oh!
There was an error while loading. Please reload this page.
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
13 commits
Select commit
Hold shift + click to select a range
14c66bd
gh-139653: Add PyUnstable_ThreadState_SetStack()
vstinner Oct 6, 2025
5d157ff
Update the doc
vstinner Oct 7, 2025
b0d1cf3
Fix TSan: use larger minimum stack size
vstinner Oct 7, 2025
b8672c8
Update Doc/c-api/init.rst
vstinner Oct 13, 2025
8290679
Update Doc/c-api/init.rst
vstinner Oct 13, 2025
b85c8b8
Mention Py_EnterRecursiveCall()
vstinner Oct 13, 2025
da8cbf0
Py_EnterRecursiveCall(): add seealso
vstinner Oct 13, 2025
2f7b92e
Update Doc/c-api/init.rst
vstinner Oct 15, 2025
34d9153
Rename to PyUnstable_ThreadState_SetStackProtection()
vstinner Nov 5, 2025
dc0dc50
Merge branch 'main' into setstack
vstinner Nov 5, 2025
f7e348b
Rename the unit test
vstinner Nov 5, 2025
0a8ba2d
Repeat "protection" in the documentation
vstinner Nov 5, 2025
1987f62
Merge branch 'main' into setstack
vstinner Nov 5, 2025
File filter
Filter by extension
.c
(3)
.h
(3)
.rst
(4)
All 3 file types selected
Viewed files
Conversations
Failed to load comments.
Loading
Uh oh!
There was an error while loading. Please reload this page.
Jump to
Jump to file
Failed to load files.
Loading
Uh oh!
There was an error while loading. Please reload this page.
Diff view
Hide whitespace
Diff view
Hide whitespace
Some comments aren't visible on the classic Files Changed page.
There are no files selected for viewing
Original file line number
Diff line number
Diff line change
Expand Up
@@ -939,6 +939,9 @@ because the :ref:`call protocol <call>` takes care of recursion handling.
be concatenated to the :exc:`RecursionError` message caused by the recursion
depth limit.
.. seealso::
The :c:func:`PyUnstable_ThreadState_SetStackProtection` function.
.. versionchanged:: 3.9
This function is now also available in the :ref:`limited API <limited-c-api>`.
Expand Down
Original file line number
Diff line number
Diff line change
Expand Up
@@ -1366,6 +1366,43 @@ All of the following functions must be called after :c:func:`Py_Initialize`.
.. versionadded:: 3.11
.. c:function:: int PyUnstable_ThreadState_SetStackProtection(PyThreadState *tstate, void *stack_start_addr, size_t stack_size)
Set the stack protection start address and stack protection size
of a Python thread state.
On success, return ``0``.
On failure, set an exception and return ``-1``.
CPython implements :ref:`recursion control <recursion>` for C code by raising
:py:exc:`RecursionError` when it notices that the machine execution stack is close
to overflow. See for example the :c:func:`Py_EnterRecursiveCall` function.
For this, it needs to know the location of the current thread's stack, which it
normally gets from the operating system.
When the stack is changed, for example using context switching techniques like the
Boost library's ``boost::context``, you must call
:c:func:`~PyUnstable_ThreadState_SetStackProtection` to inform CPython of the change.
Call :c:func:`~PyUnstable_ThreadState_SetStackProtection` either before
or after changing the stack.
Do not call any other Python C API between the call and the stack
change.
See :c:func:`PyUnstable_ThreadState_ResetStackProtection` for undoing this operation.
.. versionadded:: next
.. c:function:: void PyUnstable_ThreadState_ResetStackProtection(PyThreadState *tstate)
Reset the stack protection start address and stack protection size
of a Python thread state to the operating system defaults.
See :c:func:`PyUnstable_ThreadState_SetStackProtection` for an explanation.
.. versionadded:: next
.. c:function:: PyInterpreterState* PyInterpreterState_Get(void)
Get the current interpreter.
Expand Down
Original file line number
Diff line number
Diff line change
Expand Up
@@ -950,6 +950,12 @@ New features
* Add :c:func:`PyTuple_FromArray` to create a :class:`tuple` from an array.
(Contributed by Victor Stinner in :gh:`111489`.)
* Add :c:func:`PyUnstable_ThreadState_SetStackProtection` and
:c:func:`PyUnstable_ThreadState_ResetStackProtection` functions to set
the stack protection base address and stack protection size of a Python
thread state.
(Contributed by Victor Stinner in :gh:`139653`.)
Changed C APIs
--------------
Expand Down
Original file line number
Diff line number
Diff line change
Expand Up
@@ -276,6 +276,18 @@ PyAPI_FUNC(int) PyGILState_Check(void);
*/
PyAPI_FUNC(PyObject*) _PyThread_CurrentFrames(void);
// Set the stack protection start address and stack protection size
// of a Python thread state
PyAPI_FUNC(int) PyUnstable_ThreadState_SetStackProtection(
PyThreadState *tstate,
void *stack_start_addr, // Stack start address
size_t stack_size); // Stack size (in bytes)
// Reset the stack protection start address and stack protection size
// of a Python thread state
PyAPI_FUNC(void) PyUnstable_ThreadState_ResetStackProtection(
PyThreadState *tstate);
/* Routines for advanced debuggers, requested by David Beazley.
Don't use unless you know what you are doing! */
PyAPI_FUNC(PyInterpreterState *) PyInterpreterState_Main(void);
Expand Down
Original file line number
Diff line number
Diff line change
Expand Up
@@ -54,6 +54,12 @@ extern const char* _Py_SourceAsString(
# define _PyOS_STACK_MARGIN_SHIFT (_PyOS_LOG2_STACK_MARGIN + 2)
#endif
#ifdef _Py_THREAD_SANITIZER
# define _PyOS_MIN_STACK_SIZE (_PyOS_STACK_MARGIN_BYTES * 6)
#else
# define _PyOS_MIN_STACK_SIZE (_PyOS_STACK_MARGIN_BYTES * 3)
#endif
#ifdef __cplusplus
}
Expand Down
Original file line number
Diff line number
Diff line change
Expand Up
@@ -37,6 +37,10 @@ typedef struct _PyThreadStateImpl {
uintptr_t c_stack_soft_limit;
uintptr_t c_stack_hard_limit;
// PyUnstable_ThreadState_ResetStackProtection() values
uintptr_t c_stack_init_base;
uintptr_t c_stack_init_top;
PyObject *asyncio_running_loop; // Strong reference
PyObject *asyncio_running_task; // Strong reference
Expand Down
Original file line number
Diff line number
Diff line change
@@ -0,0 +1,4 @@
Add :c:func:`PyUnstable_ThreadState_SetStackProtection` and
:c:func:`PyUnstable_ThreadState_ResetStackProtection` functions to set the
stack protection base address and stack protection size of a Python thread
state. Patch by Victor Stinner.
Original file line number
Diff line number
Diff line change
Expand Up
@@ -2446,6 +2446,58 @@ module_get_gc_hooks(PyObject *self, PyObject *arg)
return result;
}
static void
check_threadstate_set_stack_protection(PyThreadState *tstate,
void *start, size_t size)
{
assert(PyUnstable_ThreadState_SetStackProtection(tstate, start, size) == 0);
assert(!PyErr_Occurred());
_PyThreadStateImpl *ts = (_PyThreadStateImpl *)tstate;
assert(ts->c_stack_top == (uintptr_t)start + size);
assert(ts->c_stack_hard_limit <= ts->c_stack_soft_limit);
assert(ts->c_stack_soft_limit < ts->c_stack_top);
}
static PyObject *
test_threadstate_set_stack_protection(PyObject *self, PyObject *Py_UNUSED(args))
{
PyThreadState *tstate = PyThreadState_GET();
_PyThreadStateImpl *ts = (_PyThreadStateImpl *)tstate;
assert(!PyErr_Occurred());
uintptr_t init_base = ts->c_stack_init_base;
size_t init_top = ts->c_stack_init_top;
// Test the minimum stack size
size_t size = _PyOS_MIN_STACK_SIZE;
void *start = (void*)(_Py_get_machine_stack_pointer() - size);
check_threadstate_set_stack_protection(tstate, start, size);
// Test a larger size
size = 7654321;
assert(size > _PyOS_MIN_STACK_SIZE);
start = (void*)(_Py_get_machine_stack_pointer() - size);
check_threadstate_set_stack_protection(tstate, start, size);
// Test invalid size (too small)
size = 5;
start = (void*)(_Py_get_machine_stack_pointer() - size);
assert(PyUnstable_ThreadState_SetStackProtection(tstate, start, size) == -1);
assert(PyErr_ExceptionMatches(PyExc_ValueError));
PyErr_Clear();
// Test PyUnstable_ThreadState_ResetStackProtection()
PyUnstable_ThreadState_ResetStackProtection(tstate);
assert(ts->c_stack_init_base == init_base);
assert(ts->c_stack_init_top == init_top);
Py_RETURN_NONE;
}
static PyMethodDef module_functions[] = {
{"get_configs", get_configs, METH_NOARGS},
{"get_recursion_depth", get_recursion_depth, METH_NOARGS},
Expand Down
Expand Up
@@ -2556,6 +2608,8 @@ static PyMethodDef module_functions[] = {
{"simple_pending_call", simple_pending_call, METH_O},
{"set_vectorcall_nop", set_vectorcall_nop, METH_O},
{"module_get_gc_hooks", module_get_gc_hooks, METH_O},
{"test_threadstate_set_stack_protection",
test_threadstate_set_stack_protection, METH_NOARGS},
{NULL, NULL} /* sentinel */
};
Expand Down
Original file line number
Diff line number
Diff line change
Expand Up
@@ -443,7 +443,7 @@ int pthread_attr_destroy(pthread_attr_t *a)
#endif
static void
hardware_stack_limits(uintptr_t *top , uintptr_t *base )
hardware_stack_limits(uintptr_t *base , uintptr_t *top )
{
#ifdef WIN32
ULONG_PTR low, high;
Expand Down
Expand Up
@@ -486,23 +486,86 @@ hardware_stack_limits(uintptr_t *top, uintptr_t *base)
#endif
}
void
_Py_InitializeRecursionLimits(PyThreadState *tstate)
static void
tstate_set_stack(PyThreadState *tstate,
uintptr_t base, uintptr_t top)
{
uintptr_t top;
uintptr_t base;
hardware_stack_limits(&top, &base);
assert(base < top) ;
assert((top - base) >= _PyOS_MIN_STACK_SIZE) ;
#ifdef _Py_THREAD_SANITIZER
// Thread sanitizer crashes if we use more than half the stack.
uintptr_t stacksize = top - base;
base += stacksize/ 2;
base += stacksize / 2;
#endif
_PyThreadStateImpl *_tstate = (_PyThreadStateImpl *)tstate;
_tstate->c_stack_top = top;
_tstate->c_stack_hard_limit = base + _PyOS_STACK_MARGIN_BYTES;
_tstate->c_stack_soft_limit = base + _PyOS_STACK_MARGIN_BYTES * 2;
#ifndef NDEBUG
// Sanity checks
_PyThreadStateImpl *ts = (_PyThreadStateImpl *)tstate;
assert(ts->c_stack_hard_limit <= ts->c_stack_soft_limit);
Comment thread
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more .
Be aware that this is going to conflict with #140028
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
assert(ts->c_stack_soft_limit < ts->c_stack_top);
#endif
}
void
_Py_InitializeRecursionLimits(PyThreadState *tstate)
{
uintptr_t base, top;
hardware_stack_limits(&base, &top);
assert(top != 0);
tstate_set_stack(tstate, base, top);
_PyThreadStateImpl *ts = (_PyThreadStateImpl *)tstate;
ts->c_stack_init_base = base;
ts->c_stack_init_top = top;
// Test the stack pointer
#if !defined(NDEBUG) && !defined(__wasi__)
uintptr_t here_addr = _Py_get_machine_stack_pointer();
assert(ts->c_stack_soft_limit < here_addr);
assert(here_addr < ts->c_stack_top);
#endif
}
int
PyUnstable_ThreadState_SetStackProtection(PyThreadState *tstate,
void *stack_start_addr, size_t stack_size)
{
if (stack_size < _PyOS_MIN_STACK_SIZE) {
PyErr_Format(PyExc_ValueError,
"stack_size must be at least %zu bytes",
_PyOS_MIN_STACK_SIZE);
return -1;
}
uintptr_t base = (uintptr_t)stack_start_addr;
uintptr_t top = base + stack_size;
tstate_set_stack(tstate, base, top);
return 0;
}
void
PyUnstable_ThreadState_ResetStackProtection(PyThreadState *tstate)
{
_PyThreadStateImpl *ts = (_PyThreadStateImpl *)tstate;
if (ts->c_stack_init_top != 0) {
tstate_set_stack(tstate,
ts->c_stack_init_base,
ts->c_stack_init_top);
return;
}
_Py_InitializeRecursionLimits(tstate);
}
/* The function _Py_EnterRecursiveCallTstate() only calls _Py_CheckRecursiveCall()
if the recursion_depth reaches recursion_limit. */
int
Expand Down
Original file line number
Diff line number
Diff line change
Expand Up
@@ -1495,6 +1495,9 @@ init_threadstate(_PyThreadStateImpl *_tstate,
_tstate->c_stack_top = 0;
_tstate->c_stack_hard_limit = 0;
_tstate->c_stack_init_base = 0;
_tstate->c_stack_init_top = 0;
_tstate->asyncio_running_loop = NULL;
_tstate->asyncio_running_task = NULL;
Expand Down
Loading
Oops, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.