GeezerVisonThe World Wide Web, restored to 800 pixels of glory. | |||||
| Home|About|Gaming cabinet|Archives|How it works|Starter sites | |||||
GeezerVison is showing github.com — [3.14] gh-139653: Add PyUnstable_ThreadState_SetStackProtection() (GH-139668) by encukou · Pull Request #141661 · python/cpython Conversation
|
|
@hugovk, this will change the internal ABI. |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
|
I would prefer to wait to see how the discussion https://discuss.python.org/t/python-3-14-0-is-incompatible-with-stack-switching-systems-what-do-we-do/104880 goes and see if this API solves the issue, or if we need another solution. |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
|
If this API cannot be used by most projects and a better fix can be designed, I would even suggest to remove this API from the main branch. |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
|
#141404 has been merged causing conflicts. Can you please update your PR to solve conflicts? |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was an error while loading. Please reload this page.
#141711 has been merged: it should fix #139653 for most impacted projects. But according to @markshannon, |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
LGTM
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was an error while loading. Please reload this page.
…protection to fiber stacks (#64772) # Description On Python 3.14 + Linux, every async-actor task permanently leaks ~518 KiB of live malloc (the per-task `asyncio.Task`, `concurrent.futures.Future`, Cython coroutine + scopes, and two msgpack `Packer`s with 256 KiB internal buffers). Closes #63290 ### Root cause **1. CPython 3.14 changed how it avoids stack overflow when freeing objects.** Freeing one object can recursively free many others (a dict frees its values, which free their contents, …), and each level is a nested C call. To keep that from overflowing the C stack, CPython has long had a safety mechanism (the "trashcan"): when it decides it's too deep, it doesn't free the object right away. Instead it parks the object on a per-thread *delete-later* list and drains the list once there's stack headroom again. Up to 3.13, "too deep" was a simple recursion counter. In 3.14 it's decided by comparing the actual machine **stack pointer** against the stack bounds CPython recorded for the thread when it attached (from pthreads, on Linux). **2. Ray async actors don't run task code on the thread's normal stack.** Each task executes on a small 256 KiB boost fiber stack allocated elsewhere in memory. The problem is that CPython still thinks the thread runs on its original pthread stack. So while a task runs on a fiber, every "am I near the stack limit?" check compares the fiber's stack pointer against the *pthread* stack's bounds. On Linux, fiber stacks happen to be allocated at lower addresses than the pthread stack, so CPython concludes the stack is hopelessly overflowed and parks **every** object freed during the task (including return-value serialization and end-of-task cleanup) on the delete-later list. That list is only ever drained by a later free on the same thread state at a healthy stack margin, which never happens here as the Ray thread only runs on fibers and Ray creates a fresh Python thread state per task and destroys it at task end. This means that CPython destroys a thread state **without draining its delete-later list** and the parked objects are orphaned permanently. That's the leak. Why the confusing symptoms: - `boost::make_fcontext` in the issue's flamegraphs just marks *where* the leaked allocations were made (on a fiber stack); the fiber stacks themselves are freed correctly. - macOS is unaffected only by luck: fiber stacks there land at *higher* addresses than the pthread stack, so the check passes. - 3.13 and earlier are unaffected because their trashcan uses the counter, not the stack pointer. ### Fix CPython 3.14.2 added an official API for exactly this situation: `PyUnstable_ThreadState_SetStackProtection` (python/cpython#141661) lets an embedder tell CPython "this thread is currently executing on *this* stack." We call it with the fiber's stack bounds: - at async-actor task entry in `task_execution_handler`, and - whenever a fiber resumes after `YieldCurrentFiber` (concurrent fibers share the thread state, so each must re-register its own stack). With the bounds correct, the near-limit check returns to normal behavior: objects are freed immediately, and the rare genuinely-deep free is parked and then properly drained. Implementation notes: the symbol is looked up via `dlsym`, so `_raylet` still imports on 3.14.0/3.14.1 (fix skipped there; those releases have a more severe, since-fixed stack-check bug anyway, python/cpython#141944). No-op below 3.14 (preprocessor-gated) and on Windows. Stack bounds are derived from the current stack pointer minus a conservative allowance for stack already used, so the protection errs toward triggering slightly early rather than missing an overflow. Side benefit: fibers gain real C-stack overflow protection (RecursionError) on 3.14, which they currently lack entirely (`boost::fibers::fixedsize_stack` has no guard pages). Also makes `FiberState::kStackSize` public so the anchoring uses the real fiber stack size. ## Related issue number Closes #63290. Supersedes #63284 (same diagnosis direction, but hand-rolled `_PyThreadStateImpl` offsets, a deliberate `gilstate_counter` leak that freezes non-main threads, and a crash premise that CPython 3.14.2 already fixed upstream). ## Checks - Verified with a locally built cp314 Linux (aarch64, python:3.14.6 docker) wheel: - refcount probe: **+4.00 refs/task → 0.00/task** (100 tasks) - `__del__` deferral probe: dealloc during return serialization on the fiber **deferred → immediate** - live-malloc probe (`mallinfo2`, 300 tasks/shape): **~518 KiB/task → ~3 KiB/task** across async call → dict/bytes, async generator, sync generator on async actor - reporter-shaped streaming workload (400 tasks, 10 concurrent sessions): live-malloc delta **0.2 MB total**, fiber-sized mapped regions 0 → 0 - async-actor smoke: correctness (echo, state, async generators, recursion), concurrency (20 overlapping 0.5 s sleeps in 0.51 s) - throughput A/B (500 sequential echo tasks, 3 runs fixed / 2 runs baseline, same container image): fixed 5624–6076 tasks/s vs unpatched 4551–4825 tasks/s meaning no regression (the unpatched build is slower while leaking) - baseline (unpatched) wheel from the same tree reproduces the bug: +4.00 refs/task, fiber dealloc deferred=True note: fable did a majority of the heavy lifting in this investigation with prompting on what to check next and validate the solution --------- Signed-off-by: Mark Towers <[email protected]> Signed-off-by: myan <[email protected]> Co-authored-by: Mark Towers <[email protected]> Co-authored-by: myan <[email protected]> Co-authored-by: Mengjin Yan <[email protected]>
#65177) …protection to fiber stacks (#64772) # Description On Python 3.14 + Linux, every async-actor task permanently leaks ~518 KiB of live malloc (the per-task `asyncio.Task`, `concurrent.futures.Future`, Cython coroutine + scopes, and two msgpack `Packer`s with 256 KiB internal buffers). Closes #63290 ### Root cause **1. CPython 3.14 changed how it avoids stack overflow when freeing objects.** Freeing one object can recursively free many others (a dict frees its values, which free their contents, …), and each level is a nested C call. To keep that from overflowing the C stack, CPython has long had a safety mechanism (the "trashcan"): when it decides it's too deep, it doesn't free the object right away. Instead it parks the object on a per-thread *delete-later* list and drains the list once there's stack headroom again. Up to 3.13, "too deep" was a simple recursion counter. In 3.14 it's decided by comparing the actual machine **stack pointer** against the stack bounds CPython recorded for the thread when it attached (from pthreads, on Linux). **2. Ray async actors don't run task code on the thread's normal stack.** Each task executes on a small 256 KiB boost fiber stack allocated elsewhere in memory. The problem is that CPython still thinks the thread runs on its original pthread stack. So while a task runs on a fiber, every "am I near the stack limit?" check compares the fiber's stack pointer against the *pthread* stack's bounds. On Linux, fiber stacks happen to be allocated at lower addresses than the pthread stack, so CPython concludes the stack is hopelessly overflowed and parks **every** object freed during the task (including return-value serialization and end-of-task cleanup) on the delete-later list. That list is only ever drained by a later free on the same thread state at a healthy stack margin, which never happens here as the Ray thread only runs on fibers and Ray creates a fresh Python thread state per task and destroys it at task end. This means that CPython destroys a thread state **without draining its delete-later list** and the parked objects are orphaned permanently. That's the leak. Why the confusing symptoms: - `boost::make_fcontext` in the issue's flamegraphs just marks *where* the leaked allocations were made (on a fiber stack); the fiber stacks themselves are freed correctly. - macOS is unaffected only by luck: fiber stacks there land at *higher* addresses than the pthread stack, so the check passes. - 3.13 and earlier are unaffected because their trashcan uses the counter, not the stack pointer. ### Fix CPython 3.14.2 added an official API for exactly this situation: `PyUnstable_ThreadState_SetStackProtection` (python/cpython#141661) lets an embedder tell CPython "this thread is currently executing on *this* stack." We call it with the fiber's stack bounds: - at async-actor task entry in `task_execution_handler`, and - whenever a fiber resumes after `YieldCurrentFiber` (concurrent fibers share the thread state, so each must re-register its own stack). With the bounds correct, the near-limit check returns to normal behavior: objects are freed immediately, and the rare genuinely-deep free is parked and then properly drained. Implementation notes: the symbol is looked up via `dlsym`, so `_raylet` still imports on 3.14.0/3.14.1 (fix skipped there; those releases have a more severe, since-fixed stack-check bug anyway, python/cpython#141944). No-op below 3.14 (preprocessor-gated) and on Windows. Stack bounds are derived from the current stack pointer minus a conservative allowance for stack already used, so the protection errs toward triggering slightly early rather than missing an overflow. Side benefit: fibers gain real C-stack overflow protection (RecursionError) on 3.14, which they currently lack entirely (`boost::fibers::fixedsize_stack` has no guard pages). Also makes `FiberState::kStackSize` public so the anchoring uses the real fiber stack size. ## Related issue number Closes #63290. Supersedes #63284 (same diagnosis direction, but hand-rolled `_PyThreadStateImpl` offsets, a deliberate `gilstate_counter` leak that freezes non-main threads, and a crash premise that CPython 3.14.2 already fixed upstream). ## Checks - Verified with a locally built cp314 Linux (aarch64, python:3.14.6 docker) wheel: - refcount probe: **+4.00 refs/task → 0.00/task** (100 tasks) - `__del__` deferral probe: dealloc during return serialization on the fiber **deferred → immediate** - live-malloc probe (`mallinfo2`, 300 tasks/shape): **~518 KiB/task → ~3 KiB/task** across async call → dict/bytes, async generator, sync generator on async actor - reporter-shaped streaming workload (400 tasks, 10 concurrent sessions): live-malloc delta **0.2 MB total**, fiber-sized mapped regions 0 → 0 - async-actor smoke: correctness (echo, state, async generators, recursion), concurrency (20 overlapping 0.5 s sleeps in 0.51 s) - throughput A/B (500 sequential echo tasks, 3 runs fixed / 2 runs baseline, same container image): fixed 5624–6076 tasks/s vs unpatched 4551–4825 tasks/s meaning no regression (the unpatched build is slower while leaking) - baseline (unpatched) wheel from the same tree reproduces the bug: +4.00 refs/task, fiber dealloc deferred=True note: fable did a majority of the heavy lifting in this investigation with prompting on what to check next and validate the solution --------- (cherry picked from commit 35591ba) Signed-off-by: Mark Towers <[email protected]> Signed-off-by: myan <[email protected]> Signed-off-by: elliot-barn <[email protected]> Co-authored-by: Mark Towers <[email protected]> Co-authored-by: Mark Towers <[email protected]> Co-authored-by: myan <[email protected]> Co-authored-by: Mengjin Yan <[email protected]>
…protection to fiber stacks (ray-project#64772) # Description On Python 3.14 + Linux, every async-actor task permanently leaks ~518 KiB of live malloc (the per-task `asyncio.Task`, `concurrent.futures.Future`, Cython coroutine + scopes, and two msgpack `Packer`s with 256 KiB internal buffers). Closes ray-project#63290 ### Root cause **1. CPython 3.14 changed how it avoids stack overflow when freeing objects.** Freeing one object can recursively free many others (a dict frees its values, which free their contents, …), and each level is a nested C call. To keep that from overflowing the C stack, CPython has long had a safety mechanism (the "trashcan"): when it decides it's too deep, it doesn't free the object right away. Instead it parks the object on a per-thread *delete-later* list and drains the list once there's stack headroom again. Up to 3.13, "too deep" was a simple recursion counter. In 3.14 it's decided by comparing the actual machine **stack pointer** against the stack bounds CPython recorded for the thread when it attached (from pthreads, on Linux). **2. Ray async actors don't run task code on the thread's normal stack.** Each task executes on a small 256 KiB boost fiber stack allocated elsewhere in memory. The problem is that CPython still thinks the thread runs on its original pthread stack. So while a task runs on a fiber, every "am I near the stack limit?" check compares the fiber's stack pointer against the *pthread* stack's bounds. On Linux, fiber stacks happen to be allocated at lower addresses than the pthread stack, so CPython concludes the stack is hopelessly overflowed and parks **every** object freed during the task (including return-value serialization and end-of-task cleanup) on the delete-later list. That list is only ever drained by a later free on the same thread state at a healthy stack margin, which never happens here as the Ray thread only runs on fibers and Ray creates a fresh Python thread state per task and destroys it at task end. This means that CPython destroys a thread state **without draining its delete-later list** and the parked objects are orphaned permanently. That's the leak. Why the confusing symptoms: - `boost::make_fcontext` in the issue's flamegraphs just marks *where* the leaked allocations were made (on a fiber stack); the fiber stacks themselves are freed correctly. - macOS is unaffected only by luck: fiber stacks there land at *higher* addresses than the pthread stack, so the check passes. - 3.13 and earlier are unaffected because their trashcan uses the counter, not the stack pointer. ### Fix CPython 3.14.2 added an official API for exactly this situation: `PyUnstable_ThreadState_SetStackProtection` (python/cpython#141661) lets an embedder tell CPython "this thread is currently executing on *this* stack." We call it with the fiber's stack bounds: - at async-actor task entry in `task_execution_handler`, and - whenever a fiber resumes after `YieldCurrentFiber` (concurrent fibers share the thread state, so each must re-register its own stack). With the bounds correct, the near-limit check returns to normal behavior: objects are freed immediately, and the rare genuinely-deep free is parked and then properly drained. Implementation notes: the symbol is looked up via `dlsym`, so `_raylet` still imports on 3.14.0/3.14.1 (fix skipped there; those releases have a more severe, since-fixed stack-check bug anyway, python/cpython#141944). No-op below 3.14 (preprocessor-gated) and on Windows. Stack bounds are derived from the current stack pointer minus a conservative allowance for stack already used, so the protection errs toward triggering slightly early rather than missing an overflow. Side benefit: fibers gain real C-stack overflow protection (RecursionError) on 3.14, which they currently lack entirely (`boost::fibers::fixedsize_stack` has no guard pages). Also makes `FiberState::kStackSize` public so the anchoring uses the real fiber stack size. ## Related issue number Closes ray-project#63290. Supersedes ray-project#63284 (same diagnosis direction, but hand-rolled `_PyThreadStateImpl` offsets, a deliberate `gilstate_counter` leak that freezes non-main threads, and a crash premise that CPython 3.14.2 already fixed upstream). ## Checks - Verified with a locally built cp314 Linux (aarch64, python:3.14.6 docker) wheel: - refcount probe: **+4.00 refs/task → 0.00/task** (100 tasks) - `__del__` deferral probe: dealloc during return serialization on the fiber **deferred → immediate** - live-malloc probe (`mallinfo2`, 300 tasks/shape): **~518 KiB/task → ~3 KiB/task** across async call → dict/bytes, async generator, sync generator on async actor - reporter-shaped streaming workload (400 tasks, 10 concurrent sessions): live-malloc delta **0.2 MB total**, fiber-sized mapped regions 0 → 0 - async-actor smoke: correctness (echo, state, async generators, recursion), concurrency (20 overlapping 0.5 s sleeps in 0.51 s) - throughput A/B (500 sequential echo tasks, 3 runs fixed / 2 runs baseline, same container image): fixed 5624–6076 tasks/s vs unpatched 4551–4825 tasks/s meaning no regression (the unpatched build is slower while leaking) - baseline (unpatched) wheel from the same tree reproduces the bug: +4.00 refs/task, fiber dealloc deferred=True note: fable did a majority of the heavy lifting in this investigation with prompting on what to check next and validate the solution --------- Signed-off-by: Mark Towers <[email protected]> Signed-off-by: myan <[email protected]> Co-authored-by: Mark Towers <[email protected]> Co-authored-by: myan <[email protected]> Co-authored-by: Mengjin Yan <[email protected]>
…protection to fiber stacks (ray-project#64772) # Description On Python 3.14 + Linux, every async-actor task permanently leaks ~518 KiB of live malloc (the per-task `asyncio.Task`, `concurrent.futures.Future`, Cython coroutine + scopes, and two msgpack `Packer`s with 256 KiB internal buffers). Closes ray-project#63290 ### Root cause **1. CPython 3.14 changed how it avoids stack overflow when freeing objects.** Freeing one object can recursively free many others (a dict frees its values, which free their contents, …), and each level is a nested C call. To keep that from overflowing the C stack, CPython has long had a safety mechanism (the "trashcan"): when it decides it's too deep, it doesn't free the object right away. Instead it parks the object on a per-thread *delete-later* list and drains the list once there's stack headroom again. Up to 3.13, "too deep" was a simple recursion counter. In 3.14 it's decided by comparing the actual machine **stack pointer** against the stack bounds CPython recorded for the thread when it attached (from pthreads, on Linux). **2. Ray async actors don't run task code on the thread's normal stack.** Each task executes on a small 256 KiB boost fiber stack allocated elsewhere in memory. The problem is that CPython still thinks the thread runs on its original pthread stack. So while a task runs on a fiber, every "am I near the stack limit?" check compares the fiber's stack pointer against the *pthread* stack's bounds. On Linux, fiber stacks happen to be allocated at lower addresses than the pthread stack, so CPython concludes the stack is hopelessly overflowed and parks **every** object freed during the task (including return-value serialization and end-of-task cleanup) on the delete-later list. That list is only ever drained by a later free on the same thread state at a healthy stack margin, which never happens here as the Ray thread only runs on fibers and Ray creates a fresh Python thread state per task and destroys it at task end. This means that CPython destroys a thread state **without draining its delete-later list** and the parked objects are orphaned permanently. That's the leak. Why the confusing symptoms: - `boost::make_fcontext` in the issue's flamegraphs just marks *where* the leaked allocations were made (on a fiber stack); the fiber stacks themselves are freed correctly. - macOS is unaffected only by luck: fiber stacks there land at *higher* addresses than the pthread stack, so the check passes. - 3.13 and earlier are unaffected because their trashcan uses the counter, not the stack pointer. ### Fix CPython 3.14.2 added an official API for exactly this situation: `PyUnstable_ThreadState_SetStackProtection` (python/cpython#141661) lets an embedder tell CPython "this thread is currently executing on *this* stack." We call it with the fiber's stack bounds: - at async-actor task entry in `task_execution_handler`, and - whenever a fiber resumes after `YieldCurrentFiber` (concurrent fibers share the thread state, so each must re-register its own stack). With the bounds correct, the near-limit check returns to normal behavior: objects are freed immediately, and the rare genuinely-deep free is parked and then properly drained. Implementation notes: the symbol is looked up via `dlsym`, so `_raylet` still imports on 3.14.0/3.14.1 (fix skipped there; those releases have a more severe, since-fixed stack-check bug anyway, python/cpython#141944). No-op below 3.14 (preprocessor-gated) and on Windows. Stack bounds are derived from the current stack pointer minus a conservative allowance for stack already used, so the protection errs toward triggering slightly early rather than missing an overflow. Side benefit: fibers gain real C-stack overflow protection (RecursionError) on 3.14, which they currently lack entirely (`boost::fibers::fixedsize_stack` has no guard pages). Also makes `FiberState::kStackSize` public so the anchoring uses the real fiber stack size. ## Related issue number Closes ray-project#63290. Supersedes ray-project#63284 (same diagnosis direction, but hand-rolled `_PyThreadStateImpl` offsets, a deliberate `gilstate_counter` leak that freezes non-main threads, and a crash premise that CPython 3.14.2 already fixed upstream). ## Checks - Verified with a locally built cp314 Linux (aarch64, python:3.14.6 docker) wheel: - refcount probe: **+4.00 refs/task → 0.00/task** (100 tasks) - `__del__` deferral probe: dealloc during return serialization on the fiber **deferred → immediate** - live-malloc probe (`mallinfo2`, 300 tasks/shape): **~518 KiB/task → ~3 KiB/task** across async call → dict/bytes, async generator, sync generator on async actor - reporter-shaped streaming workload (400 tasks, 10 concurrent sessions): live-malloc delta **0.2 MB total**, fiber-sized mapped regions 0 → 0 - async-actor smoke: correctness (echo, state, async generators, recursion), concurrency (20 overlapping 0.5 s sleeps in 0.51 s) - throughput A/B (500 sequential echo tasks, 3 runs fixed / 2 runs baseline, same container image): fixed 5624–6076 tasks/s vs unpatched 4551–4825 tasks/s meaning no regression (the unpatched build is slower while leaking) - baseline (unpatched) wheel from the same tree reproduces the bug: +4.00 refs/task, fiber dealloc deferred=True note: fable did a majority of the heavy lifting in this investigation with prompting on what to check next and validate the solution --------- Signed-off-by: Mark Towers <[email protected]> Signed-off-by: myan <[email protected]> Co-authored-by: Mark Towers <[email protected]> Co-authored-by: myan <[email protected]> Co-authored-by: Mengjin Yan <[email protected]> Signed-off-by: 400Ping <[email protected]>
…protection to fiber stacks (ray-project#64772) # Description On Python 3.14 + Linux, every async-actor task permanently leaks ~518 KiB of live malloc (the per-task `asyncio.Task`, `concurrent.futures.Future`, Cython coroutine + scopes, and two msgpack `Packer`s with 256 KiB internal buffers). Closes ray-project#63290 ### Root cause **1. CPython 3.14 changed how it avoids stack overflow when freeing objects.** Freeing one object can recursively free many others (a dict frees its values, which free their contents, …), and each level is a nested C call. To keep that from overflowing the C stack, CPython has long had a safety mechanism (the "trashcan"): when it decides it's too deep, it doesn't free the object right away. Instead it parks the object on a per-thread *delete-later* list and drains the list once there's stack headroom again. Up to 3.13, "too deep" was a simple recursion counter. In 3.14 it's decided by comparing the actual machine **stack pointer** against the stack bounds CPython recorded for the thread when it attached (from pthreads, on Linux). **2. Ray async actors don't run task code on the thread's normal stack.** Each task executes on a small 256 KiB boost fiber stack allocated elsewhere in memory. The problem is that CPython still thinks the thread runs on its original pthread stack. So while a task runs on a fiber, every "am I near the stack limit?" check compares the fiber's stack pointer against the *pthread* stack's bounds. On Linux, fiber stacks happen to be allocated at lower addresses than the pthread stack, so CPython concludes the stack is hopelessly overflowed and parks **every** object freed during the task (including return-value serialization and end-of-task cleanup) on the delete-later list. That list is only ever drained by a later free on the same thread state at a healthy stack margin, which never happens here as the Ray thread only runs on fibers and Ray creates a fresh Python thread state per task and destroys it at task end. This means that CPython destroys a thread state **without draining its delete-later list** and the parked objects are orphaned permanently. That's the leak. Why the confusing symptoms: - `boost::make_fcontext` in the issue's flamegraphs just marks *where* the leaked allocations were made (on a fiber stack); the fiber stacks themselves are freed correctly. - macOS is unaffected only by luck: fiber stacks there land at *higher* addresses than the pthread stack, so the check passes. - 3.13 and earlier are unaffected because their trashcan uses the counter, not the stack pointer. ### Fix CPython 3.14.2 added an official API for exactly this situation: `PyUnstable_ThreadState_SetStackProtection` (python/cpython#141661) lets an embedder tell CPython "this thread is currently executing on *this* stack." We call it with the fiber's stack bounds: - at async-actor task entry in `task_execution_handler`, and - whenever a fiber resumes after `YieldCurrentFiber` (concurrent fibers share the thread state, so each must re-register its own stack). With the bounds correct, the near-limit check returns to normal behavior: objects are freed immediately, and the rare genuinely-deep free is parked and then properly drained. Implementation notes: the symbol is looked up via `dlsym`, so `_raylet` still imports on 3.14.0/3.14.1 (fix skipped there; those releases have a more severe, since-fixed stack-check bug anyway, python/cpython#141944). No-op below 3.14 (preprocessor-gated) and on Windows. Stack bounds are derived from the current stack pointer minus a conservative allowance for stack already used, so the protection errs toward triggering slightly early rather than missing an overflow. Side benefit: fibers gain real C-stack overflow protection (RecursionError) on 3.14, which they currently lack entirely (`boost::fibers::fixedsize_stack` has no guard pages). Also makes `FiberState::kStackSize` public so the anchoring uses the real fiber stack size. ## Related issue number Closes ray-project#63290. Supersedes ray-project#63284 (same diagnosis direction, but hand-rolled `_PyThreadStateImpl` offsets, a deliberate `gilstate_counter` leak that freezes non-main threads, and a crash premise that CPython 3.14.2 already fixed upstream). ## Checks - Verified with a locally built cp314 Linux (aarch64, python:3.14.6 docker) wheel: - refcount probe: **+4.00 refs/task → 0.00/task** (100 tasks) - `__del__` deferral probe: dealloc during return serialization on the fiber **deferred → immediate** - live-malloc probe (`mallinfo2`, 300 tasks/shape): **~518 KiB/task → ~3 KiB/task** across async call → dict/bytes, async generator, sync generator on async actor - reporter-shaped streaming workload (400 tasks, 10 concurrent sessions): live-malloc delta **0.2 MB total**, fiber-sized mapped regions 0 → 0 - async-actor smoke: correctness (echo, state, async generators, recursion), concurrency (20 overlapping 0.5 s sleeps in 0.51 s) - throughput A/B (500 sequential echo tasks, 3 runs fixed / 2 runs baseline, same container image): fixed 5624–6076 tasks/s vs unpatched 4551–4825 tasks/s meaning no regression (the unpatched build is slower while leaking) - baseline (unpatched) wheel from the same tree reproduces the bug: +4.00 refs/task, fiber dealloc deferred=True note: fable did a majority of the heavy lifting in this investigation with prompting on what to check next and validate the solution --------- Signed-off-by: Mark Towers <[email protected]> Signed-off-by: myan <[email protected]> Co-authored-by: Mark Towers <[email protected]> Co-authored-by: myan <[email protected]> Co-authored-by: Mengjin Yan <[email protected]>
There was an error while loading. Please reload this page.
Successfully merging this pull request may close these issues.
There was an error while loading. Please reload this page.