GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — Clear frame locals and stack on generator close + Add dir_fd support for rmdir, remove/unlink, scandir by youknowone · Pull Request #7222 · RustPython/RustPython

Clear frame locals and stack on generator close + Add dir_fd support for rmdir, remove/unlink, scandir #7222

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
youknowone merged 3 commits into RustPython:main from youknowone:gen-os
Feb 26, 2026

Uh oh!

There was an error while loading. Please reload this page.
Merged

Clear frame locals and stack on generator close + Add dir_fd support for rmdir, remove/unlink, scandir #7222

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
055584c
Clear frame locals and stack on generator close
youknowone Feb 25, 2026
7e7832d
Add dir_fd support for rmdir, remove/unlink, scandir
youknowone Feb 24, 2026
a25d565
cells_free
youknowone Feb 26, 2026
File filter
Filter by extension
Clear filters
Conversations
Failed to load comments.
Loading

Uh oh!

There was an error while loading. Please reload this page.
Jump to
Jump to file
Failed to load files.
Loading

Uh oh!

There was an error while loading. Please reload this page.
Diff view
Diff view
Prev Previous commit
cells_free
  • Loading branch information

Uh oh!

There was an error while loading. Please reload this page.

@youknowone
youknowone committed Feb 26, 2026
commit a25d565b8932cb09d24387261a492012aa2950db

Some comments aren't visible on the classic Files Changed page.

There are no files selected for viewing

4 changes: 2 additions & 2 deletions 4 crates/vm/src/builtins/frame.rs
Show comments
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -179,8 +179,8 @@ impl Py<Frame> {
}
}

// Clear the evaluation stack
self.clear_value_stack();
// Clear the evaluation stack and cell references
self.clear_stack_and_cells();

// Clear temporary refs
self.temporary_refs.lock().clear();
Expand Down
2 changes: 1 addition & 1 deletion 2 crates/vm/src/builtins/function.rs
Show comments
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -431,7 +431,7 @@ impl PyFunction {
if let Some(cell2arg) = code.cell2arg.as_deref() {
for (cell_idx, arg_idx) in cell2arg.iter().enumerate().filter(|(_, i)| **i != -1) {
let x = fastlocals[*arg_idx as usize].take();
frame.cells_frees[cell_idx].set(x);
frame.set_cell_contents(cell_idx, x);
}
}

Expand Down
12 changes: 7 additions & 5 deletions 12 crates/vm/src/builtins/super.rs
Show comments
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -85,15 +85,17 @@ impl Initializer for PySuper {
if frame.code.arg_count == 0 {
return Err(vm.new_runtime_error("super(): no arguments"));
}
let obj = frame.fastlocals.lock()[0]
.clone()
// Lock fastlocals briefly to get arg[0], then drop the guard
// before calling get_cell_contents (which also locks fastlocals).
let first_arg = frame.fastlocals.lock()[0].clone();
let obj = first_arg
.or_else(|| {
if let Some(cell2arg) = frame.code.cell2arg.as_deref() {
cell2arg[..frame.code.cellvars.len()]
.iter()
.enumerate()
.find(|(_, arg_idx)| **arg_idx == 0)
.and_then(|(cell_idx, _)| frame.cells_frees[cell_idx].get())
.and_then(|(cell_idx, _)| frame.get_cell_contents(cell_idx))
} else {
None
}
Expand All @@ -104,8 +106,8 @@ impl Initializer for PySuper {
for (i, var) in frame.code.freevars.iter().enumerate() {
if var.as_bytes() == b"__class__" {
let i = frame.code.cellvars.len() + i;
let class = frame.cells_frees[i]
.get()
let class = frame
.get_cell_contents(i)
.ok_or_else(|| vm.new_runtime_error("super(): empty __class__ cell"))?;
typ = Some(class.downcast().map_err(|o| {
vm.new_type_error(format!(
Expand Down
77 changes: 48 additions & 29 deletions 77 crates/vm/src/frame.rs
Show comments
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -53,6 +53,8 @@ struct FrameState {
// We need 1 stack per frame
/// The main data frame of the stack machine
stack: BoxVec<Option<PyObjectRef>>,
/// Cell and free variable references (cellvars + freevars).
cells_frees: Box<[PyCellRef]>,
/// index of last instruction ran
#[cfg(feature = "threading")]
lasti: u32,
Expand Down Expand Up @@ -93,7 +95,6 @@ pub struct Frame {
pub func_obj: Option<PyObjectRef>,

pub fastlocals: PyMutex<Box<[Option<PyObjectRef>]>>,
pub(crate) cells_frees: Box<[PyCellRef]>,
pub locals: ArgMapping,
pub globals: PyDictRef,
pub builtins: PyObjectRef,
Expand Down Expand Up @@ -135,6 +136,7 @@ impl PyPayload for Frame {
unsafe impl Traverse for FrameState {
fn traverse(&self, tracer_fn: &mut TraverseFn<'_>) {
self.stack.traverse(tracer_fn);
self.cells_frees.traverse(tracer_fn);
}
}

Expand All @@ -143,7 +145,6 @@ unsafe impl Traverse for Frame {
self.code.traverse(tracer_fn);
self.func_obj.traverse(tracer_fn);
self.fastlocals.traverse(tracer_fn);
self.cells_frees.traverse(tracer_fn);
self.locals.traverse(tracer_fn);
self.globals.traverse(tracer_fn);
self.builtins.traverse(tracer_fn);
Expand Down Expand Up @@ -193,13 +194,13 @@ impl Frame {

let state = FrameState {
stack: BoxVec::new(code.max_stackdepth as usize),
cells_frees,
#[cfg(feature = "threading")]
lasti: 0,
};

Self {
fastlocals: PyMutex::new(fastlocals_vec.into_boxed_slice()),
cells_frees,
locals: scope.locals,
globals: scope.globals,
builtins,
Expand All @@ -218,21 +219,40 @@ impl Frame {
}
}

/// Clear the evaluation stack. Used by frame.clear() to break reference cycles.
pub(crate) fn clear_value_stack(&self) {
self.state.lock().stack.clear();
/// Clear evaluation stack and state-owned cell/free references.
/// For full local/cell cleanup, call `clear_locals_and_stack()`.
pub(crate) fn clear_stack_and_cells(&self) {
let mut state = self.state.lock();
state.stack.clear();
let _old = core::mem::take(&mut state.cells_frees);
}

/// Clear locals and stack after generator/coroutine close.
/// Releases references held by the frame, matching _PyFrame_ClearLocals.
pub(crate) fn clear_locals_and_stack(&self) {
self.state.lock().stack.clear();
self.clear_stack_and_cells();
let mut fastlocals = self.fastlocals.lock();
for slot in fastlocals.iter_mut() {
*slot = None;
}
}
Comment thread
coderabbitai[bot] marked this conversation as resolved.
Resolved

Uh oh!

There was an error while loading. Please reload this page.

/// Get cell contents by cell index. Reads through fastlocals (no state lock needed).
pub(crate) fn get_cell_contents(&self, cell_idx: usize) -> Option<PyObjectRef> {
let nlocals = self.code.varnames.len();
let fastlocals = self.fastlocals.lock();
fastlocals
.get(nlocals + cell_idx)
.and_then(|slot| slot.as_ref())
.and_then(|obj| obj.downcast_ref::<PyCell>())
.and_then(|cell| cell.get())
}

/// Set cell contents by cell index. Only safe to call before frame execution starts.
pub(crate) fn set_cell_contents(&self, cell_idx: usize, value: Option<PyObjectRef>) {
self.state.lock().cells_frees[cell_idx].set(value);
}

/// Store a borrowed back-reference to the owning generator/coroutine.
/// The caller must ensure the generator outlives the frame.
pub fn set_generator(&self, generator: &PyObject) {
Expand Down Expand Up @@ -306,23 +326,25 @@ impl Frame {
}
}
if !code.cellvars.is_empty() || !code.freevars.is_empty() {
let map_to_dict = |keys: &[&PyStrInterned], values: &[PyCellRef]| {
for (&k, v) in zip(keys, values) {
if let Some(value) = v.get() {
locals.mapping().ass_subscript(k, Some(value), vm)?;
} else {
match locals.mapping().ass_subscript(k, None, vm) {
Ok(()) => {}
Err(e) if e.fast_isinstance(vm.ctx.exceptions.key_error) => {}
Err(e) => return Err(e),
}
}
// Access cells through fastlocals to avoid locking state
// (state may be held by with_exec during frame execution).
for (i, &k) in code.cellvars.iter().enumerate() {
let cell_value = self.get_cell_contents(i);
match locals.mapping().ass_subscript(k, cell_value, vm) {
Ok(()) => {}
Err(e) if e.fast_isinstance(vm.ctx.exceptions.key_error) => {}
Err(e) => return Err(e),
}
Ok(())
};
map_to_dict(&code.cellvars, &self.cells_frees)?;
}
if code.flags.contains(bytecode::CodeFlags::OPTIMIZED) {
map_to_dict(&code.freevars, &self.cells_frees[code.cellvars.len()..])?;
for (i, &k) in code.freevars.iter().enumerate() {
let cell_value = self.get_cell_contents(code.cellvars.len() + i);
match locals.mapping().ass_subscript(k, cell_value, vm) {
Ok(()) => {}
Err(e) if e.fast_isinstance(vm.ctx.exceptions.key_error) => {}
Err(e) => return Err(e),
}
}
}
}
Ok(locals.clone())
Expand All @@ -336,7 +358,6 @@ impl Py<Frame> {
let exec = ExecutingFrame {
code: &self.code,
fastlocals: &self.fastlocals,
cells_frees: &self.cells_frees,
locals: &self.locals,
globals: &self.globals,
builtins: &self.builtins,
Expand Down Expand Up @@ -382,7 +403,6 @@ impl Py<Frame> {
let exec = ExecutingFrame {
code: &self.code,
fastlocals: &self.fastlocals,
cells_frees: &self.cells_frees,
locals: &self.locals,
globals: &self.globals,
builtins: &self.builtins,
Expand Down Expand Up @@ -417,7 +437,6 @@ impl Py<Frame> {
struct ExecutingFrame<'a> {
code: &'a PyRef<PyCode>,
fastlocals: &'a PyMutex<Box<[Option<PyObjectRef>]>>,
cells_frees: &'a [PyCellRef],
locals: &'a ArgMapping,
globals: &'a PyDictRef,
builtins: &'a PyObjectRef,
Expand Down Expand Up @@ -1021,7 +1040,7 @@ impl ExecutingFrame<'_> {
}
Instruction::DeleteAttr { idx } => self.delete_attr(vm, idx.get(arg)),
Instruction::DeleteDeref(i) => {
self.cells_frees[i.get(arg) as usize].set(None);
self.state.cells_frees[i.get(arg) as usize].set(None);
Ok(None)
}
Instruction::DeleteFast(idx) => {
Expand Down Expand Up @@ -1446,7 +1465,7 @@ impl ExecutingFrame<'_> {
};
self.push_value(match value {
Some(v) => v,
None => self.cells_frees[i]
None => self.state.cells_frees[i]
.get()
.ok_or_else(|| self.unbound_cell_exception(i, vm))?,
});
Expand Down Expand Up @@ -1503,7 +1522,7 @@ impl ExecutingFrame<'_> {
}
Instruction::LoadDeref(i) => {
let idx = i.get(arg) as usize;
let x = self.cells_frees[idx]
let x = self.state.cells_frees[idx]
.get()
.ok_or_else(|| self.unbound_cell_exception(idx, vm))?;
self.push_value(x);
Expand Down Expand Up @@ -2093,7 +2112,7 @@ impl ExecutingFrame<'_> {
Instruction::StoreAttr { idx } => self.store_attr(vm, idx.get(arg)),
Instruction::StoreDeref(i) => {
let value = self.pop_value();
self.cells_frees[i.get(arg) as usize].set(Some(value));
self.state.cells_frees[i.get(arg) as usize].set(Some(value));
Ok(None)
}
Instruction::StoreFast(idx) => {
Expand Down
24 changes: 17 additions & 7 deletions 24 crates/vm/src/stdlib/os.rs
Show comments
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,10 +2,10 @@

use crate::{
AsObject, Py, PyObjectRef, PyPayload, PyResult, TryFromObject, VirtualMachine,
builtins::{PyDictRef, PyModule, PySet},
builtins::{PyModule, PySet},
common::crt_fd,
convert::{IntoPyException, ToPyException, ToPyObject},
function::{ArgMapping, ArgumentError, FromArgs, FuncArgs},
function::{ArgumentError, FromArgs, FuncArgs},
};
use std::{fs, io, path::Path};

Expand Down Expand Up @@ -367,9 +367,9 @@ pub(super) mod _os {
dir_fd: DirFd<'_, { RMDIR_DIR_FD as usize }>,
vm: &VirtualMachine,
) -> PyResult<()> {
let c_path = path.clone().into_cstring(vm)?;
#[cfg(not(target_os = "redox"))]
if let Some(fd) = dir_fd.raw_opt() {
let c_path = path.clone().into_cstring(vm)?;
let res = unsafe { libc::unlinkat(fd, c_path.as_ptr(), libc::AT_REMOVEDIR) };
return if res < 0 {
let err = crate::common::os::errno_io_error();
Expand Down Expand Up @@ -434,8 +434,7 @@ pub(super) mod _os {
unsafe { libc::close(raw_fd) };
e.into_pyexception(vm)
})?;
// OwnedDir::drop calls rewinddir then closedir,
// restoring the original fd's directory position.
// OwnedDir::drop calls rewinddir (reset to start) then closedir.
let mut list = Vec::new();
loop {
nix::errno::Errno::clear();
Expand Down Expand Up @@ -736,7 +735,14 @@ pub(super) mod _os {
#[cfg(unix)]
return self.test_mode_via_stat(false, libc::S_IFLNK as _, vm);
#[cfg(not(unix))]
Err(io::Error::other("file_type unavailable").into_pyexception(vm))
match &self.file_type {
Err(e) if e.kind() == io::ErrorKind::NotFound => Ok(false),
Err(e) => {
use crate::convert::ToPyException;
Err(e.to_pyexception(vm))
}
Ok(_) => Ok(false),
}
}

#[pymethod]
Expand Down Expand Up @@ -2341,7 +2347,11 @@ pub fn module_exec(vm: &VirtualMachine, module: &Py<PyModule>) -> PyResult<()> {
/// For `os._Environ`, accesses the internal `_data` dict directly at the Rust level.
/// This avoids Python-level method calls that can deadlock after fork() when
/// parking_lot locks are held by threads that no longer exist.
pub(crate) fn envobj_to_dict(env: ArgMapping, vm: &VirtualMachine) -> PyResult<PyDictRef> {
#[cfg(any(unix, windows))]
pub(crate) fn envobj_to_dict(
env: crate::function::ArgMapping,
vm: &VirtualMachine,
) -> PyResult<crate::builtins::PyDictRef> {
let obj = env.obj();
if let Some(dict) = obj.downcast_ref_if_exact::<crate::builtins::PyDict>(vm) {
return Ok(dict.to_owned());
Expand Down
2 changes: 1 addition & 1 deletion 2 crates/vm/src/stdlib/posix.rs
Show comments
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -512,9 +512,9 @@ pub mod module {
dir_fd: DirFd<'_, { _os::UNLINK_DIR_FD as usize }>,
vm: &VirtualMachine,
) -> PyResult<()> {
let c_path = path.clone().into_cstring(vm)?;
#[cfg(not(target_os = "redox"))]
if let Some(fd) = dir_fd.raw_opt() {
let c_path = path.clone().into_cstring(vm)?;
let res = unsafe { libc::unlinkat(fd, c_path.as_ptr(), 0) };
return if res < 0 {
let err = crate::common::os::errno_io_error();
Expand Down
Loading
Oops, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
Add this suggestion to a batch that can be applied as a single commit. This suggestion is invalid because no changes were made to the code. Suggestions cannot be applied while the pull request is closed. Suggestions cannot be applied while viewing a subset of changes. Only one suggestion per line can be applied in a batch. Add this suggestion to a batch that can be applied as a single commit. Applying suggestions on deleted lines is not supported. You must change the existing code in this line in order to create a valid suggestion. Outdated suggestions cannot be applied. This suggestion has been applied or marked resolved. Suggestions cannot be applied from pending reviews. Suggestions cannot be applied on multi-line comments. Suggestions cannot be applied while the pull request is queued to merge. Suggestion cannot be applied right now. Please check back later.
You are visitor 01402520 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.