GeezerVisonThe World Wide Web, restored to 800 pixels of glory. | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Home|About|Gaming cabinet|Archives|How it works|Starter sites | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
GeezerVison is showing github.com — Add `symtable.py` from 3.13.5 by ShaharNaveh · Pull Request #6048 · RustPython/RustPython Conversation
|
WalkthroughThe changes update the symbol table module in the standard library. The module's registration key is renamed from "symtable" to "_symtable". The Changes
Estimated code review effort🎯 2 (Simple) | ⏱️ ~10 minutes Possibly related PRs
Poem
Note ⚡️ Unit Test Generation is now available in beta!Learn more here, or try it out under "Finishing Touches" below. ✨ Finishing Touches
🧪 Generate unit tests
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. 🪧 TipsChatThere are 3 ways to chat with CodeRabbit:
SupportNeed help? Create a ticket on our support page for assistance with any issues or questions. Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments. CodeRabbit Commands (Invoked using PR comments)
Other keywords and placeholders
Documentation and Community
|
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
Did we have test_symtable but no symtable.py? How did it work before?
Sorry, something went wrong.
There was an error while loading. Please reload this page.
|
|
||
| Alias = find_block(top, "Alias") | ||
| GenericAlias = find_block(top, "GenericAlias") | ||
| # TODO(RUSTPYTHON) |
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
| # TODO(RUSTPYTHON) | |
| # XXX: RUSTPYTHON |
We don't have a strict rule for this, but conventionally using this to distinguish local patch from failing tests.
Sorry, something went wrong.
There was an error while loading. Please reload this page.
| # U = find_block(GenericMine, "U") | ||
|
|
||
|
|
||
| # TODO(RUSTPYTHON) |
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
| # TODO(RUSTPYTHON) | |
| # TODO: RUSTPYTHON |
The notation itself looks good.
But we use search tools for this comment. Keeping consistency will be helpful
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
Old habits die hard 😅
Sorry, something went wrong.
There was an error while loading. Please reload this page.
Correct
We had implemented |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
| self.assertGreater(self.GenericMine.get_id(), 0) | ||
|
|
||
| # TODO: RUSTPYTHON | ||
| @unittest.expectedFailure |
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
If we had test-passing symtable before, can these tests be easily fixed?
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
I think so, I think the main issue is that "symbols" attribute used to return a list, now it's a dict. I'll investigate a bit more about the structure. For now I'll set the PR as a draft
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
Thank you so much!
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
Now more tests pass. But we still have a difference between our impl and cpython. but this is a change that needs to be made at compiler/codegen/src/symboltable.rs, I think it's a bit out of scope for this PR. I'll try to do a followup PR to fix it
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
Actionable comments posted: 1
Configuration used: .coderabbit.yml
Review profile: CHILL
Plan: Pro
Lib/test/test_symtable.py is excluded by !Lib/**vm/src/stdlib/symtable.rs (3 hunks)📄 CodeRabbit Inference Engine (.github/copilot-instructions.md)
**/*.rs: Follow the default rustfmt code style (cargo fmtto format)
Always run clippy to lint code (cargo clippy) before completing tasks. Fix any warnings or lints that are introduced by your changes
Follow Rust best practices for error handling and memory management
Use the macro system (pyclass,pymodule,pyfunction, etc.) when implementing Python functionality in Rust
Files:
vm/src/stdlib/symtable.rsLearnt from: moreal
PR: #5847
File: vm/src/stdlib/stat.rs:547-567
Timestamp: 2025-06-27T14:47:28.810Z
Learning: In RustPython's stat module implementation, platform-specific constants like SF_SUPPORTED and SF_SYNTHETIC should be conditionally declared only for the platforms where they're available (e.g., macOS), following CPython's approach of optional declaration using #ifdef checks rather than providing fallback values for other platforms.
Learnt from: moreal
PR: #5847
File: vm/src/stdlib/stat.rs:547-567
Timestamp: 2025-06-27T14:47:28.810Z
Learning: In RustPython's stat module implementation, platform-specific constants like SF_SUPPORTED and SF_SYNTHETIC should be conditionally declared only for the platforms where they're available (e.g., macOS), following CPython's approach of optional declaration rather than providing fallback values for other platforms.
Learnt from: CR
PR: RustPython/RustPython#0
File: .github/copilot-instructions.md:0-0
Timestamp: 2025-07-19T03:16:56.511Z
Learning: Applies to **/*.rs : Use the macro system (pyclass, pymodule, pyfunction, etc.) when implementing Python functionality in Rust
vm/src/stdlib/symtable.rs (4)
5-11: LGTM!The import additions are appropriate for the new functionality -
PyDictRefsupports thesymbolsproperty and the explicitcompilerimport improves code clarity.
13-105: Excellent alignment with CPython's symtable constants.The comprehensive set of constants mirrors CPython's internal symtable definitions, ensuring compatibility. The bit-shifting patterns and flag combinations are mathematically correct and follow established conventions.
144-184: Excellent refactoring to Python-style properties.The conversion from explicit methods to properties using
#[pygetset]aligns perfectly with Python conventions. The use of#[pygetset(name = "type")]properly handles Rust keyword conflicts.
186-194: Clean implementation of symbols property.The new
symbolsproperty correctly creates a dictionary mapping symbol names to their flag bits, providing a proper Python-style interface to the symbol information.
Sorry, something went wrong.
There was an error while loading. Please reload this page.
| #[pygetset] | ||
| fn id(&self) -> usize { | ||
| self as *const Self as *const std::ffi::c_void as usize | ||
| } |
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
💡 Verification agent
Consider security implications of exposing memory addresses.
The id property exposes raw memory addresses, which could potentially be used for memory layout attacks or information disclosure in security-sensitive contexts.
Consider whether this aligns with CPython's behavior and if there are security implications for your use case:
🌐 Web query:
What does Python's symtable SymbolTable.id property return and are there security considerations?
💡 Result:
The id property (retrievable via get_id()) of Python's symtable.SymbolTable returns the table’s identifier, which is an internal value associated with that specific symbol table instance[1][2][5][6]. This identifier is primarily intended for internal use, such as distinguishing between different symbol table objects generated during code analysis.
Details:
What does get_id() return?
SymbolTable.get_id() is an identifier for that block's symbol table instance. It does not return the name of the function, class, or module—that is provided by get_name(). Instead, it gives an ID unique to the table, but published documentation does not specify if this is guaranteed to be globally unique, stable between runs, or even meaningful outside the context of the current symbol table introspection session[1][2][5][6].get_id() alongside type and name to help programmers identify and differentiate symbol tables while traversing nested scopes.Security considerations:
id or the value returned by get_id() contains sensitive information such as memory addresses, code contents, or pointers to user data[1][2][5][6].Summary Table of Relevant SymbolTable Methods:
| Method | Purpose |
|---|---|
get_id() |
Returns the symbol table's internal identifier |
get_name() |
Returns the name of the scope (top for module, function/class name otherwise) |
get_type() |
Returns the table's type ('module', 'class', or 'function') |
get_lineno() |
Returns the first line number of the scope |
If you require a value that is stable and reveals no internal information, prefer using get_name() or a custom mapping on your analysis layer instead of exposing the value from get_id().
References:
[1][2][5][6]: Python's official symtable module documentation and tutorials.
Citations:
Align SymbolTable.id() with CPython’s get_id(): avoid exposing raw pointers
The current implementation in vm/src/stdlib/symtable.rs returns the struct’s memory address as a usize, which diverges from CPython’s symtable.SymbolTable.get_id() (an internal, opaque identifier) and can leak pointer layouts.
Suggested changes:
fn id(&self) -> usize with a dedicated, incrementing identifier (e.g., an AtomicUsize counter or per‐table UID stored on creation).id() returns an internal symbol-table ID, not a memory address.Locations to update:
fn id(&self) definition)Example diff sketch:
- #[pygetset] - fn id(&self) -> usize { - self as *const Self as *const std::ffi::c_void as usize - } + // Use an internal counter instead of raw pointer to avoid leaking memory addresses + #[pygetset] + fn id(&self) -> usize { + self.inner_id // populated from a global AtomicUsize at creation + }
Committable suggestion skipped: line range outside the PR's diff.
In vm/src/stdlib/symtable.rs around lines 170 to 173, the id() method currently
returns the raw memory address cast to usize, which exposes internal pointer
details and differs from CPython's opaque get_id(). To fix this, replace the
pointer cast with a unique internal identifier generated via a static
AtomicUsize counter incremented for each new SymbolTable instance or assign a
unique ID at creation stored in the struct. Update the id() method to return
this stored unique ID instead of the pointer. Also, revise the method's
documentation to clarify that id() returns an internal symbol table identifier,
not a memory address.
Sorry, something went wrong.
There was an error while loading. Please reload this page.
| #[pymethod] | ||
| const fn is_nested(&self) -> bool { | ||
| self.symtable.is_nested | ||
| } | ||
|
|
||
| #[pymethod] | ||
| const fn is_optimized(&self) -> bool { | ||
| matches!( | ||
| self.symtable.typ, | ||
| CompilerScope::Function | CompilerScope::AsyncFunction | ||
| ) |
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
let's not delete these methods but change to getset, just like get_name and get_lineno.
Unlike other more complex cases, the deletion of these 2 methods cause unnecessary regression.
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
Fix this. And made type to return an int instead of a String, so now more tests pass
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
Actionable comments posted: 2
vm/src/stdlib/symtable.rs (1)
181-184: Security concern with id() method exposing raw memory addresses.The
id()method returns raw memory addresses, which could potentially be used for memory layout attacks or information disclosure in security-sensitive contexts.
Configuration used: .coderabbit.yml
Review profile: CHILL
Plan: Pro
Lib/test/test_symtable.py is excluded by !Lib/**vm/src/stdlib/symtable.rs (3 hunks)📄 CodeRabbit Inference Engine (.github/copilot-instructions.md)
**/*.rs: Follow the default rustfmt code style (cargo fmtto format)
Always run clippy to lint code (cargo clippy) before completing tasks. Fix any warnings or lints that are introduced by your changes
Follow Rust best practices for error handling and memory management
Use the macro system (pyclass,pymodule,pyfunction, etc.) when implementing Python functionality in Rust
Files:
vm/src/stdlib/symtable.rsLearnt from: moreal
PR: RustPython/RustPython#5847
File: vm/src/stdlib/stat.rs:547-567
Timestamp: 2025-06-27T14:47:28.810Z
Learning: In RustPython's stat module implementation, platform-specific constants like SF_SUPPORTED and SF_SYNTHETIC should be conditionally declared only for the platforms where they're available (e.g., macOS), following CPython's approach of optional declaration using #ifdef checks rather than providing fallback values for other platforms.
Applied to files:
vm/src/stdlib/symtable.rsLearnt from: moreal
PR: RustPython/RustPython#5847
File: vm/src/stdlib/stat.rs:547-567
Timestamp: 2025-06-27T14:47:28.810Z
Learning: In RustPython's stat module implementation, platform-specific constants like SF_SUPPORTED and SF_SYNTHETIC should be conditionally declared only for the platforms where they're available (e.g., macOS), following CPython's approach of optional declaration rather than providing fallback values for other platforms.
Applied to files:
vm/src/stdlib/symtable.rsLearnt from: CR
PR: RustPython/RustPython#0
File: .github/copilot-instructions.md:0-0
Timestamp: 2025-07-19T03:16:56.511Z
Learning: Applies to **/*.rs : Use the macro system (`pyclass`, `pymodule`, `pyfunction`, etc.) when implementing Python functionality in Rust
Applied to files:
vm/src/stdlib/symtable.rsvm/src/stdlib/symtable.rs (5)
149-152: LGTM! Method to property conversion aligns with feedback.Converting
get_nameto anameproperty using#[pygetset]aligns well with the previous review feedback to use getset instead of deleting methods.
171-179: LGTM! Children property correctly replaces get_children method.The
childrenproperty properly converts sub-tables to Python objects and aligns with the goal of using properties instead of explicit methods.
186-195: LGTM! Identifiers property conversion is appropriate.Converting
get_identifiersto anidentifiersproperty maintains the same functionality while following the property-based interface pattern.
207-210: LGTM! Nested property appropriately replaces is_nested method.The
nestedproperty correctly exposes the symbol table's nested status, following the pattern established by other property conversions.
15-110: Constants section matches CPython internal definitionsThe bit shifts and composite values (e.g.
DEF_BOUND,SCOPE_MASK) exactly mirror those in CPython’spycore_symtable.h. No changes needed here.
Sorry, something went wrong.
There was an error while loading. Please reload this page.
| #[pygetset(name = "type")] | ||
| fn typ(&self) -> i32 { | ||
| match self.symtable.typ { | ||
| CompilerScope::Function => TYPE_FUNCTION, | ||
| CompilerScope::Class => TYPE_CLASS, | ||
| CompilerScope::Module => TYPE_MODULE, | ||
| CompilerScope::TypeParams => TYPE_TYPE_PARAMETERS, | ||
| _ => -1, // TODO: missing types from the C implementation | ||
| } | ||
| } |
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
💡 Verification agent
Verify integer constant mapping for type property.
The typ() method now returns integer constants instead of strings, which is a significant interface change. The mapping looks reasonable, but the fallback to -1 for unhandled cases may not align with CPython's behavior.
🏁 Script executed:
#!/bin/bash # Check CPython's SymbolTable.get_type() return values python3 -c " import symtable st = symtable.symtable('def f(): pass', '<string>', 'exec') print('Module type:', repr(st.get_type())) for child in st.get_children(): print('Function type:', repr(child.get_type())) "
Length of output: 297
Ensure type property matches CPython’s string-based API
CPython’s SymbolTable.get_type() returns strings like "module" and "function", not integer codes. Our Rust typ() currently returns integer constants and falls back to -1 for any unhandled cases, which:
Please update the implementation to either:
-1 fallback with a dedicated TYPE_UNKNOWN constant—and add mappings for all types in CPython’s C implementation.Points to address:
// TODO and implement mappings for missing CPython types (generator, lambda, comprehension, etc.)-1 fallback or replace it with a well-defined default constantIn vm/src/stdlib/symtable.rs lines 154 to 163, the typ() function currently
returns integer constants and uses -1 as a fallback, which breaks compatibility
with CPython's string-based SymbolTable.get_type() API and omits several type
names. To fix this, change the return type to Python strings matching CPython's
type names (e.g., "module", "function", "class", "generator", "lambda",
"comprehension"), remove the -1 fallback, and implement mappings for all missing
CPython types. Also update any related documentation to reflect the string
return type and remove the TODO comment.
Sorry, something went wrong.
There was an error while loading. Please reload this page.
| #[pygetset] | ||
| fn symbols(&self, vm: &VirtualMachine) -> PyResult<PyDictRef> { | ||
| let dict = vm.ctx.new_dict(); | ||
| for (name, symbol) in &self.symtable.symbols { | ||
| dict.set_item(name, vm.new_pyobj(symbol.flags.bits()), vm) | ||
| .unwrap(); | ||
| } | ||
| Ok(dict) | ||
| } |
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
💡 Verification agent
Significant interface change: symbols now returns dictionary instead of list.
The symbols property now returns a dictionary mapping symbol names to their flag bits, which is a major change from the previous list-based return. This appears to align better with how symbols are typically accessed.
🏁 Script executed:
#!/bin/bash # Verify CPython's SymbolTable.get_symbols() return type and structure python3 -c " import symtable st = symtable.symtable('x = 1', '<string>', 'exec') symbols = st.get_symbols() print('Type:', type(symbols)) if symbols: print('First symbol type:', type(symbols[0])) print('First symbol methods:', [m for m in dir(symbols[0]) if not m.startswith('_')]) "
Length of output: 639
Breaking change: symbols now returns dict instead of list
You’ve changed the symbols getter in vm/src/stdlib/symtable.rs (lines 197–205) from a list of Symbol objects to a dict[str, int]. In CPython, symtable.symtable(...).get_symbols() returns a list[Symbol] (each with methods like is_assigned(), get_name(), etc.), not a mapping of names to flag bits.
This will break any existing code that:
Symbol instancesSymbol objectsPlease address this by one of the following:
symbols to return PyListRef of Symbol objects (e.g. vm.new_pyobj(symbol)) for compatibilitysymbol_flags) for the dict view, and preserve symbols as the list of SymbolIn vm/src/stdlib/symtable.rs around lines 197 to 205, the symbols getter was
changed to return a dict mapping symbol names to flag bits, but in CPython it
returns a list of Symbol objects. To fix this, revert the symbols method to
return a PyListRef of Symbol objects by creating a list and adding each symbol
object (e.g. using vm.new_pyobj(symbol)) to it, preserving compatibility. If the
dict view is needed, add a new method or property like symbol_flags for that
purpose instead of changing symbols.
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
Thank you!
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was an error while loading. Please reload this page.
Successfully merging this pull request may close these issues.
There was an error while loading. Please reload this page.