GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — Use memchr consistently for NUL checks by joshuamegnauth54 · Pull Request #8372 · RustPython/RustPython

Conversation

@joshuamegnauth54

@joshuamegnauth54 joshuamegnauth54 commented Jul 25, 2026 •
edited by coderabbitai Bot
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Contributor

memchr is used throughout RustPython for searching through bytes expediently. However, for NUL checks, it's scantily used. Instead, our NUL checks either use contains or memchr.

I switched all of the contains(b'\0') I could find to using memchr instead. I marked the failure paths as cold to hint to LLVM that interior NULs are truly exceptional. This should help branch prediction a bit which is nice for string/bytes functions since they are likely called a lot.

  • Closes #xxxx
  • This PR follows our AI policy

Summary

  • Use memchr more as it's specialized for searching through bytes.
  • Hint to LLVM and the branch predictor that interior NULs are exceptional, so the error branch should remain out of the hot path.

Summary by CodeRabbit

  • Bug Fixes
    • Improved and standardized embedded NUL detection across string/bytes handling, environment variable validation, SSL/hostname/cipher setup, codecs lookup, file I/O wrapper checks, and path conversions.
    • Unix semaphore name/open/unlink errors are now reported more precisely for interior-NUL cases, raising the correct NUL-related exceptions.
  • Performance
    • Faster embedded-NUL scanning using byte-level search, plus “rare error” paths are marked to minimize impact on the common case.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
@coderabbitai

coderabbitai Bot commented Jul 25, 2026 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Contributor

Review Change Stack

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

  • @coderabbitai resume to resume automatic reviews.
  • @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

  • ▶️ Resume reviews
  • 🔍 Trigger review
📝 Walkthrough

Walkthrough

The change centralizes NUL-byte detection through Python string and byte helpers, marks validation failures as cold paths, propagates Unix semaphore interior-NUL errors explicitly, and updates host and standard-library integrations.

Changes

NUL validation and error propagation

Layer / File(s) Summary
Shared NUL primitives and conversions
crates/vm/src/builtins/{bytes.rs,str.rs}, crates/vm/src/bytes_inner.rs, crates/vm/src/utils.rs
Adds shared contains_nuls() helpers, makes byte accessors const, and removes separate ToCString NUL validation.
Runtime path and environment validation
crates/vm/src/function/fspath.rs, crates/vm/src/ospath.rs, crates/vm/src/stdlib/{_codecs.rs,_io.rs,nt.rs,os.rs,pwd.rs}
Updates path, codec, I/O, process, environment, and account-name validation to use shared NUL checks and cold error branches.
Standard-library API validation
crates/stdlib/src/{grp.rs,mmap.rs,multiprocessing.rs,openssl.rs,ssl.rs}
Applies shared NUL detection, preserves InteriorNul semaphore errors, and centralizes SSL hostname validation.
Host-platform error boundaries
crates/host_env/{Cargo.toml,src/{multiprocessing.rs,time.rs,winapi.rs}}
Propagates Unix semaphore interior-NUL errors and updates Windows wide-string, dependency, and environment-block handling.

Estimated code review effort: 3 (Moderate) | ~25 minutes

Possibly related PRs

  • RustPython/RustPython#8339: Updates overlapping Unix multiprocessing NUL-error propagation and exception mapping.

Suggested reviewers: youknowone, shaharnaveh

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title accurately summarizes the main change: replacing NUL checks with memchr across the codebase.
Docstring Coverage ✅ Passed Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share
  • X
  • Mastodon
  • Reddit
  • LinkedIn

Comment @coderabbitai help to get the list of available commands.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@joshuamegnauth54
joshuamegnauth54 force-pushed the use-memchr-consistently branch from 8549281 to 6788a1e Compare July 25, 2026 19:48
coderabbitai[bot]
coderabbitai Bot reviewed Jul 25, 2026
View reviewed changes

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 3

🧹 Nitpick comments (1)
crates/stdlib/src/multiprocessing.rs (1)

805-822: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

Missing cold_path() on the SemError::InteriorNul branches.

Every other NUL-error return site touched by this PR (grp.rs, openssl.rs::set_ciphers/new_py_ssl_socket, mmap.rs, nt.rs::execve, ssl.rs::validate_hostname) calls cold_path() immediately before returning the interior-NUL error, per the PR's stated goal of marking these exceptional paths cold. These two branches in py_new and sem_unlink return exceptions::nul_char_error(vm) on SemError::InteriorNul without a cold_path() call, leaving this file's semaphore error paths on the "hot" path unlike everywhere else.

♻️ Add cold_path() to match the cohort convention
+use core::hint::cold_path;
...
             SemHandle::create(&args.name, value, args.unlink).map_err(|err| {
                 if err == SemError::InteriorNul {
+                    cold_path();
                     exceptions::nul_char_error(vm)
                 } else {
                     os_error(vm, err)
                 }
             })?;
...
     fn sem_unlink(name: String, vm: &VirtualMachine) -> PyResult<()> {
         host_multiprocessing::sem_unlink(&name).map_err(|err| {
             if err == SemError::InteriorNul {
+                cold_path();
                 exceptions::nul_char_error(vm)
             } else {
                 os_error(vm, err)
             }
         })
     }

Also applies to: 838-847

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@crates/stdlib/src/multiprocessing.rs` around lines 805 - 822, Add cold_path()
immediately before returning exceptions::nul_char_error(vm) in both the
SemError::InteriorNul branch of py_new and the corresponding branch in
sem_unlink. Keep the existing non-NUL error handling unchanged and match the
cold-path convention used by the other touched NUL-error sites.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@crates/host_env/src/winapi.rs`:
- Around line 312-315: Update the NUL-check condition in the environment-block
validation flow to call .is_some() on both memchr results before combining them
with ||. Keep the existing cold_path() and
BuildEnvironmentBlockError::ContainsNul return behavior unchanged.

In `@crates/stdlib/src/multiprocessing.rs`:
- Around line 13-16: Remove the unused cold_path import from the core imports in
the Windows-only multiprocessing module. Keep the AtomicI32, AtomicU32, and
Ordering imports unchanged.

In `@crates/vm/src/stdlib/nt.rs`:
- Around line 555-558: Update the NUL checks in the affected code path to call
contains_nuls() on both key and value PyStrRef instances instead of
contains_null(), preserving the existing cold_path() and nul_char_error return
behavior.

---

Nitpick comments:
In `@crates/stdlib/src/multiprocessing.rs`:
- Around line 805-822: Add cold_path() immediately before returning
exceptions::nul_char_error(vm) in both the SemError::InteriorNul branch of
py_new and the corresponding branch in sem_unlink. Keep the existing non-NUL
error handling unchanged and match the cold-path convention used by the other
touched NUL-error sites.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yml

Review profile: CHILL

Plan: Pro Plus

Run ID: 9548f16e-55a1-4f9c-b3d5-72ded4122018

📥 Commits

Reviewing files that changed from the base of the PR and between 003ebec and 8549281.

📒 Files selected for processing (19)
  • crates/host_env/src/multiprocessing.rs
  • crates/host_env/src/time.rs
  • crates/host_env/src/winapi.rs
  • crates/stdlib/src/grp.rs
  • crates/stdlib/src/mmap.rs
  • crates/stdlib/src/multiprocessing.rs
  • crates/stdlib/src/openssl.rs
  • crates/stdlib/src/ssl.rs
  • crates/vm/src/builtins/bytes.rs
  • crates/vm/src/builtins/str.rs
  • crates/vm/src/bytes_inner.rs
  • crates/vm/src/function/fspath.rs
  • crates/vm/src/ospath.rs
  • crates/vm/src/stdlib/_codecs.rs
  • crates/vm/src/stdlib/_io.rs
  • crates/vm/src/stdlib/nt.rs
  • crates/vm/src/stdlib/os.rs
  • crates/vm/src/stdlib/pwd.rs
  • crates/vm/src/utils.rs
💤 Files with no reviewable changes (1)
  • crates/vm/src/utils.rs

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
Comment thread crates/host_env/src/winapi.rs Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread crates/stdlib/src/multiprocessing.rs Outdated
Comment on lines +13 to +16
use core::{
hint::cold_path,
sync::atomic::{AtomicI32, AtomicU32, Ordering},
};

@coderabbitai coderabbitai Bot Jul 25, 2026

Copy link
Copy Markdown
Contributor

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick win

Unused cold_path import in this Windows-only module.

cold_path is imported here but never called anywhere in this mod _multiprocessing (the Windows implementation has no NUL-check branch that uses it). This will trigger an unused_imports warning under cargo clippy.

As per coding guidelines, "Always run clippy to lint code with cargo clippy... and fix any warnings or lints introduced by changes."

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@crates/stdlib/src/multiprocessing.rs` around lines 13 - 16, Remove the unused
cold_path import from the core imports in the Windows-only multiprocessing
module. Keep the AtomicI32, AtomicU32, and Ordering imports unchanged.

Source: Coding guidelines

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
Comment thread crates/vm/src/stdlib/nt.rs Resolved

Uh oh!

There was an error while loading. Please reload this page.

@joshuamegnauth54
joshuamegnauth54 force-pushed the use-memchr-consistently branch 2 times, most recently from 70a29fa to 6b4f1d6 Compare July 26, 2026 16:46
coderabbitai[bot]
coderabbitai Bot reviewed Jul 26, 2026
View reviewed changes

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (1)
crates/vm/src/function/fspath.rs (1)

43-53: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

Extract the shared NUL-error path.

The two arms differ only in the source value and constructed variant; derive the NUL flag and FsPath in the match, then perform the check once.

Proposed refactor
-                s @ PyStr => {
-                    if check_for_nul && s.contains_nuls() {
-                        cold_path();
-                        return Err(crate::exceptions::nul_char_error(vm));
-                    }
-                    Self::Str(s)
-                }
-                b @ PyBytes => {
-                    if check_for_nul && b.contains_nuls() {
-                        cold_path();
-                        return Err(crate::exceptions::nul_char_error(vm));
-                    }
-                    Self::Bytes(b)
-                }
+                s @ PyStr => (s.contains_nuls(), Self::Str(s)),
+                b @ PyBytes => (b.contains_nuls(), Self::Bytes(b)),

As per coding guidelines, “When branches differ only in a value but share common logic, extract the differing value first, then call the common logic once to avoid duplicate code.”

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@crates/vm/src/function/fspath.rs` around lines 43 - 53, Refactor the match
arms for PyStr and PyBytes to first derive the shared NUL-check flag and
corresponding FsPath value, then perform the check_for_nul/contains_nuls
validation once before returning the constructed value. Preserve the existing
cold_path and nul_char_error behavior, while retaining the distinct Str and
Bytes variants.

Source: Coding guidelines

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In `@crates/vm/src/function/fspath.rs`:
- Around line 43-53: Refactor the match arms for PyStr and PyBytes to first
derive the shared NUL-check flag and corresponding FsPath value, then perform
the check_for_nul/contains_nuls validation once before returning the constructed
value. Preserve the existing cold_path and nul_char_error behavior, while
retaining the distinct Str and Bytes variants.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yml

Review profile: CHILL

Plan: Pro Plus

Run ID: 53b83214-f265-47d2-87f7-1e1197f2ec0a

📥 Commits

Reviewing files that changed from the base of the PR and between 70a29fa and 6b4f1d6.

⛔ Files ignored due to path filters (1)
  • Cargo.lock is excluded by !**/*.lock
📒 Files selected for processing (20)
  • crates/host_env/Cargo.toml
  • crates/host_env/src/multiprocessing.rs
  • crates/host_env/src/time.rs
  • crates/host_env/src/winapi.rs
  • crates/stdlib/src/grp.rs
  • crates/stdlib/src/mmap.rs
  • crates/stdlib/src/multiprocessing.rs
  • crates/stdlib/src/openssl.rs
  • crates/stdlib/src/ssl.rs
  • crates/vm/src/builtins/bytes.rs
  • crates/vm/src/builtins/str.rs
  • crates/vm/src/bytes_inner.rs
  • crates/vm/src/function/fspath.rs
  • crates/vm/src/ospath.rs
  • crates/vm/src/stdlib/_codecs.rs
  • crates/vm/src/stdlib/_io.rs
  • crates/vm/src/stdlib/nt.rs
  • crates/vm/src/stdlib/os.rs
  • crates/vm/src/stdlib/pwd.rs
  • crates/vm/src/utils.rs
💤 Files with no reviewable changes (1)
  • crates/vm/src/utils.rs
🚧 Files skipped from review as they are similar to previous changes (14)
  • crates/vm/src/ospath.rs
  • crates/host_env/Cargo.toml
  • crates/host_env/src/winapi.rs
  • crates/vm/src/stdlib/_codecs.rs
  • crates/vm/src/builtins/bytes.rs
  • crates/vm/src/stdlib/nt.rs
  • crates/stdlib/src/grp.rs
  • crates/stdlib/src/multiprocessing.rs
  • crates/vm/src/bytes_inner.rs
  • crates/stdlib/src/ssl.rs
  • crates/host_env/src/time.rs
  • crates/stdlib/src/openssl.rs
  • crates/vm/src/stdlib/_io.rs
  • crates/host_env/src/multiprocessing.rs

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
@joshuamegnauth54
joshuamegnauth54 force-pushed the use-memchr-consistently branch 2 times, most recently from 3cb02f0 to 93d0a9c Compare July 27, 2026 03:23
@joshuamegnauth54
Use memchr consistently for NUL checks
9dacb3e
`memchr` is used throughout RustPython for searching through bytes
expediently. However, for NUL checks, it's scantily used. Instead, our
NUL checks either use `contains` or `memchr`.

I switched all of the `contains(b'\0')` I could find to using memchr
instead. I marked the failure paths as cold to hint to LLVM that
interior NULs are truly exceptional. This should help branch prediction
a bit which is nice for string/bytes functions since they are likely
called a lot.
@joshuamegnauth54
joshuamegnauth54 force-pushed the use-memchr-consistently branch from 93d0a9c to 9dacb3e Compare July 27, 2026 03:59
youknowone
youknowone approved these changes Jul 27, 2026
View reviewed changes

@youknowone youknowone left a comment

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks reasonable, thanks!

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
@youknowone
youknowone merged commit f0708f5 into RustPython:main Jul 27, 2026
25 of 26 checks passed

Uh oh!

There was an error while loading. Please reload this page.

@joshuamegnauth54
joshuamegnauth54 deleted the use-memchr-consistently branch July 27, 2026 15:58
youknowone pushed a commit to youknowone/RustPython that referenced this pull request Jul 29, 2026
@joshuamegnauth54 @youknowone
Use memchr consistently for NUL checks (RustPython#8372)
67bdeb3
`memchr` is used throughout RustPython for searching through bytes
expediently. However, for NUL checks, it's scantily used. Instead, our
NUL checks either use `contains` or `memchr`.

I switched all of the `contains(b'\0')` I could find to using memchr
instead. I marked the failure paths as cold to hint to LLVM that
interior NULs are truly exceptional. This should help branch prediction
a bit which is nice for string/bytes functions since they are likely
called a lot.
youknowone pushed a commit that referenced this pull request Sep 16, 2026
@joshuamegnauth54
Use memchr consistently for NUL checks (#8372)
71f2ffc
`memchr` is used throughout RustPython for searching through bytes
expediently. However, for NUL checks, it's scantily used. Instead, our
NUL checks either use `contains` or `memchr`.

I switched all of the `contains(b'\0')` I could find to using memchr
instead. I marked the failure paths as cold to hint to LLVM that
interior NULs are truly exceptional. This should help branch prediction
a bit which is nice for string/bytes functions since they are likely
called a lot.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@coderabbitai coderabbitai[bot] coderabbitai[bot] left review comments
@youknowone youknowone youknowone approved these changes

Uh oh!

There was an error while loading. Please reload this page.

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Uh oh!

There was an error while loading. Please reload this page.

2 participants

@joshuamegnauth54 @youknowone
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
You are visitor 01402520 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.