GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — gh-136059: docs: pathlib: Mention that iterdir() is surprisingly not streaming by nh2 · Pull Request #136060 · python/cpython

Conversation

@nh2

@nh2 nh2 commented Jun 28, 2025 •
edited by github-actions Bot
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown

This undocumented gotcha can cause excessive memory usage when "iterating" over very large directories.

This is because iterdir() does

entries = list(scandir_it)

Fixing (if at all desired) will likel need least significant amounts of discussion and testing, so first document the behaviour.

See:

  • pathlib: Path.iterdir() is surprisingly not streaming #136059
  • Issue: pathlib: Path.iterdir() is surprisingly not streaming #136059

📚 Documentation preview 📚: https://cpython-previews--136060.org.readthedocs.build/

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
All reactions
@nh2
docs: pathlib: Mention that iterdir() is surprisingly not streaming
3b1af86 
This undocumented gotcha can cause excessive memory usage
when "iterating" over very large directories.

This is because iterdir() does

    entries = list(scandir_it)

Fixing (if at all desired) will likel need least significant amounts of
discussion and testing, so first document the behaviour.
@nh2
nh2 requested a review from barneygale as a code owner June 28, 2025 08:09
@bedevere-app bedevere-app Bot added the awaiting review label Jun 28, 2025
@python-cla-bot

python-cla-bot Bot commented Jun 28, 2025 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown

All commit authors signed the Contributor License Agreement.

CLA signed
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
@bedevere-app bedevere-app Bot added docs Documentation in the Doc dir skip news labels Jun 28, 2025
@github-project-automation github-project-automation Bot added this to Docs PRs Jun 28, 2025
@github-project-automation github-project-automation Bot moved this to Todo in Docs PRs Jun 28, 2025
@bedevere-app bedevere-app Bot mentioned this pull request Jun 28, 2025
pathlib: Path.iterdir() is surprisingly not streaming #136059
Open
serhiy-storchaka
serhiy-storchaka reviewed Jun 29, 2025
View reviewed changes
Comment thread Doc/library/pathlib.rst
If the path is not a directory or otherwise inaccessible, :exc:`OSError` is
raised.

.. warning::

@serhiy-storchaka serhiy-storchaka Jun 29, 2025

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

warning and even note directives attract too much attention. We usually avoid using them except in serious cases. Try also to shorten the text and don't describe the current behavior as a bug. For example, somethink like this:

"The contents of the directory are read in one go, but path objects are created during iteration. If you need to reduce latency for large directories, use :func:`os.scandir` instead."

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
All reactions
@github-actions

github-actions Bot commented Apr 26, 2026

Copy link
Copy Markdown

This PR is stale because it has been open for 30 days with no activity.
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
@github-actions github-actions Bot added the stale Stale PR or inactive for long period of time. label Apr 26, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@serhiy-storchaka serhiy-storchaka serhiy-storchaka left review comments
@barneygale barneygale Awaiting requested review from barneygale barneygale is a code owner

Uh oh!

There was an error while loading. Please reload this page.

Assignees

No one assigned

Labels

awaiting review docs Documentation in the Doc dir skip news stale Stale PR or inactive for long period of time.

Projects

Docs PRs
Status: Todo

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Uh oh!

There was an error while loading. Please reload this page.

2 participants

@nh2 @serhiy-storchaka
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
You are visitor 01402503 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.