GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — `os.walk()` no longer supports late edits to `dirnames` · Issue #102932 · python/cpython

os.walk() no longer supports late edits to dirnames #102932

New issue
Copy link
New issue
Copy link
Open
Open
os.walk() no longer supports late edits to dirnames#102932
Copy link
Labels
3.12only security fixesonly security fixesstdlibStandard Library Python modules in the Lib/ directoryStandard Library Python modules in the Lib/ directorytype-bugAn unexpected behavior, bug, or errorAn unexpected behavior, bug, or error

Description

@barneygale
barneygale
opened on Mar 22, 2023
Issue body actions

Prior to the fix for #89727, it was possible influence which subdirectories were visited by os.walk() even after the walk had descended into siblings of those subdirectories. Such "late" modifications no longer have any effect.

This was reported by @pochmann:

I think you changed os.walk's behavior. The recursive one supported late modifications of subdirs lists, due to being lazier. The iterative one doesn't (if I'm not mistaken... I can't test).

With topdown, you can modify a directory's subdirs, for example remove one and it won't be visited. Usually you'd do that while your walk is on that parent directory. But with the recursive one, it was possible to do it later. With the iterative one, I don't think that works anymore.

Here's a demo where I remove the second subdir after having walked onto the first:

import os

# Create demo dir with three subdirs
os.makedirs('demo/a')
os.makedirs('demo/b')
os.makedirs('demo/c')

# Walk onto "demo"
walk = os.walk('demo')
demo = next(walk)
print(demo)

# Walk onto first subdir
first = next(walk)
print(first)

# Remove the second subdir
del demo[1][1]

# Walk onto the remaining subdirs
for x in walk:
    print(x)

Output (Try it online!), note that the second subdir wasn't walked:

('demo', ['c', 'a', 'b'], [])
('demo/c', [], [])
('demo/b', [], [])

Excerpt from the recursive one:

        for dirname in dirs:
            new_path = join(top, dirname)
            yield from _walk(new_path, topdown, onerror, followlinks)

The iteration of dirs and the walking of the subdirs are intertwined. After walking a subdir, the paused iteration of dirs resumes. That allows the late modifications of dirs to have an effect.

Excerpt from the iterative one:

            for dirname in reversed(dirs):
                new_path = join(top, dirname)
                stack.append(new_path)

This eagerly puts all subdirs onto the stack, before they're getting walked, and then dirs is never used again. So modifying dirs during the subdirs walking doesn't have an effect anymore.

Activity

  1. barneygale
    added
    type-bugAn unexpected behavior, bug, or errorAn unexpected behavior, bug, or error
    3.12only security fixesonly security fixes
    on Mar 22, 2023
  2. jonburdo commented on Mar 24, 2023

    @jonburdo
    jonburdo
    on Mar 24, 2023
    Contributor
    More actions

    If we do want to revert to the old behavior, this PR should do it (or at least shows how it could be done, including a unit test): #100703

    Currently, the latest commit splits the implementation for topdown=True and topdown=False into two functions, but a previous commit also shows how it could be done without splitting:
    https://github.com/python/cpython/pull/100703/files/1011e199529ccdbd778226574c379403708b2e5d

  3. iritkatriel
    added
    stdlibStandard Library Python modules in the Lib/ directoryStandard Library Python modules in the Lib/ directory
    on Nov 26, 2023
  4. serhiy-storchaka
    added this to OS issues 💻on Jul 3, 2026
  5. github-project-automation
    moved this to Todo in OS issues 💻on Jul 3, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    3.12only security fixesonly security fixesstdlibStandard Library Python modules in the Lib/ directoryStandard Library Python modules in the Lib/ directorytype-bugAn unexpected behavior, bug, or errorAn unexpected behavior, bug, or error

    Projects

    • OS issues 💻
      Status
      Todo
      Show more project fields

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions

    You are visitor 01402503 — GeezerVison does not store the pages you look up.
    Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.