GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — Strange import errors with Python 3.12 on Windows · Issue #104820 · python/cpython

Strange import errors with Python 3.12 on Windows #104820

New issue
Copy link
New issue
Copy link
Closed
Closed
Strange import errors with Python 3.12 on Windows#104820
Copy link
Assignees
zooba
Labels
3.12only security fixesonly security fixes3.13only security fixesonly security fixesOS-windowsrelease-blockertype-bugAn unexpected behavior, bug, or errorAn unexpected behavior, bug, or error

Description

@pekkaklarck
pekkaklarck
opened on May 23, 2023
Issue body actions

I tried to test our project with Python 3.12 beta 1 on Windows but everything failed. After some debugging I noticed that module imports seem to fail when modules aren't on my C-drive:

C:\Users\peke>echo print(1) > test312.py

C:\Users\peke>py -3.12 -c "import test312"
1

C:\Users\peke>e:

E:\>echo print(1) > test312.py

E:\>py -3.12 -c "import test312"
Traceback (most recent call last):
  File "<string>", line 1, in <module>
ModuleNotFoundError: No module named 'test312'

No problems with earlier Python versions:

E:\>py -3.11 -c "import test312"
1

Not sure does it matter, but I'm running Windows on VirtualBox and that E-drive is mapped to a directory on the Linux host.

Linked PRs

  • gh-104820: Fixes os.stat on Windows to better handle file systems that do not support FileIdInformation #104892
  • [3.12] gh-104820: Fixes os.stat on Windows to better handle file systems that do not support FileIdInformation (GH-104892) #104905

Activity

  1. pekkaklarck
    added
    type-bugAn unexpected behavior, bug, or errorAn unexpected behavior, bug, or error
    on May 23, 2023
  2. eryksun commented on May 23, 2023

    @eryksun
    eryksun
    on May 23, 2023
    Contributor
    More actions

    This is due to a bug in os.stat() for filesystems that lack support for FileIdInfo. The same bug is also the cause of the problem with pathlib.Path.is_dir() that's reported in gh-104806. pathlib.Path.is_dir() uses os.stat(), while ntpath.isdir() uses nt._path_isdir(). For example, volume "G:" on my system contains an exFAT filesystem, which doesn't support FileIdInfo.

    >>> nt._path_isdir('G:\\')
    True
    >>> stat.S_ISDIR(os.stat('G:\\').st_mode)
    False
    >>> stat.S_ISBLK(os.stat('G:\\').st_mode)
    True
    >>> nt._path_isfile('G:\\spam.txt')
    True
    >>> stat.S_ISREG(os.stat('G:\\spam.txt').st_mode)
    False
    >>> stat.S_ISBLK(os.stat('G:\\spam.txt').st_mode)
    True

    As shown above, os.stat() is mistakenly reporting files and directories on this volume as block devices.

    @zooba, in win32_xstat_slow_impl() in "Modules/posixmodule.c", the FileIdInfo request isn't universally supported by filesystem drivers. For example, it's not supported by FAT32/exFAT and, as demonstrated by this issue, it's not supported by the VirtualBox shared-folder filesystem.

            if (!GetFileInformationByHandle(hFile, &fileInfo) ||
                !GetFileInformationByHandleEx(hFile, FileBasicInfo,
                                              &basicInfo, sizeof(basicInfo)) ||
                !GetFileInformationByHandleEx(hFile, FileIdInfo,
                                              &idInfo, sizeof(idInfo))) {
                switch (GetLastError()) {
                case ERROR_INVALID_PARAMETER:
                case ERROR_INVALID_FUNCTION:
                case ERROR_NOT_SUPPORTED:
                    /* Volumes and physical disks are block devices, e.g.
                       \\.\C: and \\.\PhysicalDrive0. */
                    memset(result, 0, sizeof(*result));
                    result->st_mode = 0x6000; /* S_IFBLK */
                    goto cleanup;
                }
                retval = -1;
                goto cleanup;
            }

    I'd add a new pointer variable, p_idInfo. If the request fails, set p_idInfo = NULL. Otherwise set p_idInfo = &idInfo. _Py_attribute_data_to_stat() in "Python/fileutils.c" falls back on the 64-bit file ID from the BY_HANDLE_FILE_INFORMATION if the id_info parameter is a NULL pointer.

    Also, to err on the side of caution, _Py_attribute_data_to_stat() should fall back on the 64-bit file ID if the 128-bit file ID is 0 (i.e. both the low and high 64-bit parts are 0). The latter is the required value specified in [MS-FSCC] if a filesystem doesn't support a 128-bit file ID (even with the high 64-bit part set to 0) but for some reason the driver implements the FileIdInformation information class. I don't have an example of this. Usually it's either supported or requesting FileIdInformation fails, but the specification says we should be prepared to handle a zero value.

    Also, when id_info is available, I'd prefer to use its 64-bit VolumeSerialNumber field for st_dev, which is consistent with the new by-name fast path. Else fall back on the 32-bit dwVolumeSerialNumber from the BY_HANDLE_FILE_INFORMATION. NTFS and ReFS have always supported a 64-bit volume serial number.

    Reacted by Pekka Klärck, Barney Gale, sunmy2019, Steve Dower, Jason Bowen and An Long
  3. eryksun
    added
    OS-windows
    release-blocker
    3.12only security fixesonly security fixes
    3.13only security fixesonly security fixes
    on May 23, 2023
  4. ezio-melotti
    added this to Release and Deferred blockers 🚫on May 23, 2023
  5. github-project-automation
    moved this to Todo in Release and Deferred blockers 🚫on May 23, 2023
  6. eryksun
    assigned
    zooba
    on May 23, 2023
  7. zooba
    added a commit that references this issue on May 24, 2023

    pythongh-104820: Fixes os.stat on Windows to better handle file syste…

    8bc077a
  8. eryksun commented on May 24, 2023

    @eryksun
    eryksun
    on May 24, 2023
    Contributor
    More actions

    @zooba

    Also, when id_info is available, I'd prefer to use its 64-bit VolumeSerialNumber field for st_dev

    Sorry, Steve. I missed that you had already implemented this when I scanned over the code yesterday. I should have read it more carefully.

  9. zooba
    added a commit that references this issue on May 24, 2023

    gh-104820: Fixes os.stat on Windows to better handle file systems tha…

    6031727
  10. miss-islington
    added a commit that references this issue on May 24, 2023

    pythongh-104820: Fixes os.stat on Windows to better handle file syste…

    6afb37e
  11. zooba
    closed this as completedon May 24, 2023
  12. github-project-automation
    moved this from Todo to Done in Release and Deferred blockers 🚫on May 24, 2023
  13. zooba
    added a commit that references this issue on May 29, 2023

    gh-104820: Fixes os.stat on Windows to better handle file systems tha…

    68bf3fe
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

  • @zooba
    zooba

Labels

3.12only security fixesonly security fixes3.13only security fixesonly security fixesOS-windowsrelease-blockertype-bugAn unexpected behavior, bug, or errorAn unexpected behavior, bug, or error

Projects

  • Release and Deferred blockers 🚫
    Status
    Done
    Show more project fields

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions

You are visitor 01402503 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.