GeezerVison

Copy link

Copy Markdown

Contributor

barneygale marked this pull request as ready for review May 8, 2024 18:51

barneygale requested a review from zooba May 8, 2024 18:53

zooba reviewed May 13, 2024

View reviewed changes

zooba approved these changes May 13, 2024

View reviewed changes

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

barneygale enabled auto-merge (squash) May 14, 2024 17:37

nineteendo reviewed May 14, 2024

View reviewed changes

…`is_*()` (python#118243)

Suppress all `OSError` exceptions from `pathlib.Path.exists()` and `is_*()`
rather than a selection of more common errors as we do presently. Also
adjust the implementations to call `os.path.exists()` etc, which are much
faster on Windows thanks to pythonGH-101196.

One unit test fails on Python 3.14 as follows:

```
_____________ TestFolderTraversal.test_dir_without_exec_permission _____________

self = <test.unit.scan.test_folder_traversal.TestFolderTraversal object at 0x7f5530606350>
tmp_path = PosixPath('/tmp/pytest-of-sbuild/pytest-0/test_dir_without_exec_permissi0')
fs_perm_tool = <test.unit.conftest.UnixPermTool object at 0x7f55301ca510>

    @pytest.mark.skipif(
        platform.system() == 'Windows',
        reason='Unix-only filesystem permissions are tested',
    )
    def test_dir_without_exec_permission(self, tmp_path, fs_perm_tool):
        """Test that a excluded directory/file without permissions emits warnings."""
        no_perm_dir = tmp_path / 'no_perm_dir'
        no_perm_dir.mkdir()
        (no_perm_dir / 'file.txt').touch()
        (no_perm_dir / 'file2.txt').touch()
        # chmod -x no_perm_dir
        no_perm_dir.chmod(0o600)

        scan_policy = ScanPoliciesManager()
        reporter = ProgressReport(sys.stdout, False)

        folder = LocalFolder(str(tmp_path))
        local_paths = folder.all_files(reporter=reporter, policies_manager=scan_policy)
        absolute_paths = [path.absolute_path for path in local_paths]
        assert not absolute_paths

        # Check that no access warnings are issued for the excluded directory/file
>       assert set(reporter.warnings) == {
            f'WARNING: {tmp_path/"no_perm_dir/file.txt"} could not be accessed (no permissions to read?)',
            f'WARNING: {tmp_path/"no_perm_dir/file2.txt"} could not be accessed (no permissions to read?)',
        }
E       AssertionError: assert {'WARNING: /t...en symlink?)'} == {'WARNING: /t...ns to read?)'}
E
E         Extra items in the left set:
E         'WARNING: /tmp/pytest-of-sbuild/pytest-0/test_dir_without_exec_permissi0/no_perm_dir/file2.txt could not be accessed (broken symlink?)'
E         'WARNING: /tmp/pytest-of-sbuild/pytest-0/test_dir_without_exec_permissi0/no_perm_dir/file.txt could not be accessed (broken symlink?)'
E         Extra items in the right set:
E         'WARNING: /tmp/pytest-of-sbuild/pytest-0/test_dir_without_exec_permissi0/no_perm_dir/file.txt could not be accessed (no permissions to read?)'
E         'WARNING: /tmp/pytest-of-sbuild/pytest-0/test_dir_without_exec_permissi0/no_perm_dir/file2.txt could not be accessed (no permissions to read?)'
E         Use -v to get more diff

test/unit/scan/test_folder_traversal.py:721: AssertionError
```

This is due to python/cpython#118243.  I'm not
sure how important it is to preserve the exact same warning message
here, but the conservative approach is to do so;
https://docs.python.org/3/library/pathlib.html advises using `Path.stat`
to distinguish these cases.

One unit test fails on Python 3.14 as follows:

```
_____________ TestFolderTraversal.test_dir_without_exec_permission _____________

self = <test.unit.scan.test_folder_traversal.TestFolderTraversal object at 0x7f5530606350>
tmp_path = PosixPath('/tmp/pytest-of-sbuild/pytest-0/test_dir_without_exec_permissi0')
fs_perm_tool = <test.unit.conftest.UnixPermTool object at 0x7f55301ca510>

    @pytest.mark.skipif(
        platform.system() == 'Windows',
        reason='Unix-only filesystem permissions are tested',
    )
    def test_dir_without_exec_permission(self, tmp_path, fs_perm_tool):
        """Test that a excluded directory/file without permissions emits warnings."""
        no_perm_dir = tmp_path / 'no_perm_dir'
        no_perm_dir.mkdir()
        (no_perm_dir / 'file.txt').touch()
        (no_perm_dir / 'file2.txt').touch()
        # chmod -x no_perm_dir
        no_perm_dir.chmod(0o600)

        scan_policy = ScanPoliciesManager()
        reporter = ProgressReport(sys.stdout, False)

        folder = LocalFolder(str(tmp_path))
        local_paths = folder.all_files(reporter=reporter, policies_manager=scan_policy)
        absolute_paths = [path.absolute_path for path in local_paths]
        assert not absolute_paths

        # Check that no access warnings are issued for the excluded directory/file
>       assert set(reporter.warnings) == {
            f'WARNING: {tmp_path/"no_perm_dir/file.txt"} could not be accessed (no permissions to read?)',
            f'WARNING: {tmp_path/"no_perm_dir/file2.txt"} could not be accessed (no permissions to read?)',
        }
E       AssertionError: assert {'WARNING: /t...en symlink?)'} == {'WARNING: /t...ns to read?)'}
E
E         Extra items in the left set:
E         'WARNING: /tmp/pytest-of-sbuild/pytest-0/test_dir_without_exec_permissi0/no_perm_dir/file2.txt could not be accessed (broken symlink?)'
E         'WARNING: /tmp/pytest-of-sbuild/pytest-0/test_dir_without_exec_permissi0/no_perm_dir/file.txt could not be accessed (broken symlink?)'
E         Extra items in the right set:
E         'WARNING: /tmp/pytest-of-sbuild/pytest-0/test_dir_without_exec_permissi0/no_perm_dir/file.txt could not be accessed (no permissions to read?)'
E         'WARNING: /tmp/pytest-of-sbuild/pytest-0/test_dir_without_exec_permissi0/no_perm_dir/file2.txt could not be accessed (no permissions to read?)'
E         Use -v to get more diff

test/unit/scan/test_folder_traversal.py:721: AssertionError
```

This is due to python/cpython#118243.  I'm not
sure how important it is to preserve the exact same warning message
here, but the conservative approach is to do so;
https://docs.python.org/3/library/pathlib.html advises using `Path.stat`
to distinguish these cases.

`Path.is_dir` on Python 3.14 now returns False for folders without search permissions:
python/cpython#118243
Therefore, the logic in the catch block will be skipped.