GeezerVison

I think this probably shouldn't have a NEWS entry, because the import shutil was introduced only a couple of weeks ago, and there have been no releases from the main branch in that time. A user reading about a 15% performance improvement could be misled.

The command to run is ./python -m pyperf timeit 'import pathlib' -l 1 -w 0 -n 1 -p 250. This will ensure that you don't have warmups and don't have cached modules.

1.6ms seems suspiciously fast, are you sure the module is not being cached?

I don't think fnmatch is the issue here, I think in the icicle graph I posted above it is already counted as part of glob. Most of the import time of shutil comes from importing compression modules, lzma etc. Measured with python -Ximporttime -c 'import pathlib'. I think I used the PGO build, not sure what is the recommended build for benchmarking.

I think I used the PGO build, not sure what is the recommended build for benchmarking.

Ideally use PGO+LTO.

On macOS with PGO+LTO, running ./python.exe -X importtime -c "import pathlib", the pathlib import takes 7 ms with main and 6 ms with this PR.

You can see the big shutil block is missing underneath pathlib._local afterwards in the tuna importtime visualisations (I ran ./python.exe -c "import pathlib" && ./python.exe -X importtime -c "import pathlib" 2> import.log && tuna import.log):

Before:

After:

And with hyperfine, this branch is about 1.5 ms faster than main:

❯ hyperfine --warmup 32 \
--prepare "git checkout speedup-pathlib"  './python.exe -c "import pathlib"' \
--prepare "git checkout main"             './python.exe -c "import    pathlib"'
Benchmark 1: ./python.exe -c "import pathlib"
  Time (mean ± σ):      14.8 ms ±   0.6 ms    [User: 11.9 ms, System: 2.4 ms]
  Range (min … max):    14.1 ms …  18.2 ms    96 runs

  Warning: Statistical outliers were detected. Consider re-running this benchmark on a quiet system without any interferences from other programs.

Benchmark 2: ./python.exe -c "import    pathlib"
  Time (mean ± σ):      16.4 ms ±   0.8 ms    [User: 13.1 ms, System: 2.7 ms]
  Range (min … max):    15.6 ms …  23.2 ms    91 runs

  Warning: Statistical outliers were detected. Consider re-running this benchmark on a quiet system without any interferences from other programs.

Summary
  ./python.exe -c "import pathlib" ran
    1.11 ± 0.07 times faster than ./python.exe -c "import    pathlib"

Thank you all for the reviews, I believe I addressed all the comments.

1.6ms seems suspiciously fast, are you sure the module is not being cached?

With a caching it's in the realm of nanoseconds on my laptop. But my specs are a bit... biaised sometimes:

OS: openSUSE Leap 15.5 x86_64
Host: ROG Strix G814JZ_G814JZ 1.0
Kernel: 5.14.21-150500.55.73-default
CPU: 13th Gen Intel i9-13980HX (32) @ 5.400GHz
GPU: NVIDIA GeForce RTX 4080 Max-Q / Mobile
GPU: Intel Raptor Lake-S UHD Graphics
Memory: 4124MiB / 31698MiB

EDIT: It's a bit weird that pyperf reports timings in the realm of 1.6 ms but hyperfine does not:

Benchmark 1: ./python -c "import pathlib"
  Time (mean ± σ):      11.5 ms ±   2.1 ms    [User: 9.7 ms, System: 1.9 ms]
  Range (min … max):    10.2 ms …  20.8 ms    157 runs

  Warning: Statistical outliers were detected. Consider re-running this benchmark on a quiet system without any interferences from other programs.

Benchmark 2: ./python -c "import    pathlib"
  Time (mean ± σ):      11.9 ms ±   1.3 ms    [User: 10.5 ms, System: 1.4 ms]
  Range (min … max):    11.0 ms …  21.7 ms    149 runs

  Warning: Statistical outliers were detected. Consider re-running this benchmark on a quiet system without any interferences from other programs.

Summary
  './python -c "import pathlib"' ran
    1.03 ± 0.22 times faster than './python -c "import    pathlib"'

I'm approving the PR though it does not really change on my system much. But since it may affect other users, I think it's worth the change.

Thanks for this!

EDIT: It's a bit weird that pyperf reports timings in the realm of 1.6 ms but hyperfine does not:

hyperfine also measures the overhead of python startup.

Another possible explanation why you don't see a difference, do you have the compression modules available? You can try python -c "import lzma; import zlib; import bz2

I do have them available :( it's an interesting observation though but it's my problem I think.