Description

ncoghlan

opened on May 7, 2018

Issue body actions

BPO	33440
Nosy	@ncoghlan, @pitrou, @njsmith, @serhiy-storchaka
PRs	bpo-33440: Defer imports in pathlib to reduce its import time. #6820

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = None
created_at = <Date 2018-05-07.12:28:40.679>
labels = ['3.8', 'library', 'performance']
title = 'Possible lazy import opportunities in `pathlib`'
updated_at = <Date 2018-05-14.20:03:34.394>
user = 'https://github.com/ncoghlan'

bugs.python.org fields:

activity = <Date 2018-05-14.20:03:34.394>
actor = 'serhiy.storchaka'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = ['Library (Lib)']
creation = <Date 2018-05-07.12:28:40.679>
creator = 'ncoghlan'
dependencies = []
files = []
hgrepos = []
issue_num = 33440
keywords = ['patch']
message_count = 4.0
messages = ['316266', '316276', '316567', '316570']
nosy_count = 4.0
nosy_names = ['ncoghlan', 'pitrou', 'njs', 'serhiy.storchaka']
pr_nums = ['6820']
priority = 'normal'
resolution = None
stage = 'patch review'
status = 'open'
superseder = None
type = 'performance'
url = 'https://bugs.python.org/issue33440'
versions = ['Python 3.8']

Linked PRs

• GH-77621: Delay some imports from pathlib #112244

Activity

1. ncoghlan commented on May 7, 2018

   

   ncoghlan

   on May 7, 2018

   ContributorAuthor

   More actions

   Due to a python-ideas discussion about reducing boilerplate for __file__-relative path calculations, I was running "./python -X importtime -S -c 'import pathlib'" and noticed three potential candidates where it may be worthwhile deferring the imports until the modules are actually needed:

   • re (used in _WildcardSelector)
   • fnmatch (used in PurePath.match and _WildcardSelector)
   • urllib.parse (used in PurePath.as_uri, by way of self._flavour.make_uri)

   Using an optimised Python 3.7 on an SSD with warm disk caches, commenting out those 3 imports reduced my typical import times for pathlib from 12-13 ms to 7-8 ms)
2. ncoghlan

   added

   3.8 (EOL)end of lifeend of life

   stdlibStandard Library Python modules in the Lib/ directoryStandard Library Python modules in the Lib/ directory

   performancePerformance or resource usagePerformance or resource usage

   on May 7, 2018
3. njsmith commented on May 7, 2018

   

   njsmith

   on May 7, 2018

   Contributor

   More actions

   Also it's a smaller win, but it might be worth considering whether we can avoid the import of 'nt' on posix platforms and vice-versa.
4. serhiy-storchaka commented on May 14, 2018

   

   serhiy-storchaka

   on May 14, 2018

   Member

   More actions

   Did you mean the idea about module-level alias to dirname(file)? I think this is a dead horse. See bpo-33277.
5. serhiy-storchaka commented on May 14, 2018

   

   serhiy-storchaka

   on May 14, 2018

   Member

   More actions

   Personally I support moving imports of rarely used modules into functions that need them. But this is against PEP-8 and should be discussed more widely. Perhaps this rule should be weaken, we have to state what exceptions are permitted.
6. ezio-melotti

   transferred this issue fromon Apr 10, 2022
7. brettcannon

   added

   topic-pathlib

   on Jun 21, 2022
8. iritkatriel

   added

   3.12only security fixesonly security fixes

   and removed

   3.8 (EOL)end of lifeend of life

   on Sep 7, 2022
9. barneygale commented on Nov 18, 2023

   

   barneygale

   on Nov 18, 2023

   Contributor

   More actions

   I've moved the urllib imports into as_uri() / from_uri() as their use is uncommon. Everything else is likely to be hit in general usage, I think.
10. AlexWaygood commented on Nov 18, 2023

    

    AlexWaygood

    on Nov 18, 2023

    Member

    More actions

    You could probably make this one lazy:

    cpython/Lib/pathlib.py

    Line 17 in 607b5e3

    import warnings

    Users probably shouldn't pay the cost of importing warnings unless they're actually using a deprecated function. There are quite a few uses of warnings in pathlib, though, which is why I held off from changing warnings to be a lazy import in pathlib as part of #110286 -- maybe making it a lazy import in pathlib just adds too much ugliness for it to be worthwhile.

    The import-time speedup from making warnings a lazy import is small, but measurable.
11. barneygale commented on Nov 18, 2023

    

    barneygale

    on Nov 18, 2023

    Contributor

    More actions

    Three of four uses of warnings will be removed in 3.14, so perhaps we could look again when main becomes 3.14?

    Reacted by Alex Waygood
12. AlexWaygood commented on Nov 18, 2023

    

    AlexWaygood

    on Nov 18, 2023

    Member

    More actions

    Three of four uses of warnings will be removed in 3.14, so perhaps we could look again when main becomes 3.14?

    SGTM!
13. barneygale

    added 2 commits that reference this issue on Nov 18, 2023

    pythonGH-77621: Delay some imports from pathlib

    90c3018

    GH-77621: Delay some imports from pathlib (#112244)

    bbb4367
14. barneygale

    added this to pathlib issueson Dec 2, 2023
15. barneygale

    moved this to In Progress in pathlib issueson Dec 4, 2023
16. barneygale commented on Dec 23, 2023

    

    barneygale

    on Dec 23, 2023

    Contributor

    More actions

    Resolving this issue as I believe there's nothing left to do!

    • The urllib import was made lazy in GH-107465: Add pathlib.Path.from_uri() classmethod. #107640
    • The fnmatch, glob and re imports were made lazy in GH-77621: Delay some imports from pathlib #112244
    • The top-level warnings import will be removed when we branch for Python 3.14
    • Everything else is cheap or needed at a module level

    Reacted by Daniel Hollas and Itamar Oren

    Reacted by Alex Waygood
17. barneygale

    closed this as completedon Dec 23, 2023
18. github-project-automation

    moved this from In Progress to Done in pathlib issueson Dec 23, 2023
19. aisk

    added a commit that references this issue on Feb 11, 2024

    pythonGH-77621: Delay some imports from pathlib (python#112244)

    ab8fe83
20. Glyphack

    added a commit that references this issue on Sep 2, 2024

    pythonGH-77621: Delay some imports from pathlib (python#112244)

    03a4396

Issue actions