GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — GH-101362: Omit path anchor from `pathlib.PurePath()._parts` by barneygale · Pull Request #102476 · python/cpython

Conversation

@barneygale

@barneygale barneygale commented Mar 6, 2023 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Contributor

Improve performance of path construction by skipping the addition of the path anchor (drive + root) to the internal _parts list. Rename this attribute to _tail for clarity.

Also:

  • Simplify the implementations of name, parent, and parents a little.
  • Optimize _make_child_relpath() by building the new string path with naive string concatination rather than _format_parsed_parts().
  • Optimize __hash__() and __eq__() by referring to the unsplit case-normalized path.
  • Optimize __lt__() etc by referring to the naively split case-normalized path (i.e. split on sep without consideration of anchor)

The public parts tuple is unaffected.

$ ./python -m timeit -s 'from pathlib import PureWindowsPath as P' 'P("C:/foo/bar.py")'
100000 loops, best of 5: 1.98 usec per loop  # before
200000 loops, best of 5: 1.75 usec per loop  # after
  • Issue: Optimize pathlib path construction #101362

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
All reactions
@barneygale
pythonGH-101362: Omit path anchor from pathlib.PurePath()._parts
8bf4600 
Improve performance of path construction by skipping the addition of the
path anchor (`drive + root`) to the internal `_parts` list. This change
allows us to simplify the implementations of `joinpath()`, `name`,
`parent`, and `parents` a little. The public `parts` tuple is unaffected.
@bedevere-bot bedevere-bot mentioned this pull request Mar 6, 2023
Optimize pathlib path construction #101362
Closed
@bedevere-bot bedevere-bot added the awaiting review label Mar 6, 2023
@barneygale

barneygale commented Mar 6, 2023 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Contributor Author

I'm not sure about this change - I don't like how it makes _parts and parts have different contents. Too subtle for future maintainers to pick up on, perhaps?

edit: addressed by renaming _parts to _tail.
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
barneygale added 2 commits March 6, 2023 18:50
@barneygale
Add news blurb.
e7a58c3
@barneygale
Rename _parts to _tail for clarity.
1d58aed
@barneygale barneygale added performance Performance or resource usage topic-pathlib labels Mar 6, 2023
barneygale added 2 commits March 10, 2023 17:51
@barneygale
Merge branch 'main' into pythongh-101362-exclude-anchor-from-parts
ac26f84
@barneygale
Undo some changes to _format_parsed_parts()
cce5d7b
@barneygale barneygale mentioned this pull request Mar 12, 2023
Path.rglob performance issues in deeply nested directories compared to glob.glob(recursive=True) #102613
Closed
@barneygale

barneygale commented Mar 12, 2023 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Contributor Author

This change is necessary in order to efficiently fix #65238, which is required before we can return unnormalized paths from __fspath__() in #102783.
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
@barneygale barneygale mentioned this pull request Mar 13, 2023
pathlib strips trailing slash #65238
Closed
barneygale added 3 commits April 3, 2023 20:50
@barneygale
Merge branch 'main' into pythongh-101362-exclude-anchor-from-parts
f529d51
@barneygale
Move str() call out of initialiser.
0c0f5bd
@barneygale
Optimize _make_child_relpath()
7a4e92f
@barneygale

barneygale commented Apr 3, 2023 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Contributor Author

Updated timings:

$ ./python -m timeit -s 'from pathlib import PureWindowsPath as P' 'str(P("C:/foo/bar.py"))'
100000 loops, best of 5: 3.92 usec per loop  # before
100000 loops, best of 5: 3.57 usec per loop  # after

Something like a 10% speedup.
AlexWaygood reacted with rocket emoji
All reactions
  • 🚀 1 reaction

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
barneygale added 3 commits April 4, 2023 00:54
@barneygale
Remove caching of Path.parts
c8d4b38
@barneygale
Cache case-normalized string.
d2a578d
@barneygale
Rename _make_child back to _make_child_relpath
de4df7a
AlexWaygood
AlexWaygood reviewed Apr 8, 2023
View reviewed changes
Comment thread Lib/test/test_pathlib.py Resolved

Uh oh!

There was an error while loading. Please reload this page.
barneygale added 2 commits April 8, 2023 16:37
@barneygale
Add comments to PurePath.__slots__
5a34f7e
@barneygale
Merge branch 'main' into pythongh-101362-exclude-anchor-from-parts
078ced9
@barneygale
barneygale merged commit 2c673d5 into python:main Apr 9, 2023
@bedevere-bot bedevere-bot removed the awaiting review label Apr 9, 2023
warsaw pushed a commit to warsaw/cpython that referenced this pull request Apr 11, 2023
@barneygale @warsaw
pythonGH-101362: Omit path anchor from pathlib.PurePath()._parts (p…
f81083c 
…ythonGH-102476)

Improve performance of path construction by skipping the addition of the path anchor (`drive + root`) to the internal `_parts` list. Rename this attribute to `_tail` for clarity.
aisk pushed a commit to aisk/cpython that referenced this pull request Apr 18, 2023
@barneygale @aisk
pythonGH-101362: Omit path anchor from pathlib.PurePath()._parts (p…
86b1904 
…ythonGH-102476)

Improve performance of path construction by skipping the addition of the path anchor (`drive + root`) to the internal `_parts` list. Rename this attribute to `_tail` for clarity.
@domdfcoding domdfcoding mentioned this pull request Apr 19, 2023
PurePosixPath no longer correctly parses PureWindowsPath #103631
Closed
@anjakefala anjakefala mentioned this pull request Jun 26, 2023
Path test failure with Python 3.12 saulpw/visidata#1934
Closed
anjakefala added a commit to saulpw/visidata that referenced this pull request Jun 27, 2023
@anjakefala
[path] update Path for Python 3.12
2b3a569 
`pathlib.Path._parts` has been removed in Python 3.12:
python/cpython#102476

Switch to pathlib.Path.parts which is a tuple.

Closes #1934
anjakefala added a commit to saulpw/visidata that referenced this pull request Jun 27, 2023
@anjakefala
[path] update Path for Python 3.12
f5cabec 
`pathlib.Path._parts` has been removed in Python 3.12:
python/cpython#102476

Switch to pathlib.Path.parts which is a tuple.

Closes #1934
anjakefala added a commit to saulpw/visidata that referenced this pull request Jul 16, 2023
@anjakefala
[path] update Path for Python 3.12
910ca47 
`pathlib.Path._parts` has been removed in Python 3.12:
python/cpython#102476

Switch to pathlib.Path.parts which is a tuple.

Closes #1934
anjakefala added a commit to saulpw/visidata that referenced this pull request Jul 17, 2023
@anjakefala
[path] update Path for Python 3.12
06e7f32 
`pathlib.Path._parts` has been removed in Python 3.12:
python/cpython#102476

Switch to pathlib.Path.parts which is a tuple.

Closes #1934
anjakefala added a commit to saulpw/visidata that referenced this pull request Jul 17, 2023
@anjakefala
[path] update Path for Python 3.12
a271b44 
`pathlib.Path._parts` has been removed in Python 3.12:
python/cpython#102476

Switch to pathlib.Path.parts which is a tuple.

Closes #1934
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@AlexWaygood AlexWaygood AlexWaygood left review comments

Uh oh!

There was an error while loading. Please reload this page.

Assignees

No one assigned

Labels

performance Performance or resource usage topic-pathlib

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Uh oh!

There was an error while loading. Please reload this page.

3 participants

@barneygale @AlexWaygood @bedevere-bot
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
You are visitor 01402503 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.