GeezerVison is showing github.com — Add tarfile.TarPath · Issue #89812 · python/cpython
Add tarfile.TarPath #89812Description
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
assignee = None
closed_at = None
created_at = <Date 2021-10-28.15:59:21.866>
labels = ['type-feature', 'library', '3.11']
title = 'Add tarinfo.Path'
updated_at = <Date 2022-01-02.00:01:26.196>
user = 'https://github.com/FFY00' bugs.python.org fields:
activity = <Date 2022-01-02.00:01:26.196>
actor = 'barneygale'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = ['Library (Lib)']
creation = <Date 2021-10-28.15:59:21.866>
creator = 'FFY00'
dependencies = []
files = []
hgrepos = []
issue_num = 45649
keywords = []
message_count = 6.0
messages = ['405194', '405206', '405210', '409479', '409481', '409482']
nosy_count = 4.0
nosy_names = ['jaraco', 'ethan.furman', 'FFY00', 'barneygale']
pr_nums = []
priority = 'normal'
resolution = None
stage = None
status = 'open'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue45649'
versions = ['Python 3.11'] Linked PRs
ActivityFFY00 commented on Oct 28, 2021 It would be helpful to have a pathlib-compatible object in tarfile, similarly to zipfile.Path. jaraco commented on Oct 28, 2021 I vaguely recall exploring this concept and finding that tarfiles don’t supply the requisite interface because they’re not random access. I’m only 10% confident in that recollection, so worth exploring. FFY00 commented on Oct 28, 2021 That is good to know. This isn't very high on my priority list, but I will try to explore when I have some time. barneygale commented on Jan 1, 2022 It's possible to do, but will be a little slow due to the nature of tar files. They're a big linked list of files, so you need to do a bunch of reads/seeks from the start to the end to enumerate all files.
I'd ask that we try to get bpo-24132 solved first. That would let us write:
# tarfile.py
class Path(pathlib.AbstractPath):
def iterdir(self):
...
def stat(self):
...We'd fill in a smallish number of abstract methods to get a full Path-compatible class with read_text(), is_symlink() etc methods. jaraco commented on Jan 1, 2022 I'd recommend not to block on bpo-24132. It's not obvious to me that subclassing would be valuable. It depends on how it's implemented, but in my experience, zipfile.Path doesn't and cannot implement the full interface of pathlib.Path. Instead zipfile.Path attempts to implement a protocol. At the time, the protocol was undefined, but now there exists importlib.resources.abc.Traversable (https://docs.python.org/3/library/importlib.html#importlib.abc.Traversable), the interface needed by importlib.resources. I'd honestly just create a standalone class, see if it can implement Traversable, and only then consider if it should implement a more complicated interface (such as something with symlink support or perhaps even later subclassing from pathlib.Path). barneygale commented on Jan 2, 2022 If you're only aiming for Traversable compatibility, sure.
The original bug description asks for something that's pathlib-compatible and similar to zipfile.Path, which goes beyond the Traversable interface in attempting to emulate pathlib.Path.
The pathlib.Path interface is a good one - I see no reason it can't apply to zip and tar archives in full. Methods of Path objects already raise NotImplementedError if operations aren't supported (e.g. creating symlinks)
Some prototyping from a couple years back, including a tar path implementation: https://github.com/barneygale/pathlab/tree/master/pathlab encukou commented on May 3, 2023 Anyone implementing this should be aware that tar members don't have unique names.
A path should generally refer to the last member of the given name (which would overwrite previous ones when extracting -- unless the previous members do shenanigans with symlinks/directories, but API for accessing individual members isn't expected to take that into account).
TarFile API like getmember will take care of most of this, but not all: for example, a “listdir” operation should probably de-duplicate its result. barneygale commented on Jul 3, 2023 PR available that adds tarfile.TarPath! #106337 barneygale commented on Jul 9, 2023 What should tarfile.TarPath do with symlinks in archives? In the PR above they're followed in the same circumstances as in pathlib.PosixPath, but that might not be desirable/safe default. We could add an initialiser argument that disabled following symlinks entirely - either omit them or represent them as regular files, somehow?
If the answer isn't clear then I'd like to incubate this in a PyPI project for a while first
@encukou I saw you were working on tarfile recently. Pinging in case this interests you! encukou commented on Jul 10, 2023 IMO, symlinks to files that are in the archive should continue to work.
Perhaps relative symlinks to files within the archive root should work too, though these become dangerous if there are any directory symlinks pointing outside.
In general, if an archive contains directory symlinks, you can only get the final destination of a file by extracting the archive (or by doing a very elaborate dry run, for which tarfile isn't suited).
It's very much not clear-cut. A tarball is, essentially, a collection of procedural instructions that allow merging contents with an existing filesystem in ”interesting” ways, while TarPath would like to work with a final tree structure.
IMO, to answer questions like this, we first need a good conceptual model of what TarPath represents.
A PyPI project that handles the simple cases sounds like a good start. barneygale commented on Jul 12, 2023 Thank you, @encukou! I'll move the TarPath implementation in #106337 into an experimental PyPI package then, leaving just the (private) pathlib._VirtualPath in that PR. encukou commented on Jul 13, 2023
we first need a conceptual model of what TarPath represents
FWIW, I won't make one any time soon -- I'm taking a break from tarfile after a few months of dealing with it.
But if you want to bounce ideas off me, I'm here.
MetadataMetadataStatusShow more project fields
DevelopmentNo branches or pull requestsIssue actions
|