GeezerVison

The World Wide Web, restored to 800 pixels of glory.

GeezerVison is showing github.com — shutil.make_archive should not need to chdir (alternatively: make shutil.make_archive thread-safe) · Issue #74696 · python/cpython

shutil.make_archive should not need to chdir (alternatively: make shutil.make_archive thread-safe) #74696

New issue

Copy link

New issue

Copy link

Closed

shutil.make_archive should not need to chdir (alternatively: make shutil.make_archive thread-safe)#74696

Copy link

Assignees

Labels

stdlibStandard Library Python modules in the Lib/ directoryStandard Library Python modules in the Lib/ directorytype-bugAn unexpected behavior, bug, or errorAn unexpected behavior, bug, or errortype-featureA feature request or enhancementA feature request or enhancement

Description

AlexGaynor

mannequin

opened

on May 30, 2017

Issue body actions

BPO	30511
Nosy	@tarekziade, @ambv, @serhiy-storchaka, @miss-islington, @michael-o, @FFY00, @akulakov
PRs	bpo-30511: Add note on thread safety to shutil.make_archive() #26933 [3.10] bpo-30511: Add note on thread safety to shutil.make_archive() (GH-26933) #27274 [3.9] bpo-30511: Add note on thread safety to shutil.make_archive() (GH-26933) #27275 [3.8] bpo-30511: Add note on thread safety to shutil.make_archive() (GH-26933) #27276 bpo-25625: add contextlib.chdir #28271

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = None
created_at = <Date 2017-05-30.13:46:18.840>
labels = ['3.7', 'library']
title = 'shutil.make_archive should not need to chdir (alternatively: make shutil.make_archive thread-safe)'
updated_at = <Date 2021-09-17.13:26:13.713>
user = 'https://bugs.python.org/AlexGaynor'

bugs.python.org fields:

activity = <Date 2021-09-17.13:26:13.713>
actor = 'FFY00'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = ['Library (Lib)']
creation = <Date 2017-05-30.13:46:18.840>
creator = 'Alex Gaynor'
dependencies = []
files = []
hgrepos = []
issue_num = 30511
keywords = ['patch']
message_count = 10.0
messages = ['294756', '294758', '294759', '294760', '297902', '389849', '396633', '397928', '397932', '397934', '397935']
nosy_count = 9.0
nosy_names = ['tarek', 'lukasz.langa', 'serhiy.storchaka', 'Alex Gaynor', 'Joey Harrington', 'miss-islington', 'michael-o', 'FFY00', 'andrei.avk']
pr_nums = ['26933', '27274', '27275', '27276', '28271']
priority = 'normal'
resolution = None
stage = 'patch review'
status = 'open'
superseder = None
type = None
url = 'https://bugs.python.org/issue30511'
versions = ['Python 2.7', 'Python 3.7']

Activity

AlexGaynor commented on May 30, 2017
AlexGaynormannequin
on May 30, 2017
MannequinAuthor
More actions
Currently shutil.make_archive uses os.chdir, however there's no need for that. Everything that's done could be equally accomplished with path manipulation:

https://github.com/python/cpython/blob/master/Lib/shutil.py#L773-L779

We should switch to using path manipulation in order to make shutil.make_archive thread safe.

(Flag: This is probably a good bug for someone with Python skills interested in contributing to CPython!)
AlexGaynor
added
3.7 (EOL)end of lifeend of life

stdlibStandard Library Python modules in the Lib/ directoryStandard Library Python modules in the Lib/ directory
on May 30, 2017
serhiy-storchaka commented on May 30, 2017
serhiy-storchaka
on May 30, 2017
Member
More actions
Unfortunately this is not possible. The signature of functions registered with register_archive_format() doesn't support this.
AlexGaynor commented on May 30, 2017
AlexGaynormannequin
on May 30, 2017
MannequinAuthor
More actions
None of those functions are a public API, so changing them shouldn't be a problem IMO.
AlexGaynor commented on May 30, 2017
AlexGaynormannequin
on May 30, 2017
MannequinAuthor
More actions
Ugh, except via |register_archive_format|. |register_archive_format| could wrap callables passed to it to maintain the current behavior.
JoeyHarrington commented on Jul 7, 2017
JoeyHarringtonmannequin
on Jul 7, 2017
Mannequin
More actions
It would be nice if there was at least a warning in the docs that make_archive is not thread-safe, and that if you have two threads creating archives that it's extremely likely you'll get erroneous results since the race condition lasts for the entire duration of the archive creation.
michael-o commented on Mar 30, 2021
michael-omannequin
on Mar 30, 2021
Mannequin
More actions
Just wasted two hours for this. Can someone really update the documentation of it if this is not going to change.
akulakov commented on Jun 28, 2021
akulakov
on Jun 28, 2021
Contributor
More actions
PR is added here:
#26933
ambv commented on Jul 21, 2021
ambv
on Jul 21, 2021
Contributor
More actions
New changeset 64f54b7 by andrei kulakov in branch 'main':
bpo-30511: Add note on thread safety to shutil.make_archive() (bpo-26933)
64f54b7
ambv commented on Jul 21, 2021
ambv
on Jul 21, 2021
Contributor
More actions
New changeset d17449f by Miss Islington (bot) in branch '3.10':
bpo-30511: Add note on thread safety to shutil.make_archive() (GH-26933) (GH-27274)
d17449f
ambv commented on Jul 21, 2021
ambv
on Jul 21, 2021
Contributor
More actions
New changeset c8e35ab by Miss Islington (bot) in branch '3.9':
bpo-30511: Add note on thread safety to shutil.make_archive() (GH-26933) (GH-27275)
c8e35ab
ambv commented on Jul 21, 2021
ambv
on Jul 21, 2021
Contributor
More actions
New changeset 54d3871 by Miss Islington (bot) in branch '3.8':
bpo-30511: Add note on thread safety to shutil.make_archive() (GH-26933) (bpo-27276)
54d3871
ezio-melotti
transferred this issue from on Apr 10, 2022
serhiy-storchaka
added this to Threading issues 🧵on May 20, 2022
14 remaining items
ambv
added 2 commits that reference this issue on Jun 22, 2022
gh-74696: Do not change the current working directory in shutil.make_…
c1bfff4
gh-74696: Do not change the current working directory in shutil.make_…
e51a22b
ambv commented on Jun 22, 2022
ambv
on Jun 22, 2022
Contributor
More actions
@gpshead I'm +1 to deprecate register_archive_format() in 3.12 and remove in 𝛑-Py. Can we use this issue for that or should I open a new one?
serhiy-storchaka commented on Jun 22, 2022
serhiy-storchaka
on Jun 22, 2022
Member
More actions
I was going to extend register_archive_format() by adding support of custom unpackers which support root_dir.

It can be implemented either by adding a boolean parameter supports_root_dir in register_archive_format(), or checking the supports_root_dir attribute of the unpacker function.
ambv commented on Jun 22, 2022
ambv
on Jun 22, 2022
Contributor
More actions
Sure, Serhiy. Greg's point is that this functionality looks unused so it's probably an unnecessary support burden.
serhiy-storchaka commented on Jun 22, 2022
serhiy-storchaka
on Jun 22, 2022
Member
More actions
py7zr provides helpers to be registered in shutil. Other archiver modules can provide such helpers too or even register them by default. Removing register_archive_format() and register_unpack_format() will make other archive formats second-class citizens.
serhiy-storchaka
added a commit that references this issue on Jun 25, 2022
pythongh-74696: Pass root_dir to custom archivers which support it
e5ebb8d
serhiy-storchaka commented on Jun 25, 2022
serhiy-storchaka
on Jun 25, 2022
Member
More actions
I have implemented the second option in #94251. It seems better because does not need a collaboration between the author and users of the custom archiver.
serhiy-storchaka
added a commit that references this issue on Jun 26, 2022
Revert 30935 defer bpo45162 to 312 (#94285)
1ffde3e
serhiy-storchaka
added a commit that references this issue on Oct 5, 2022
gh-74696: Pass root_dir to custom archivers which support it (GH-94251)
e3ef400
carljm
added a commit that references this issue on Oct 6, 2022
Merge branch 'main' into dictwatch
4bb5d9d
mpage
added a commit that references this issue on Oct 11, 2022
pythongh-74696: Pass root_dir to custom archivers which support it (p…
c34bef6
serhiy-storchaka
closed this as completed on Oct 27, 2022
Repository owner moved this from Todo to Done in Threading issues 🧵on Oct 27, 2022

Metadata

Assignees

serhiy-storchaka

Labels

Projects

Threading issues 🧵
Status
Done
Show more project fields

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions

You are visitor 01402503 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.

GeezerVison

shutil.make_archive should not need to chdir (alternatively: make shutil.make_archive thread-safe) #74696

Description

Activity

AlexGaynor commented on May 30, 2017

serhiy-storchaka commented on May 30, 2017

AlexGaynor commented on May 30, 2017

AlexGaynor commented on May 30, 2017

JoeyHarrington commented on Jul 7, 2017

michael-o commented on Mar 30, 2021

akulakov commented on Jun 28, 2021

ambv commented on Jul 21, 2021

ambv commented on Jul 21, 2021

ambv commented on Jul 21, 2021

ambv commented on Jul 21, 2021

14 remaining items

ambv commented on Jun 22, 2022

serhiy-storchaka commented on Jun 22, 2022

ambv commented on Jun 22, 2022

serhiy-storchaka commented on Jun 22, 2022

serhiy-storchaka commented on Jun 25, 2022

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions