Description

assignee = None
closed_at = <Date 2019-02-25.09:27:42.340>
created_at = <Date 2017-02-27.00:57:24.705>
labels = ['3.8', 'type-feature', 'library']
title = 'Expose the `length` arg from shutil.copyfileobj for public use'
updated_at = <Date 2019-02-25.09:27:42.339>
user = 'https://github.com/tgoodlet'

activity = <Date 2019-02-25.09:27:42.339>
actor = 'methane'
assignee = 'none'
closed = True
closed_date = <Date 2019-02-25.09:27:42.340>
closer = 'methane'
components = ['Library (Lib)']
creation = <Date 2017-02-27.00:57:24.705>
creator = 'goodboy'
dependencies = []
files = []
hgrepos = []
issue_num = 29659
keywords = []
message_count = 5.0
messages = ['288616', '288962', '288974', '305625', '336504']
nosy_count = 5.0
nosy_names = ['rhettinger', 'terry.reedy', 'methane', 'serhiy.storchaka', 'goodboy']
pr_nums = ['328']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue29659'
versions = ['Python 3.8']

Activity

1. tgoodlet commented on Feb 27, 2017

   

   tgoodletmannequin

   on Feb 27, 2017

   MannequinAuthor

   More actions

   It would be handy to be able to pass the length default argument defined by shutil.copyfileobj() to other public functions througout the module. When copying very large files (1GB +), increasing the memory buffer can divide the copy time in half and currently the only way to accomplish this is by overriding the needed function(s) which call shutil.copyfileobj().

   I propose a simple non-invasive change where the following functions will also expose the length kwarg and pass it downwards to copyfileobj:

   • copyfile
   • copy
   • copy2
2. tgoodlet

   added

   3.7 (EOL)end of lifeend of life

   on Feb 27, 2017
3. Mariatta

   added

   stdlibStandard Library Python modules in the Lib/ directoryStandard Library Python modules in the Lib/ directory

   on Feb 27, 2017
4. terryjreedy

   added

   type-featureA feature request or enhancementA feature request or enhancement

   on Mar 3, 2017
5. rhettinger commented on Mar 4, 2017

   

   rhettinger

   on Mar 4, 2017

   Contributor

   More actions

   This seems reasonable to me.
6. methane commented on Mar 4, 2017

   

   methane

   on Mar 4, 2017

   Member

   More actions

   How about increasing default value to 32KiB or 64KiB too?

   binutils's cp works well for most cases, while it doesn't have option to specify blocksize.
   I want Python's copy functions works nice enough for common cases too.

   binutils cp uses 128KiB block for normal file. see more information in https://github.com/coreutils/coreutils/blob/master/src/ioblksize.h

   But copyfileobj can be used other file-likes.
   On Linux, typical socket buffer size is 128KiB, and typical pipe buffer size is 64KiB. So block size larger than 64KiB may cause unneeded blocking.
   That's why I suggested 32KiB or 64KiB.
7. serhiy-storchaka commented on Nov 6, 2017

   

   serhiy-storchaka

   on Nov 6, 2017

   Member

   More actions

   Could you please provide any benchmarks Tyler?
8. methane commented on Feb 25, 2019

   

   methane

   on Feb 25, 2019

   Member

   More actions

   bpo-33671 fixed this.
9. methane

   added

   3.8 (EOL)end of lifeend of life

   and removed

   3.7 (EOL)end of lifeend of life

   on Feb 25, 2019
10. methane

    closed this as completedon Feb 25, 2019
11. ezio-melotti

    transferred this issue fromon Apr 10, 2022

Issue actions