Description

assignee = None
closed_at = None
created_at = <Date 2021-12-01.05:08:15.763>
labels = ['3.11', 'expert-IO', 'performance']
title = 'Avoid calling isatty() for most open() calls'
updated_at = <Date 2021-12-01.11:53:13.388>
user = 'https://github.com/collinanderson'

activity = <Date 2021-12-01.11:53:13.388>
actor = 'eryksun'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = ['IO']
creation = <Date 2021-12-01.05:08:15.763>
creator = 'collinanderson'
dependencies = []
files = []
hgrepos = []
issue_num = 45944
keywords = ['patch']
message_count = 3.0
messages = ['407427', '407434', '407444']
nosy_count = 6.0
nosy_names = ['pitrou', 'benjamin.peterson', 'stutzbach', 'serhiy.storchaka', 'eryksun', 'collinanderson']
pr_nums = ['29870']
priority = 'normal'
resolution = None
stage = 'patch review'
status = 'open'
superseder = None
type = 'performance'
url = 'https://bugs.python.org/issue45944'
versions = ['Python 3.11']

Activity

1. collinanderson

   added

   3.7 (EOL)end of lifeend of life

   3.8 (EOL)end of lifeend of life

   3.9 (EOL)end of lifeend of life

   3.10 (EOL)end of lifeend of life

   3.11only security fixesonly security fixes

   topic-IO

   performancePerformance or resource usagePerformance or resource usage

   on Dec 1, 2021
2. collinanderson commented on Dec 1, 2021

   

   collinandersonmannequin

   on Dec 1, 2021

   MannequinAuthor

   More actions

   isatty() is a system call on linux. Most open()s are files, and we're already getting the size of the file. If it has a size, then we know it's not a atty, and can avoid calling it.
3. 17 remaining items

4. serhiy-storchaka commented on Dec 14, 2023

   

   serhiy-storchaka

   on Dec 14, 2023

   Member

   More actions

   One test fails with #112495 if the stdin is not a terminal. For example:

   $ ./python -m test -v test_builtin -m test_input_no_stdout_fileno <python
   ...
   test test_builtin failed -- Traceback (most recent call last):
     File "/home/serhiy/py/cpython/Lib/test/test_builtin.py", line 2393, in test_input_no_stdout_fileno
       self.assertSequenceEqual(lines, expected)
       ~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^
   AssertionError: Sequences differ: ['stdin.isatty(): False', "captured: 'prompt'"] != ('stdin.isatty(): True', "captured: 'prompt'")
   
   First differing element 0:
   'stdin.isatty(): False'
   'stdin.isatty(): True'
   
   - ['stdin.isatty(): False', "captured: 'prompt'"]
   ? ^                 ^^^^                        ^
   
   + ('stdin.isatty(): True', "captured: 'prompt'")
   ? ^                 ^^^                        ^
   

   This is because the result of isatty() is not constant during the lifetime of the file object. The file descriptor can be reassigned to other file or device, so it should not be cached.

   #29870 uses the cached value only in open(), immediately after creating the FileIO object, so it is not affected by this. But it is more complex, and making it more portable will make it even more complex. How much time it saves?
5. vstinner commented on Jul 11, 2024

   

   vstinner

   on Jul 11, 2024

   Member

   More actions

   PR gh-112495 was closed. @serhiy-storchaka wrote:

   It does not work, because the file descriptor can be reassigned from tty to non-tty and vice versa. The result of isatty() can not be cached.

   Should we close the issue?
6. cmaloney commented on Aug 25, 2024

   

   cmaloney

   on Aug 25, 2024

   Contributor

   More actions

   I think it's okay to rely on the fd not being modified inside of open() which is what #121593 does. It doesn't cache isatty generally (any future call still does a isatty libc call), just says "we can skip this call/check during open if we already have enough information from stat that the fd isn't a TTY".

   Reacted by Victor Stinner
7. cmaloney

   added 2 commits that reference this issue on Sep 10, 2024

   pythongh-90102: Shortcut isatty in _pyio open()

   1a16077

   pythongh-90102: Shortcut isatty in _io open()

   be84f90
8. cmaloney

   added 2 commits that reference this issue on Oct 3, 2024

   pythongh-90102: Shortcut isatty in _pyio open()

   a1a1cb7

   pythongh-90102: Shortcut isatty in _io open()

   b07dac7
9. vstinner

   added a commit that references this issue on Oct 8, 2024

   gh-90102: Remove isatty call during regular open (#124922)

   cc9b9be
10. cmaloney

    added a commit that references this issue on Oct 8, 2024

    pythongh-90102: Fix pyio return value

    f68e2b8
11. vstinner

    added a commit that references this issue on Oct 8, 2024

    gh-90102: Fix pyio _isatty_open_only() (#125089)

    43ad3b5
12. vstinner commented on Oct 8, 2024

    

    vstinner

    on Oct 8, 2024

    Member

    More actions

    Optimization implemented by change cc9b9be.

    Reacted by Collin Anderson
13. vstinner

    closed this as completedon Oct 8, 2024
14. cmaloney

    added a commit that references this issue on Oct 8, 2024

    Update to validate pythongh-90102 changes

    c2715b1
15. efimov-mikhail

    added 2 commits that reference this issue on Oct 9, 2024

    pythongh-90102: Remove isatty call during regular open (python#124922)

    c214282

    pythongh-90102: Fix pyio _isatty_open_only() (python#125089)

    148e3fe

Issue actions