GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — gh-119273: Don't run test_ioctl in a process group by vstinner · Pull Request #119275 · python/cpython

Conversation

@vstinner

@vstinner vstinner commented May 20, 2024 •
edited by bedevere-app Bot
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Member

Python test runner no longer runs tests using TTY (ex: test_ioctl) in a process group (using setsid()). Previously, tests using TTY were skipped.

  • Issue: test_ioctl is skipped because of setsid() #119273

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
@vstinner
pythongh-119273: Don't run test_ioctl in a process group
91cae2d
Python test runner no longer runs tests using TTY (ex: test_ioctl) in
a process group (using setsid()). Previously, tests using TTY were
skipped.
@bedevere-app bedevere-app Bot mentioned this pull request May 20, 2024
test_ioctl is skipped because of setsid() #119273
Closed
@vstinner

vstinner commented May 20, 2024

Copy link
Copy Markdown
Member Author

Without this change, the test is skipped:

$ ./python -m test test_ioctl -j1
(...)
test_ioctl skipped -- Unable to open /dev/tty

With the change, the test is no longer skipped:

$ ./python -m test test_ioctl -j1
(...)
Result: SUCCESS
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

gpshead
gpshead approved these changes May 21, 2024
View reviewed changes

@gpshead gpshead left a comment

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. While I wish there were a better way to avoid maintaining a special case list within libregrtest itself, the implementation of how not to use a process group is the same even if we gain that more direct ability instead of a far removed list in the future.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
Comment thread Lib/test/libregrtest/worker.py Resolved

Uh oh!

There was an error while loading. Please reload this page.

@bedevere-app bedevere-app Bot added the awaiting merge label May 21, 2024
Yhg1s
Yhg1s reviewed May 21, 2024
View reviewed changes
Comment thread Lib/test/libregrtest/worker.py

USE_PROCESS_GROUP = (hasattr(os, "setsid") and hasattr(os, "killpg"))
NEED_TTY = set('''
test_ioctl

@Yhg1s Yhg1s May 21, 2024

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There's a couple of other tests that skip some of the test methods/classes when stdout is not a tty: test_builtin, test_curses, test_os, test_shutil at least, from a quick grep. I'm not sure it's going to be maintainable to keep a list like this up to date...

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

@vstinner vstinner May 21, 2024

Copy link
Copy Markdown
Member Author

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One option would be to move "tty" tests in separated test files and disable setsid() if the test name contains _tty. For example, we already do that with GUI tests (ex: test_ttk and test_ttk_textonly).

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
@vstinner

vstinner commented May 23, 2024

Copy link
Copy Markdown
Member Author

Test on fork+setsid:

import io
import os

def check_tty(context):
    try:
        f = io.FileIO("/dev/tty", "a")
        tty = f.isatty()
        f.close()
    except OSError as exc:
        print(f"{context}: Is a TTY? {exc}")
    else:
        print(f"{context}: Is a TTY? {tty}")

check_tty("parent")
pid = os.fork()
if pid:
    os.waitpid(pid, 0)
else:
    os.setsid()
    check_tty("child after fork+setsid")

Result on Linux:

parent: Is a TTY? True
child after fork+setsid: Is a TTY? [Errno 6] No such device or address: '/dev/tty'

The child process cannot open /dev/tty after fork+setsid.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@vstinner

vstinner commented May 23, 2024 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Member Author

The following tests contains the word tty (outside comments). I checked if they are skipped when run in a worker process.

Skipped tests:

  • test_builtin: always skipped. test_input_tty*() tests need a TTY and requires that readline is not loaded.
  • test_fileio: always skipped silently. testAbles()
  • test_ioctl: always skipped (whole file).
  • test_shutil: always skipped. test_shutil.test_stty_match() requires sys.__stdout__ to be a TTY to run the command stty size. When using worker process, stdout is a pipe and not a TTY.

Tests not skipped (ok):

  • test_curses
  • test_file
  • test_getpass: accessing /dev/tty uses a mock
  • test_os: stdin is a TTY.
  • test_pty
  • test_readline
  • test_sundry
  • test_threading
  • test_tty: use os.openpty()
  • test_utf8_mode
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@vstinner

vstinner commented May 27, 2024

Copy link
Copy Markdown
Member Author

Using os.openpty() is fine with fork+setsid:

import io
import os

def check_tty(context):
    try:
        parent_fd, child_fd = os.openpty()
        tty = os.isatty(parent_fd)
        os.close(parent_fd)
        os.close(child_fd)
    except OSError as exc:
        print(f"{context}: Is a TTY? {exc}")
    else:
        print(f"{context}: Is a TTY? {tty}")

check_tty("parent")
pid = os.fork()
if pid:
    os.waitpid(pid, 0)
else:
    os.setsid()
    check_tty("child after fork+setsid")

Output:

parent: Is a TTY? True
child after fork+setsid: Is a TTY? True
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@vstinner
vstinner marked this pull request as ready for review May 29, 2024 12:43
@bedevere-app bedevere-app Bot added awaiting core review and removed awaiting merge labels May 29, 2024
@vstinner
vstinner enabled auto-merge (squash) May 29, 2024 12:44
@vstinner
vstinner merged commit 1f481fd into python:main May 29, 2024
@vstinner
vstinner deleted the regrtest_tty branch May 29, 2024 12:44
@bedevere-app bedevere-app Bot removed the awaiting core review label May 29, 2024
@vstinner

vstinner commented May 29, 2024

Copy link
Copy Markdown
Member Author

I merged this PR as it is. I will prepare a following PR to handle test_builtin, test_fileio and test_shutil.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@vstinner

vstinner commented May 29, 2024

Copy link
Copy Markdown
Member Author

test_builtin: always skipped. test_input_tty*() tests need a TTY and requires that readline is not loaded.
test_shutil: always skipped. test_shutil.test_stty_match() requires sys.stdout to be a TTY to run the command stty size. When using worker process, stdout is a pipe and not a TTY.

Sadly, these two tests need stdout to be a TTY. It's only doable if tests are run sequentially.

test_fileio: always skipped silently. testAbles()

Well, it's just a subset of a minor test. I don't think that it's worth it to create a whole test file just for a few lines.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@vstinner

vstinner commented May 29, 2024

Copy link
Copy Markdown
Member Author

Follow-up: I created issue gh-119727: Add --sequentially option to regrtest to always run tests sequentially (ignore -jN option).

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@vstinner vstinner mentioned this pull request May 30, 2024
test_ioctl fails on macOS #119770
Closed
estyxx pushed a commit to estyxx/cpython that referenced this pull request Jul 17, 2024
@vstinner @estyxx
pythongh-119273: Don't run test_ioctl in a process group (python#119275)
e2d80b1
Python test runner no longer runs tests using TTY (ex: test_ioctl) in
a process group (using setsid()). Previously, tests using TTY were
skipped.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@Yhg1s Yhg1s Yhg1s left review comments
@gpshead gpshead gpshead approved these changes

Uh oh!

There was an error while loading. Please reload this page.

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Uh oh!

There was an error while loading. Please reload this page.

3 participants

@vstinner @gpshead @Yhg1s
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
You are visitor 01402520 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.