GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — gh-94518: [_posixsubprocess] Replace variable validity flags with reserved values by arhadthedev · Pull Request #94687 · python/cpython

Conversation

@arhadthedev

@arhadthedev arhadthedev commented Jul 8, 2022 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Member

Currently, _posixsubprocess.c uses group and user ids together with flags of whether the corrsponding variables were initialized. However such an implicit "either initialized or look somewhere else" confuses both a reader (another mental connection to constantly track between functions) and a compiler (warnings on potentially uninitialized variables). Instead, we can utilize a special group/user id as a flag value defined by POSIX but uses nowhere else. Namely:

  • gid: call_setgid = False → gid = -1
  • uid: call_setuid = False → uid = -1
  • groups: call_setgroups = False → groups = NULL (obtained with (groups_list != Py_None) ? groups : NULL)

This PR is required for gh-94519.


PS.: @tiran I applied your suggestion from gh-94519 as this, separate PR to keep that one limited in scope and reviewable.

  • Issue: Port 23-argument _posixsubprocess.fork_exec to Argument Clinic #94518

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
@arhadthedev
Replace call_setgid/call_setuid in _posixsubprocess with gid/uid==-1
1b3250e
@arhadthedev
arhadthedev requested a review from gpshead as a code owner July 8, 2022 11:31
@bedevere-bot bedevere-bot added the awaiting review label Jul 8, 2022
@arhadthedev arhadthedev changed the title gh-94518: [_posixsubprocess] Replace each variable validity flag with a reserved value of that variable gh-94518: [_posixsubprocess] Replace variable validity flags with reserved values Jul 8, 2022
@arhadthedev

arhadthedev commented Jul 12, 2022

Copy link
Copy Markdown
Member Author

This change has no user-visible effect so no NEWS entry is required.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@arhadthedev
Merge branch 'main' into posixsubprocess-no-callsetxid
c292331
@arhadthedev arhadthedev mentioned this pull request Jul 22, 2022
gh-94518: Port 23-argument _posixsubprocess.fork_exec to Argument Clinic #94519
Merged
@arhadthedev
Add a NEWS entry
7d431a3
@arhadthedev

arhadthedev commented Jul 22, 2022

Copy link
Copy Markdown
Member Author

Please include a news entry.

We typically only skip news if the change is a trivial internal change or change is already covered by a previous PR. Non-trivial commits get a news entry so other core devs, release manager, distributors, and users have an easier way to locate a problem when a change introduces a regression.

@tiran Done (initially I've thought this change is invisible to users so skipped it).

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@arhadthedev
Address Gregory's review in pythongh-94519
06f2a01
gpshead
gpshead requested changes Jul 27, 2022
View reviewed changes

@gpshead gpshead left a comment

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like this PR :)

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
Comment thread Modules/_posixsubprocess.c Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread Modules/_posixsubprocess.c Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread Modules/_posixsubprocess.c Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread Modules/_posixsubprocess.c Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread Modules/_posixsubprocess.c Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread Modules/_posixsubprocess.c Resolved

Uh oh!

There was an error while loading. Please reload this page.

@bedevere-bot bedevere-bot added awaiting changes and removed awaiting review labels Jul 27, 2022
@bedevere-bot

bedevere-bot commented Jul 27, 2022

Copy link
Copy Markdown

A Python core developer has requested some changes be made to your pull request before we can consider merging it. If you could please address their requests along with any other requests in other reviews from core developers that would be appreciated.

Once you have made the requested changes, please leave a comment on this pull request containing the phrase I have made the requested changes; please review again. I will then notify any core developers who have left a review that you're ready for them to take another look at this pull request.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@gpshead gpshead self-assigned this Jul 27, 2022
arhadthedev added 5 commits July 27, 2022 16:21
@arhadthedev
Use negative groups_size for "don't call setgroups"
aaec16e
When we query `groups_list` size using `PySequence_Size()`, we get -1
for non-lists. It returns an ability to pass an empty list to
`setgroups()` that is valid but treated as a no-op:

> If `size` is zero, `list` is not modified, but the total number of
> supplementary group IDs for the process is returned. This allows the
> caller to determine the size of a dynamically allocated list to be
> used in a further call to getgroups().
>
> - <https://linux.die.net/man/2/setgroups>
@arhadthedev
Check num_groups for emptiness, not groups_list
c9bf4f6
`num_groups < 0` already implies `groups_list == Py_None` because
`num_groups = PySequence_Size(groups_list)` does this check for us.

Also, it guarantees that `groups_list == Py_None` <=> `groups == NULL`;
it allows to replace `groups_list != Py_None ? groups : NULL` with just
`groups` in a call of `do_fork_exec()`.
@arhadthedev
Restore assertions in do_fork_exec()
069d769
@arhadthedev
Tighten scope of relevant short lived variables
6a236e5
@arhadthedev
Fix PermissionError: [Errno 1] Operation not permitted exception
8c06be1
@arhadthedev

arhadthedev commented Jul 27, 2022 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Member Author

if (groups_size >= 0)

setgroups(0, NULL) is valid.

For some reason, setgroups(0, NULL) causes Doctest, Check if generated files are up to date, and SSL workflows to fail child processes with PermissionError: [Errno 1] Operation not permitted exception:

Stacktrace
Run ./python Lib/test/ssltests.py
Traceback (most recent call last):
OpenSSL 1.1.1n  15 Mar 2022
  File "/home/runner/work/cpython/cpython/Lib/test/ssltests.py", line 37, in <module>
    run_regrtests(*sys.argv[1:])
  File "/home/runner/work/cpython/cpython/Lib/test/ssltests.py", line 33, in run_regrtests
    result = subprocess.call(args)
             ^^^^^^^^^^^^^^^^^^^^^
  File "/home/runner/work/cpython/cpython/Lib/subprocess.py", line 378, in call
    with Popen(*popenargs, **kwargs) as p:
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/runner/work/cpython/cpython/Lib/subprocess.py", line 1011, in __init__
    self._execute_child(args, executable, preexec_fn, close_fds,
  File "/home/runner/work/cpython/cpython/Lib/subprocess.py", line 1888, in _execute_child
    raise child_exception_type(errno_num, err_msg, err_filename)
PermissionError: [Errno 1] Operation not permitted
Error: Process completed with exit code 1.

However, this is fixed by excluding a zero from the comparison: if (groups_size > 0). I think we may do it this way because setgroups(0, NULL) is a no-op in child_exec (a positive returned value is dropped anyway):

If size is zero, list is not modified, but the total number of supplementary group IDs for the process is returned.

-- https://linux.die.net/man/2/setgroups

For Bedevere: I have made the requested changes; please review again. For correct jump from Mentioned to this message instead of the following Bedevere's one: @gpshead.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@bedevere-bot bedevere-bot added awaiting change review and removed awaiting changes labels Jul 27, 2022
@bedevere-bot

bedevere-bot commented Jul 27, 2022

Copy link
Copy Markdown

Thanks for making the requested changes!

@gpshead: please review the changes made to this pull request.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@bedevere-bot
bedevere-bot requested a review from gpshead July 27, 2022 21:17
@arhadthedev

arhadthedev commented Aug 18, 2022

Copy link
Copy Markdown
Member Author

@gpshead @tiran could you reconsider this PR please? This is a prerequirement for gh-94519, created to get rid of uninitialized variables.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@tiran

tiran commented Aug 18, 2022

Copy link
Copy Markdown
Member

We are currently busy with 3.11 release. It may take until 3.11.0 final before we come back to this patch. It's definitely on the list for 3.12.

arhadthedev reacted with thumbs up emoji
All reactions
  • 👍 1 reaction

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@arhadthedev arhadthedev mentioned this pull request Aug 18, 2022
Document CPython lifecycle periods of reduced core dev capacity python/devguide#937
Closed
@arhadthedev

arhadthedev commented Oct 6, 2022

Copy link
Copy Markdown
Member Author

Todo: replace groups_size/groups parameter names with extra_group_size/extra_groups across the whole file to avoid confusion of passers-by. It's necessary to mitigate a misleading name of setgroups that supposes replacement of not supplimentary group affinities, but all ones. The mislead results in seeing the omission of setgroups call for groups_size == 0 as a bug (like we should reset the inherited affinities but do nothing instead).

I'll do it in a separate PR because both this PR and gh-94519 are already big enough to clobber them with extra details.

gpshead reacted with thumbs up emoji
All reactions
  • 👍 1 reaction

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@arhadthedev
Merge branch 'main' into posixsubprocess-no-callsetxid
3ccdb78
@arhadthedev
Merge branch 'main' into posixsubprocess-no-callsetxid
d3c0e6c
@arhadthedev arhadthedev mentioned this pull request Nov 14, 2022
Port 23-argument _posixsubprocess.fork_exec to Argument Clinic #94518
Closed
arhadthedev added 3 commits November 14, 2022 12:52
@arhadthedev
Merge branch 'main' into posixsubprocess-no-callsetxid
cfdf639
@arhadthedev
Merge branch 'main' into posixsubprocess-no-callsetxid
0e997c0
@arhadthedev
Merge branch 'main' into posixsubprocess-no-callsetxid
66c836b
@arhadthedev

arhadthedev commented Jan 14, 2023

Copy link
Copy Markdown
Member Author

@tiran could you review and possibly merge this PR please? It's an implementation of your proposal #94519 (comment) created as a separate PR to preserve diffs of both PRs readable.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

gpshead
gpshead approved these changes Jan 14, 2023
View reviewed changes

@gpshead gpshead left a comment

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks for doing this!

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
@bedevere-bot bedevere-bot added awaiting merge and removed awaiting change review labels Jan 14, 2023
@gpshead
gpshead merged commit 124af17 into python:main Jan 14, 2023
@bedevere-bot bedevere-bot removed the awaiting merge label Jan 14, 2023
@arhadthedev

arhadthedev commented Jan 15, 2023

Copy link
Copy Markdown
Member Author

Thank you for merging, Gregory!

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@arhadthedev
arhadthedev deleted the posixsubprocess-no-callsetxid branch January 15, 2023 05:00
@arhadthedev arhadthedev mentioned this pull request Jan 15, 2023
gh-94518: Rename group* to extra_group* to avoid confusion #101054
Merged
@coderabbitai coderabbitai Bot mentioned this pull request May 25, 2026
Hostenv pyre-lang/pyre#100
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@gpshead gpshead gpshead approved these changes

Uh oh!

There was an error while loading. Please reload this page.

Assignees

@gpshead gpshead

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Uh oh!

There was an error while loading. Please reload this page.

4 participants

@arhadthedev @bedevere-bot @tiran @gpshead
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
You are visitor 01402503 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.