Description

kulikjak

opened on Dec 14, 2023

Issue body actions

Bug report

Bug description:

Subprocess created with posix_spawn can have different environment when compared to a process created with stardard fork/exec.

The reason is that posix_spawn uses os.environ (when no env was given), which is not affected by os.putenv and os.unsetenv. This is different from execv, which uses process environment (available through environ).

This can be easily reproduced with test_putenv_unsetenv modified to call subprocess.run with close_fds=False (which makes it use the posix_spawn call):

import subprocess
import sys
import os

name = "PYTHONTESTVAR"
value = "testvalue"
code = f'import os; print(repr(os.environ.get({name!r})))'

os.putenv(name, value)
proc = subprocess.run([sys.executable, '-c', code], check=True,
                      stdout=subprocess.PIPE, text=True, close_fds=False)
assert proc.stdout.rstrip() == repr(value)

os.unsetenv(name)
proc = subprocess.run([sys.executable, '-c', code], check=True,
                      stdout=subprocess.PIPE, text=True, close_fds=False)
assert proc.stdout.rstrip() == repr(None)

I found it when I patched Python with #113118 (on Solaris, but this is pretty platform independent, I think).

CPython versions tested on:

3.9, 3.11

Operating systems tested on:

Other

Linked PRs

• gh-113119 fix environment handling in subprocess.Popen when posix_spawn is used #113120
• gh-113119: Fix the macOS framework installer build #113268

Reacted by Gregory P. Smith, qqwqqw689 and Maximilian Knespel

Reacted by Jakub Kulík and qqwqqw689

Activity

1. kulikjak

   added

   type-bugAn unexpected behavior, bug, or errorAn unexpected behavior, bug, or error

   on Dec 14, 2023
2. gpshead

   self-assigned this

   on Dec 15, 2023
3. gpshead

   closed this as completedin #113120on Dec 17, 2023
4. gpshead

   added a commit that references this issue on Dec 17, 2023

   gh-113119 fix environment handling in subprocess.Popen when posix_spa…

   48c907a
5. gpshead

   reopened this on Dec 19, 2023
6. gpshead commented on Dec 19, 2023

   

   gpshead

   on Dec 19, 2023

   Member

   More actions

   reopened due to @ned-deily reporting on a parallel issue #113117 about this change:

   This breaks macOS framework builds such as those used by the python.org installer:

   $ ./configure --enable-framework && make -j
   [...]
   ./Modules/posixmodule.c:7170:19: error: use of undeclared identifier 'environ'
           envlist = environ;
                     ^
   ./Modules/posixmodule.c:7238:31: error: use of undeclared identifier 'environ'
       if (envlist && envlist != environ) {
                                 ^
   

   Note the comments elsewhere in posixmodule.c such as at line 1582:

   #elif defined(WITH_NEXT_FRAMEWORK) || (defined(__APPLE__) && defined(Py_ENABLE_SHARED))
       /* environ is not accessible as an extern in a shared object on OSX; use
          _NSGetEnviron to resolve it. The value changes if you add environment
          variables between calls to Py_Initialize, so don't cache the value. */
       e = *_NSGetEnviron();
   
   

   Reacted by Jakub Kulík
7. gpshead

   added a commit that references this issue on Dec 19, 2023

   gh-113119: Fix the macOS framework installer build (#113268)

   c895403
8. gpshead

   closed this as completedon Dec 19, 2023
9. kulikjak commented on Dec 19, 2023

   

   kulikjak

   on Dec 19, 2023

   ContributorAuthor

   More actions

   Sorry for that, and thank you for the fix. Since CI was green, I didn't look into it much more and missed that.

   Reacted by Gregory P. Smith
10. ryan-duve

    added a commit that references this issue on Dec 26, 2023

    pythongh-113119: Fix the macOS framework installer build (python#113268)

    ef388a4
11. aisk

    added 2 commits that reference this issue on Feb 11, 2024

    pythongh-113119 fix environment handling in subprocess.Popen when pos…

    bd6de62

    pythongh-113119: Fix the macOS framework installer build (python#113268)

    7fa9a0a
12. Glyphack

    added 2 commits that reference this issue on Sep 2, 2024

    pythongh-113119 fix environment handling in subprocess.Popen when pos…

    3ea4a9f

    pythongh-113119: Fix the macOS framework installer build (python#113268)

    7a9581b

Issue actions