GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — bpo-39406: Implement os.putenv() with setenv() if available by vstinner · Pull Request #18095 · python/cpython

Conversation

@vstinner

@vstinner vstinner commented Jan 21, 2020 •
edited by bedevere-bot
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Member

If setenv() C function is available, os.putenv() is now implemented
with setenv() instead of putenv(), so Python doesn't have to handle
the environment variable memory.

If setenv() is available, posix_putenv_garbage state is not needed.

https://bugs.python.org/issue39406

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
All reactions
@vstinner
vstinner requested review from pablogsal, pitrou and serhiy-storchaka January 21, 2020 08:56
@the-knights-who-say-ni the-knights-who-say-ni added the CLA signed label Jan 21, 2020
@bedevere-bot bedevere-bot added the awaiting core review label Jan 21, 2020
serhiy-storchaka
serhiy-storchaka approved these changes Jan 21, 2020
View reviewed changes
Comment thread configure Resolved

Uh oh!

There was an error while loading. Please reload this page.
Comment thread Modules/posixmodule.c Resolved

Uh oh!

There was an error while loading. Please reload this page.
Comment thread Modules/posixmodule.c Resolved

Uh oh!

There was an error while loading. Please reload this page.
Comment thread Modules/posixmodule.c Resolved

Uh oh!

There was an error while loading. Please reload this page.
@bedevere-bot bedevere-bot added awaiting merge and removed awaiting core review labels Jan 21, 2020
@vstinner
bpo-39406: Implement os.putenv() with setenv() if available
472b8cf 
If setenv() C function is available, os.putenv() is now implemented
with setenv() instead of putenv(), so Python doesn't have to handle
the environment variable memory.

Renane posix_putenv_garbage to posix_putenv_dict in posixmodule.c.
On Windows or if setenv() is available, posix_putenv_dict is no
longer needed.
@vstinner

vstinner commented Jan 21, 2020

Copy link
Copy Markdown
Member Author

I updated the PR to use SetEnvironmentVariable() on Windows.

@serhiy-storchaka: Would you mind to review the updated PR?
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
@vstinner
Fix usage of PyErr_SetFromWindowsErr()
617f7f4
@vstinner

vstinner commented Jan 21, 2020

Copy link
Copy Markdown
Member Author

On Windows, I removed the explicit check on _MAX_ENV:

    if (size > _MAX_ENV) {
        PyErr_Format(PyExc_ValueError,
                     "the environment variable is longer than %u characters",
                     _MAX_ENV);
        goto error;
    }

On my Windows 10 VM, _MAX_ENV is equal to 32767, but SetEnvironmentVariableW() seems to accept way longer values!

>>> value='x' * (32767); os.environ['long'] = value; os.environ['long'] == value, len(value)    
(True, 32767)
>>> value='x' * (32767+10); os.environ['long'] = value; os.environ['long'] == value, len(value) 
(True, 32777)
>>> value='abcd' * (32767); os.environ['long'] = value; os.environ['long'] == value, len(value) 
(True, 131068)

But I got ERROR_NO_SYSTEM_RESOURCES (error 1450: "Insufficient system resources exist to complete the requested service") for a value of 1 GB:

>>> import os
>>> value='x'*(2**30)
>>> os.putenv('long', value)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
OSError: [WinError 1450] Ressources système insuffisantes pour terminer le service demandé

The limit seems to be the memory rather than an hardcoded limit. A "shorter" value (64 MB) is fine:

>>> import os
>>> value='x'*(2**26)
>>> os.putenv('long', value)
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
serhiy-storchaka
serhiy-storchaka reviewed Jan 21, 2020
View reviewed changes
Comment thread Modules/posixmodule.c
/* putenv() requires to manage the memory manually: use a Python dict
object mapping name (bytes) to env (bytes) where env is a bytes string
like "name=value\0". */
PyObject *posix_putenv_dict;

@serhiy-storchaka serhiy-storchaka Jan 21, 2020

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why rename it? This increases the size of the diff.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
All reactions
Comment thread Modules/posixmodule.c
return NULL;
}
#else /* MS_WINDOWS */
#elif defined(HAVE_PUTENV)

@serhiy-storchaka serhiy-storchaka Jan 21, 2020

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
#elif defined(HAVE_PUTENV)
#elif defined(HAVE_SETENV) || defined(HAVE_PUTENV)

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
All reactions
Comment thread Modules/posixmodule.c
}
#endif /* MS_WINDOWS */
#endif /* HAVE_PUTENV */
#endif /* HAVE_PUTENV */

@serhiy-storchaka serhiy-storchaka Jan 21, 2020

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
#endif /* HAVE_PUTENV */
#endif /* HAVE_SETENV || HAVE_PUTENV */

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
All reactions
@vstinner

vstinner commented Jan 21, 2020

Copy link
Copy Markdown
Member Author

Oh, a test fail on Windows:


2020-01-21T10:50:49.1283745Z ======================================================================
2020-01-21T10:50:49.1283887Z ERROR: test_unset_error (test.test_os.EnvironTests)
2020-01-21T10:50:49.1284029Z ----------------------------------------------------------------------
2020-01-21T10:50:49.1284166Z Traceback (most recent call last):
2020-01-21T10:50:49.1284296Z   File "D:\a\cpython\cpython\lib\test\support\__init__.py", line 695, in wrapper
2020-01-21T10:50:49.1284429Z     return func(*args, **kw)
2020-01-21T10:50:49.1284504Z   File "D:\a\cpython\cpython\lib\test\test_os.py", line 962, in test_unset_error
2020-01-21T10:50:49.1284628Z     self.assertRaises(ValueError, os.environ.__delitem__, key)
2020-01-21T10:50:49.1284764Z   File "D:\a\cpython\cpython\lib\unittest\case.py", line 799, in assertRaises
2020-01-21T10:50:49.1284895Z     return context.handle('assertRaises', args, kwargs)
2020-01-21T10:50:49.1285989Z   File "D:\a\cpython\cpython\lib\unittest\case.py", line 202, in handle
2020-01-21T10:50:49.1286157Z     callable_obj(*args, **kwargs)
2020-01-21T10:50:49.1286268Z   File "D:\a\cpython\cpython\lib\os.py", line 691, in __delitem__
2020-01-21T10:50:49.1286377Z     raise KeyError(key) from None
2020-01-21T10:50:49.1295463Z KeyError:
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
@vstinner

vstinner commented Jan 22, 2020

Copy link
Copy Markdown
Member Author

I made other changes in the meanwhile which introduced conflicts. So I wrote a new PR instead: PR #18128.
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
@vstinner vstinner mentioned this pull request Jan 22, 2020
bpo-39406: Implement os.putenv() with setenv() if available #18128
Merged
@vstinner vstinner closed this Jan 22, 2020
@vstinner
vstinner deleted the setenv branch January 22, 2020 21:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@serhiy-storchaka serhiy-storchaka serhiy-storchaka approved these changes
@pitrou pitrou Awaiting requested review from pitrou
@pablogsal pablogsal Awaiting requested review from pablogsal

Uh oh!

There was an error while loading. Please reload this page.

Assignees

No one assigned

Labels

awaiting merge

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Uh oh!

There was an error while loading. Please reload this page.

4 participants

@vstinner @serhiy-storchaka @the-knights-who-say-ni @bedevere-bot
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
You are visitor 01402503 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.