GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — GH-83417: Allow `venv` to add a `.gitignore` file to environments via a new `scm_ignore_file` parameter by brettcannon · Pull Request #108125 · python/cpython

Conversation

@brettcannon

@brettcannon brettcannon commented Aug 18, 2023 •
edited by github-actions Bot
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Member

Off by default via code but on by default via the CLI, the .gitignore file contains * which causes the entire directory to be ignored.

  • Issue: [venv] Adding a .gitignore file to virtual environments #83417

📚 Documentation preview 📚: https://cpython-previews--108125.org.readthedocs.build/

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

ofek, edgarrmondragon, adamchainz, Glyphack, edmorley, nikaro, and afeld reacted with hooray emoji Pierre-Sassoulas reacted with heart emoji
All reactions
  • 🎉 7 reactions
  • ❤️ 1 reaction
@brettcannon
pythonGH-83417: Allow venv add a .gitignore file to environments
f889dc6
Off by default via code but on by default via the CLI, the `.gitignore` file contains `*` which causes the entire directory to be ignored.
@brettcannon
brettcannon requested a review from vsajip as a code owner August 18, 2023 22:56
@bedevere-bot bedevere-bot mentioned this pull request Aug 18, 2023
[venv] Adding a .gitignore file to virtual environments #83417
Closed
@bedevere-bot bedevere-bot added the awaiting core review label Aug 18, 2023
@brettcannon
Add a news entry
1faf4b0
@brettcannon brettcannon linked an issue Aug 18, 2023 that may be closed by this pull request
[venv] Adding a .gitignore file to virtual environments #83417
Closed
vsajip
vsajip reviewed Aug 19, 2023
View reviewed changes
Comment thread Doc/library/venv.rst Resolved

Uh oh!

There was an error while loading. Please reload this page.

AA-Turner
AA-Turner reviewed Aug 20, 2023
View reviewed changes
Comment thread Doc/library/venv.rst Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread Doc/library/venv.rst Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread Doc/library/venv.rst Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread Lib/test/test_venv.py Resolved

Uh oh!

There was an error while loading. Please reload this page.

brettcannon and others added 2 commits August 21, 2023 15:11
@brettcannon @AA-Turner
Apply suggestions from code review
31d0558
Co-authored-by: Adam Turner <[email protected]>
@brettcannon
Merge branch 'main' into issue-83417-venv-gitignore
41e541c
@ambv

ambv commented Aug 24, 2023

Copy link
Copy Markdown
Contributor

Realistically no other version control systems are popular enough to warrant this feature. I like the discoverability of the current wording of the new arguments.

The barrier of entry for a future version control system to become popular enough to warrant addition here is pretty high. Therefore, I don't think we need to bikeshed this anymore. Let's just add it so we can test it in practice.

hugovk, ofek, AA-Turner, brettcannon, and erlend-aasland reacted with thumbs up emoji gaborbernat reacted with thumbs down emoji
All reactions
  • 👍 5 reactions
  • 👎 1 reaction

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

ambv
ambv approved these changes Aug 24, 2023
View reviewed changes
@bedevere-bot bedevere-bot added awaiting merge and removed awaiting core review labels Aug 24, 2023
@brettcannon

brettcannon commented Aug 25, 2023

Copy link
Copy Markdown
Member Author

Looking at the 👍 on the comment from @ambv , people so far seem to prefer keeping the gitignore argument. Anyone prefer the approach suggested by @vsajip ?

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

gaborbernat
gaborbernat suggested changes Sep 1, 2023
View reviewed changes
Comment thread Doc/library/venv.rst Resolved

Uh oh!

There was an error while loading. Please reload this page.

gaborbernat
gaborbernat reviewed Sep 1, 2023
View reviewed changes
Comment thread Lib/venv/__init__.py Resolved

Uh oh!

There was an error while loading. Please reload this page.

@brettcannon
Switch to scm_ignore_file
ce17684
@brettcannon

brettcannon commented Sep 4, 2023

Copy link
Copy Markdown
Member Author

I updated the PR using an scm_ignore_file parameter that takes as a string the name of the SCM to create an ignore file for. It dispatches to create_{scm_ignore_file}_ignore_file() for the file creation. I also made the CLI option be --without-scm-ignore-file. That should give @gaborbernat the flexibility he was after for virtualenv and anyone who customizing virtual environment creation via subclassing EnvBuilder.

I made the parameter take a single SCM name as no one realistically stores their source code in multiple version control systems. And since the ignore file prevents you from checking in the virtual environment, there's no need to commit a multitude of ignore files when developing a package.

I stuck with an opt-out flag since there's only git support right now and you can't make the CLI use a custom EnvBuilder class. If there's a desire to add a CLI option in the future we can add that in. But honestly, while I was writing the tests for a --with-scm-ignore-file, I realized the effort was not worth it for the CLI right now, especially when even pyvenv.cfg wouldn't record the option as it's the default semantics and defaults are left out. So YAGNI, avoiding pre-mature API optimization, and better things to do with my time on a long weekend made me drop --with-scm-ignore-file.

Hopefully this works for everyone. The API becomes flexible while the CLI stays simple which I think covers everyone's key concerns. If @ambv wants to drop his approval due to this change I would understand.

ofek reacted with heart emoji
All reactions
  • ❤️ 1 reaction

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@brettcannon brettcannon changed the title GH-83417: Allow venv add a .gitignore file to environments GH-83417: Allow venv to add a .gitignore file to environments via a new scm_ignore_file parameter Sep 4, 2023
@gaborbernat

gaborbernat commented Sep 4, 2023 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Contributor

I made the parameter take a single SCM name as no one realistically stores their source code in multiple version control systems

Well this might be true, but imagine the case when I work at the company that for all internal projects uses SCM x. That being said, company still operates in a world where most upstream projects will use git. Shouldn't in this case a Hypothetically internal plugin by default disable version control on both SCM types? Or do you imagine that the caller will manually every time explicitly set the type of SCM the project in the current folder uses? The alternative would be too allow having multiple SCM targets in parallel. The additional SCM files for a different type of SCM would be automatically ignored by the matching SCMs configuration file, so there's no real downside in generating multiple such configurations, one per each SCM. Now granted even with the current design one could set this SCM to x,git as a string marker to generate both SCM configuration files but feels a bit odd to me.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@hugovk

hugovk commented Sep 4, 2023 •
edited by AA-Turner
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Member

I made the parameter take a single SCM name as no one realistically stores their source code in multiple version control systems.

It's not common, but there are repos that are stored in one SCM and mirrored to another, sometimes mirroring to GitHub to use the CI.

It's hard to find one on request, but for example https://sourceforge.net/p/docutils/code/HEAD/tree/ is SVN and https://repo.or.cz/docutils.git is Git, although this only contains a .gitignore.

(https://github.com/python/devinabox and https://github.com/pypa/distlib contain both a .hgignore and .gitgnore. https://git.launchpad.net/beautifulsoup/tree/ has both .bzrignore and .gitignore. I'm unsure if these are relics or still both in use.)

But I don't know if it's worth the extra complexity for these rare cases.

AA-Turner reacted with thumbs up emoji
All reactions
  • 👍 1 reaction

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@ambv

ambv commented Sep 4, 2023

Copy link
Copy Markdown
Contributor

If @ambv wants to drop his approval due to this change I would understand.

I don't mind. I don't have any fundamental reason against supporting many SCMs, I just think it's unnecessary to spend so much brain power on it.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@ofek

ofek commented Sep 4, 2023

Copy link
Copy Markdown
Contributor

FWIW Hatchling supports multiple version control systems for default exclusion patterns: https://github.com/pypa/hatch/blob/hatchling-v1.18.0/backend/src/hatchling/builders/config.py#L798-L838

It's annoying to have in the code base but the risk of not supporting hypothetical users who would then open feature requests anyway is worse than the maintenance cost.

edgarrmondragon reacted with thumbs up emoji
All reactions
  • 👍 1 reaction

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@brettcannon

brettcannon commented Sep 6, 2023

Copy link
Copy Markdown
Member Author

Well this might be true, but imagine the case when I work at the company that for all internal projects uses SCM x. That being said, company still operates in a world where most upstream projects will use git. Shouldn't in this case a Hypothetically internal plugin by default disable version control on both SCM types?

It's not common, but there are repos that are stored in one SCM and mirrored to another, sometimes mirroring to GitHub to use the CI.

But the file isn't getting committed to the repository just as the virtual environment isn't. So you only have to care about the version control system where you're creating the virtual environment which won't be multiple SCMs but just the one being used locally.

Or do you imagine that the caller will manually every time explicitly set the type of SCM the project in the current folder uses?

The caller can determine what it wants to specify however it chooses. Also note we are talking about companies that are big enough to be running two SCMs; they have the staff to support this. 😉

The question is whether we should optimize the API for all possibilities because we only expect people with non-standard SCMs to set anything, or do we optimize for the the 99% of the world who only operate under a single SCM?

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@brettcannon
Merge branch 'main' into issue-83417-venv-gitignore
90eb700
@brettcannon

brettcannon commented Sep 6, 2023

Copy link
Copy Markdown
Member Author

Also, if we are talking about multiple SCMs, then that means you're already subclassing which means you can override __init__() and create() however you want.

If it bothers people that much we could add a create_ignore_file() method which handles the dispatch to the appropriate create_*_ignore_file() method(s) and that lets you do whatever you want.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@gaborbernat

gaborbernat commented Sep 6, 2023

Copy link
Copy Markdown
Contributor

I don't think supporting 1+ SCMs is any more complicated than exactly 1 in this case, but the current ApI is a bit less user-friendly but I digress, I made my point.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

gaborbernat
gaborbernat reviewed Sep 6, 2023
View reviewed changes
Comment thread Lib/test/test_venv.py Resolved

Uh oh!

There was an error while loading. Please reload this page.

@brettcannon
Make scm_ignore_files accept an iterable
c3b96e0
vsajip
vsajip approved these changes Sep 14, 2023
View reviewed changes
@bedevere-app bedevere-app Bot added awaiting core review and removed awaiting merge labels Sep 14, 2023
@brettcannon
Add a What's new entry
f75b26c
@brettcannon
brettcannon enabled auto-merge (squash) September 15, 2023 22:14
@brettcannon
brettcannon disabled auto-merge September 15, 2023 22:14
@brettcannon
brettcannon enabled auto-merge (squash) September 15, 2023 22:14
@brettcannon
brettcannon merged commit e218e50 into python:main Sep 15, 2023
@brettcannon
brettcannon deleted the issue-83417-venv-gitignore branch September 15, 2023 22:38
@bedevere-app bedevere-app Bot removed the awaiting core review label Sep 15, 2023
@brettcannon

brettcannon commented Sep 15, 2023

Copy link
Copy Markdown
Member Author

Thanks everyone!

ofek reacted with heart emoji
All reactions
  • ❤️ 1 reaction

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@bedevere-bot

bedevere-bot commented Sep 15, 2023

Copy link
Copy Markdown

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot s390x RHEL7 LTO + PGO 3.x has failed when building commit e218e50.

What do you need to do:

  1. Don't panic.
  2. Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
  3. Go to the page of the buildbot that failed (https://buildbot.python.org/all/#builders/244/builds/5454) and take a look at the build logs.
  4. Check if the failure is related to this commit (e218e50) or if it is a false positive.
  5. If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/all/#builders/244/builds/5454

Failed tests:

  • test.test_asyncio.test_subprocess

Failed subtests:

  • test_subprocess_consistent_callbacks - test.test_asyncio.test_subprocess.SubprocessThreadedWatcherTests.test_subprocess_consistent_callbacks

Summary of the results of the build (if available):

==

Click to see traceback logs
Traceback (most recent call last):
  File "/home/dje/cpython-buildarea/3.x.edelsohn-rhel-z.lto-pgo/build/Lib/test/test_asyncio/test_subprocess.py", line 788, in test_subprocess_consistent_callbacks
    self.loop.run_until_complete(main())
  File "/home/dje/cpython-buildarea/3.x.edelsohn-rhel-z.lto-pgo/build/Lib/asyncio/base_events.py", line 664, in run_until_complete
    return future.result()
           ^^^^^^^^^^^^^^^
  File "/home/dje/cpython-buildarea/3.x.edelsohn-rhel-z.lto-pgo/build/Lib/test/test_asyncio/test_subprocess.py", line 780, in main
    self.assertEqual(events, [
AssertionError: Lists differ: [('pi[29 chars]t'), 'pipe_connection_lost', ('pipe_data_recei[57 chars]ted'] != [('pi[29 chars]t'), ('pipe_data_received', 2, b'stderr'), 'pi[57 chars]ted']
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@brettcannon brettcannon mentioned this pull request Sep 15, 2023
Support scm_ignore_files brettcannon/microvenv#43
Closed
@bedevere-bot

bedevere-bot commented Sep 16, 2023

Copy link
Copy Markdown

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot PPC64LE RHEL7 3.x has failed when building commit e218e50.

What do you need to do:

  1. Don't panic.
  2. Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
  3. Go to the page of the buildbot that failed (https://buildbot.python.org/all/#builders/446/builds/3899) and take a look at the build logs.
  4. Check if the failure is related to this commit (e218e50) or if it is a false positive.
  5. If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/all/#builders/446/builds/3899

Failed tests:

  • test.test_multiprocessing_fork.test_misc

Failed subtests:

  • test_nested_startmethod - test.test_multiprocessing_fork.test_misc.TestStartMethod.test_nested_startmethod

Summary of the results of the build (if available):

==

Click to see traceback logs
Traceback (most recent call last):
  File "/home/buildbot/buildarea/3.x.cstratak-RHEL7-ppc64le/build/Lib/test/_test_multiprocessing.py", line 5475, in test_nested_startmethod
    self.assertEqual(results, [2, 1])
AssertionError: Lists differ: [1, 2] != [2, 1]
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@bedevere-bot

bedevere-bot commented Sep 16, 2023

Copy link
Copy Markdown

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot PPC64LE RHEL7 LTO + PGO 3.x has failed when building commit e218e50.

What do you need to do:

  1. Don't panic.
  2. Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
  3. Go to the page of the buildbot that failed (https://buildbot.python.org/all/#builders/43/builds/4239) and take a look at the build logs.
  4. Check if the failure is related to this commit (e218e50) or if it is a false positive.
  5. If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/all/#builders/43/builds/4239

Failed tests:

  • test.test_asyncio.test_events

Failed subtests:

  • test_create_connection_local_addr_nomatch_family - test.test_asyncio.test_events.PollEventLoopTests.test_create_connection_local_addr_nomatch_family

Summary of the results of the build (if available):

==

Click to see traceback logs
Traceback (most recent call last):
  File "/home/buildbot/buildarea/3.x.cstratak-RHEL7-ppc64le.lto-pgo/build/Lib/test/test_asyncio/test_events.py", line 715, in test_create_connection_local_addr_nomatch_family
    with self.assertRaises(OSError):
AssertionError: OSError not raised
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@AA-Turner AA-Turner mentioned this pull request Sep 17, 2023
GH-80789: Get rid of the ensurepip infra for many wheels #109245
Merged
This was referenced Sep 18, 2023
Add .gitignore file to .pixi/env/myenv prefix-dev/pixi#343
Closed
Idea: Add .gitignore file to envs mamba-org/mamba#2846
Open
@konstin konstin mentioned this pull request Sep 21, 2023
Fix gitignore to not ignore files that are required astral-sh/ruff#7538
Merged
csm10495 pushed a commit to csm10495/cpython that referenced this pull request Sep 28, 2023
@brettcannon @AA-Turner
@hugovk @csm10495
pythonGH-83417: Allow venv to add a .gitignore file to environmen…
a4284f9
…ts via a new `scm_ignore_file` parameter (pythonGH-108125)

This feature is off by default via code but on by default via the CLI. The `.gitignore` file contains `*` which causes the entire directory to be ignored.

Co-authored-by: Adam Turner <[email protected]>
Co-authored-by: Hugo van Kemenade <[email protected]>
@jthomasmock jthomasmock mentioned this pull request Sep 28, 2023
Provide option to automatically add specific directories/files to .gitignore posit-dev/positron#1372
Open
@bersbersbers bersbersbers mentioned this pull request Nov 8, 2023
Create .gitignore file in scoop directory ScoopInstaller/Install#72
Open
@hroncok hroncok mentioned this pull request Nov 28, 2023
tests/unit/create/test_creator.py::test_create_no_seed --creator venv fails on Python 3.13 pypa/virtualenv#2670
Closed
hroncok added a commit to hroncok/virtualenv that referenced this pull request Nov 29, 2023
@hroncok
Fix test expectations for .gitignore comment created by Python 3.13+ …
e3bd8e8
…venv

Since Python 3.13, the .gitignore file is created by venv:
python/cpython#108125

Fixes pypa#2670
@hroncok hroncok mentioned this pull request Nov 29, 2023
Fix test expectations for .gitignore comment created by Python 3.13+ venv pypa/virtualenv#2672
Closed
3 of 5 tasks
hroncok added a commit to hroncok/virtualenv that referenced this pull request Nov 29, 2023
@hroncok
Fix test expectations for .gitignore comment created by Python 3.13+ …
3e9d7ac
…venv

Since Python 3.13, the .gitignore file is created by venv:
python/cpython#108125

Fixes pypa#2670
@AlexWaygood AlexWaygood mentioned this pull request Apr 8, 2024
gh-117225: Add color to doctest output #117583
Merged
@Privat33r-dev Privat33r-dev mentioned this pull request Apr 8, 2024
Clarify usage of type hints in the standard-library modules python/devguide#1304
Open
@hugovk hugovk mentioned this pull request Sep 16, 2024
Docs: Update and proofread library/venv.rst #124121
Merged
@fiveop fiveop mentioned this pull request Jan 6, 2025
Adapting Checking-in Changes to Our Project to new versions of venv carpentries-incubator/python-intermediate-development#417
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@AA-Turner AA-Turner AA-Turner left review comments
@AlexWaygood AlexWaygood AlexWaygood left review comments
@ambv ambv ambv approved these changes
@vsajip vsajip vsajip approved these changes
@hugovk hugovk hugovk approved these changes
+2 more reviewers
@edgarrmondragon edgarrmondragon edgarrmondragon left review comments
@gaborbernat gaborbernat gaborbernat approved these changes
Reviewers whose approvals may not affect merge requirements

Uh oh!

There was an error while loading. Please reload this page.

Assignees

@brettcannon brettcannon

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

[venv] Adding a .gitignore file to virtual environments

Uh oh!

There was an error while loading. Please reload this page.

10 participants

@brettcannon @ambv @gaborbernat @hugovk @ofek @bedevere-bot @vsajip @AA-Turner @edgarrmondragon @AlexWaygood
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
You are visitor 01402503 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.