GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — Output `python-versions`, containing all Python versions, for cache-busting by kurtmckee · Pull Request #611 · actions/setup-python

Conversation

@kurtmckee

@kurtmckee kurtmckee commented Feb 13, 2023 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Contributor

Description:
This introduces a python-versions output, which is a comma-separated string of all installed Python versions and implementations. This is useful for cache-busting when multiple Python versions are installed, as described in #606.

For example, it is now possible to cache a .tox/ test directory and invalidate the cache when any of the Python versions change. Caching the .tox/ directory offers a significant speedup to test setup times, but if a non-default Python's version changes (say, if 3.11.1 is the default but 3.10.x increments to 3.10.y), it is currently very difficult to invalidate the cache. This new output variable allows for trivial cache invalidation in this situation.

To demonstrate the new output variable, I created a workflow with a single setup-python step that installs multiple CPython and PyPy versions, then used echo to show the value of python-versions.

The value displayed is:

3.10.9,3.11.1,3.12.0-alpha.5,3.7.15,3.8.16,3.8.16-pypy7.3.11,3.9.16,3.9.16-pypy7.3.11

which can be used directly in a cache key, or written to a file and included in a call to hashFiles(). The versions are sorted to avoid sensitivity to the order of Python versions specific in the workflow, but are sorted merely as strings for this purpose, not as semantic versions. Therefore a user can create a workflow that installs Python versions in any order they want, but the python-versions output string will be stable.

NOTE: This work depends on #610 to remove trailing newlines in PyPy versions.

Related issue:
Closes #606

Check list:

  • Mark if documentation changes are required.
  • Mark if tests were added or updated to cover the changes.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
@kurtmckee
kurtmckee requested a review from a team as a code owner February 13, 2023 15:48
dmitry-shibanov
dmitry-shibanov reviewed Mar 13, 2023
View reviewed changes
Comment thread src/setup-python.ts
Comment on lines +98 to +114
pythonVersions.push(
`${installed.resolvedPythonVersion}-pypy${installed.resolvedPyPyVersion}`
);

@dmitry-shibanov dmitry-shibanov Mar 13, 2023

Copy link
Copy Markdown
Contributor

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we need to change pypy and python version parts order ?

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

@kurtmckee kurtmckee Mar 13, 2023 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Contributor Author

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm fine with either order.

I did it this way for potential parsing purposes later in the workflow. It seemed like if someone wanted to determine all of the Python versions (regardless of CPython vs PyPy implementation), it would be easier if the Python version appeared at the beginning of the string. If -pypy appeared later in the string it could be stripped off or extracted out as needed.

However, I don't view the ordering as a critical need, and can change the order if desired.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
Comment thread src/setup-python.ts
core.info(`Successfully set up ${installed.impl} (${pythonVersion})`);
}
}
core.setOutput('python-versions', pythonVersions.sort().join(','));

@dmitry-shibanov dmitry-shibanov Mar 13, 2023

Copy link
Copy Markdown
Contributor

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we need to sort versions ?

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

@kurtmckee kurtmckee Mar 13, 2023 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Contributor Author

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I sorted it to help stabilize the cache key. If one developer writes their workflow YAML file as:

- uses: actions/setup-python@v4
  with:
    python-version: |
      3.11
      3.10
      3.9
      3.8
      3.7

and the versions are later rearranged to:

- uses: actions/setup-python@v4
  with:
    python-version: |
      3.7
      3.8
      3.9
      3.10
      3.11

sorting the versions will provide stability to the cache key.

I currently think this is beneficial behavior, but can remove the sorting if desired.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
@kurtmckee

kurtmckee commented May 12, 2023

Copy link
Copy Markdown
Contributor Author

@dmitry-shibanov I just saw the new support for "allow-prereleases", which is fantastic! I would love to get additional feedback about this PR, or to have it merged, so that I can use this new output as a cache key.

I don't know if I communicated the use case well, so to summarize:

Setting up a GitHub runner for each Python version is comparatively expensive for some of my projects -- it can take 30 seconds to spin up a runner, 5 seconds to set up the tox environment, but the test suite only takes 2 seconds to run for each Python version.

Therefore I install all Python versions on a single runner:

  • 30 seconds to spin up the runner
  • 5 seconds to restore the entire .tox directory -- for all Python versions -- from cache
  • 2 seconds to run each Python version

The cache needs to be invalidated if any Python version changes, and this PR makes that possible. Python 3.12 releases are coming out with frequency, and I would like to take advantage of the new "allow-prereleases" feature and have this python-versions output available for cache busting.

Thanks for your time, and your work on setup-python!

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@kurtmckee kurtmckee changed the title Output python-versions, containing all Python versions Output python-versions, containing all Python versions, for cache-busting Sep 11, 2023
@kurtmckee

kurtmckee commented Sep 11, 2023

Copy link
Copy Markdown
Contributor Author

@dmitry-shibanov, please let me know if you have additional feedback. I'm interested in having this PR merge for effective cache-busting when multiple Python versions are installed.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@kurtmckee
kurtmckee force-pushed the output-python-versions-issue-606 branch 2 times, most recently from 197e69d to 949d439 Compare October 6, 2023 11:59
@kurtmckee

kurtmckee commented Oct 6, 2023

Copy link
Copy Markdown
Contributor Author

Rebased on main and resolved CI failures.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@kurtmckee kurtmckee mentioned this pull request Oct 6, 2023
Add GraalPy support #694
Merged
1 of 2 tasks
@kurtmckee
Output python-versions for cache-busting
2d235cd
Closes actions#606
@kurtmckee
kurtmckee force-pushed the output-python-versions-issue-606 branch from 949d439 to 2d235cd Compare October 10, 2023 14:29
@kurtmckee

kurtmckee commented Oct 27, 2023

Copy link
Copy Markdown
Contributor Author

I'm closing this PR because I've written a GitHub action to address my need for strong cache-busting based on installed Python versions.

https://github.com/kurtmckee/detect-pythons

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@kurtmckee kurtmckee closed this Oct 27, 2023
@kurtmckee kurtmckee mentioned this pull request May 13, 2024
Identify all installed Python versions #606
Closed
@kurtmckee
kurtmckee deleted the output-python-versions-issue-606 branch May 13, 2024 15:25
@kurtmckee kurtmckee mentioned this pull request Jun 9, 2025
Python-version output for PyPy isn't unique across different versions #1109
Closed
4 of 5 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

1 more reviewer
@dmitry-shibanov dmitry-shibanov dmitry-shibanov left review comments
Reviewers whose approvals may not affect merge requirements

Uh oh!

There was an error while loading. Please reload this page.

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Identify all installed Python versions

Uh oh!

There was an error while loading. Please reload this page.

2 participants

@kurtmckee @dmitry-shibanov
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
You are visitor 01402503 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.