GeezerVison is showing github.com — Build & release to PyPI via trusted publishing by hugovk · Pull Request #148 · python/python-docs-theme
Conversation
hugovk
commented
Aug 20, 2023
•
edited
Loading
Uh oh!
There was an error while loading. Please reload this page.
We discussed in the last monthly docs meeting to automate PyPI deploys using the new "Trusted Publishing", and the consensus was it's a good idea:
This would make releasing easier, and also open the release process to other core devs with the commit bit.
Here's a PR to set up the workflow, we'll also need to set up trusted publishing on TestPyPI and PyPI.
This workflow does three things:
It is based on @hynek's neat https://github.com/python-attrs/attrs/blob/main/.github/workflows/pypi-package.yml, which I'm successfully using in my own projects.
1.
For all runs, it builds and lints the PyPI sdist and wheel. This makes sure the non-publishing steps are working smoothly, ahead of uploading. It uses @hynek's excellent https://github.com/hynek/build-and-inspect-python-package action, which creates a nice summary, for example:
We can see it creates three handy zips if we want to check what would be uploaded to PyPI, and three handy toggles that show the contents. Try it here:
https://github.com/python/python-docs-theme/actions/runs/5919197594?pr=148
2.
For commits to main we publish to TestPyPI. This helps us know the publishing process is nicely oiled to avoid surprises on release day. It uses the packages that were already created in step 1.
3.
For published GitHub releases we publish to the real PyPI. Also uses packages created in step 1.
TODO
screenshot
@willingc I see you're the only maintainer for https://test.pypi.org/project/python-docs-theme/, please could you set up the above for TestPyPI? It's probably a good opportunity to add some extra owners/maintainers such as @Mariatta, to improve the bus factor.
@willingc You're also one of the maintainers for https://pypi.org/project/python-docs-theme/, so perhaps you could do PyPI too?
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I've reviewed this back-to-front -- sorry. I'm a tad uncertain if we should be going with this though. Overarching thoughts:
- I think that the 'check the package is alright' step could happen via a linting run, e.g. just running
twine check, which avoids the kerfuffle with uploading artefacts & to Test PyPI
- On Test PyPI, I'm unconvinced that we should use it for every commit -- we know that the package works, and there are limited resources on Test PyPI, even if the distribution is small, so I think we ought be conservative in what we upload.
- I'm unsure what's behind
hynek/build-and-inspect-python-package -- whilst I could go and look it up, it adds a layer of indirection when (to my understanding) this workflow should be replicating the release steps one would run locally -- e.g. python -m build; twine upload. Making it explicit what's going on I think is useful to observability and just having fewer moving parts in general (I'd have the same objection to pypa/gh-action-pypi-publish, save that it implements the OIDC flow for us).
Sorry again for several comments, but I hope this outlines my position to some extent.
A
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
- I think that the 'check the package is alright' step could happen via a linting run, e.g. just running
twine check, which avoids the kerfuffle with uploading artefacts & to Test PyPI
There is no linter that does all the checks that PyPI does.
twine check only does a subset of the checks. I've seen packages pass twine check yet fail to upload.
It's no kerfuffle, this PR adds automation so machines take care of it for us. 🤖
|
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
hugovk
commented
Aug 21, 2023
•
edited
Loading
Uh oh!
There was an error while loading. Please reload this page.
- On Test PyPI, I'm unconvinced that we should use it for every commit -- we know that the package works, and there are limited resources on Test PyPI, even if the distribution is small, so I think we ought be conservative in what we upload.
We only know for sure the package works and can be uploaded by uploading it.
These pure Python packages are very small, especially when compared to those with C extensions which may have 60+ multi-MB wheels.
Resources are indeed limited on TestPyPI, and I doubt we'll ever reach the project limit. If we do, we can safely delete old uploads.
Disregard, removed: #148 (comment)
|
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
- I'm unsure what's behind
hynek/build-and-inspect-python-package -- whilst I could go and look it up, it adds a layer of indirection when (to my understanding) this workflow should be replicating the release steps one would run locally -- e.g. python -m build; twine upload. Making it explicit what's going on I think is useful to observability and just having fewer moving parts in general (I'd have the same objection to pypa/gh-action-pypi-publish, save that it implements the OIDC flow for us).
I find the build and content summaries can be useful. It does lots of extra lint checks beyond twine check. It makes packages avaliable before upload, so we can test them locally if we like.
Best of all, we don't need to maintain it! We can benefit from @hynek and other contributors maintaining and improving it. We can also re-use it in other repos without duplicating all the extra checks across our repos.
(Here's the source: https://github.com/hynek/build-and-inspect-python-package)
|
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
|
I removed the TestPyPI upload because we hardcode the version and that would have meant the second and subsequent uploads would fail because you cannot reuse filenames on PyPI.
I don't want to expand the scope to propose using something like hatch-vcs or setuptools_scm in this PR, it's more important to get trusted publishing in place. 👍
|
AA-Turner reacted with thumbs up emoji
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
AA-Turner
commented
Aug 26, 2023
•
edited
Loading
Uh oh!
There was an error while loading. Please reload this page.
|
Thanks for your patience with my questions--excited to see trusted publishing enabled for python-docs-theme!
A
|
hugovk reacted with hooray emoji
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
|
@hugovk I'm just re-engaging with core development a bit more now that I'm semi-retired. Let me know what you need me to do re: pypi. Thanks.
|
hynek, hugovk, and pradyunsg reacted with hooray emoji
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
pradyunsg reacted with heart emoji
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
|
@hugovk Thanks for the great instructions. It's now completed. ✨
|
AA-Turner and pradyunsg reacted with hooray emoji
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
|
Thank you!
Let's merge this now, and I can try out a release with #153 next week, after 3.12 is out.
|
willingc reacted with thumbs up emoji
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
|
Success!
I merged #153, updated version and changelog in #157 and followed these steps (which I'll update in CONTRIBUTING.md):
The tag/release:
The CI build and deploy:
Screenshot

Screenshot

The release on PyPI:
|
hynek and ezio-melotti reacted with rocket emoji
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
(which I'll update in CONTRIBUTING.md):
Please see PR #158.
|
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
hugovk reacted with thumbs up emoji
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Assignees
No one assigned
Milestone
No milestone
Development
Successfully merging this pull request may close these issues.
Uh oh!
There was an error while loading. Please reload this page.
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
|