GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — Build & release to PyPI via trusted publishing by hugovk · Pull Request #148 · python/python-docs-theme

Conversation

@hugovk

@hugovk hugovk commented Aug 20, 2023 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Member

We discussed in the last monthly docs meeting to automate PyPI deploys using the new "Trusted Publishing", and the consensus was it's a good idea:

  • https://blog.pypi.org/posts/2023-04-20-introducing-trusted-publishers/

This would make releasing easier, and also open the release process to other core devs with the commit bit.

Here's a PR to set up the workflow, we'll also need to set up trusted publishing on TestPyPI and PyPI.


This workflow does three things:

image

It is based on @hynek's neat https://github.com/python-attrs/attrs/blob/main/.github/workflows/pypi-package.yml, which I'm successfully using in my own projects.

1.

For all runs, it builds and lints the PyPI sdist and wheel. This makes sure the non-publishing steps are working smoothly, ahead of uploading. It uses @hynek's excellent https://github.com/hynek/build-and-inspect-python-package action, which creates a nice summary, for example:

image

We can see it creates three handy zips if we want to check what would be uploaded to PyPI, and three handy toggles that show the contents. Try it here:

https://github.com/python/python-docs-theme/actions/runs/5919197594?pr=148

2.

For commits to main we publish to TestPyPI. This helps us know the publishing process is nicely oiled to avoid surprises on release day. It uses the packages that were already created in step 1.

3.

For published GitHub releases we publish to the real PyPI. Also uses packages created in step 1.


TODO

  • [n/a] Set up trusted publishing on TestPyPI:
    1. Go to https://test.pypi.org/manage/project/python-docs-theme/settings/publishing/
    2. Add a new publisher like:
screenshot image
  • Set up trusted publishing on PyPI:
    1. Go to https://pypi.org/manage/project/python-docs-theme/settings/publishing/
    2. Add a new publisher as above

@willingc I see you're the only maintainer for https://test.pypi.org/project/python-docs-theme/, please could you set up the above for TestPyPI? It's probably a good opportunity to add some extra owners/maintainers such as @Mariatta, to improve the bus factor.

@willingc You're also one of the maintainers for https://pypi.org/project/python-docs-theme/, so perhaps you could do PyPI too?

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
@hugovk
Build & maybe upload PyPI package via trusted publishing
3f1b3b3
@hugovk hugovk added the enhancement label Aug 20, 2023
AA-Turner
AA-Turner reviewed Aug 20, 2023
View reviewed changes

@AA-Turner AA-Turner left a comment

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've reviewed this back-to-front -- sorry. I'm a tad uncertain if we should be going with this though. Overarching thoughts:

  • I think that the 'check the package is alright' step could happen via a linting run, e.g. just running twine check, which avoids the kerfuffle with uploading artefacts & to Test PyPI
  • On Test PyPI, I'm unconvinced that we should use it for every commit -- we know that the package works, and there are limited resources on Test PyPI, even if the distribution is small, so I think we ought be conservative in what we upload.
  • I'm unsure what's behind hynek/build-and-inspect-python-package -- whilst I could go and look it up, it adds a layer of indirection when (to my understanding) this workflow should be replicating the release steps one would run locally -- e.g. python -m build; twine upload. Making it explicit what's going on I think is useful to observability and just having fewer moving parts in general (I'd have the same objection to pypa/gh-action-pypi-publish, save that it implements the OIDC flow for us).

Sorry again for several comments, but I hope this outlines my position to some extent.

A

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
Comment thread .github/workflows/pypi-package.yml Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread .github/workflows/pypi-package.yml Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread .github/workflows/pypi-package.yml Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread .github/workflows/pypi-package.yml Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread .github/workflows/pypi-package.yml Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread .github/workflows/pypi-package.yml Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread .github/workflows/pypi-package.yml Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread .github/workflows/pypi-package.yml Resolved

Uh oh!

There was an error while loading. Please reload this page.

@hugovk

hugovk commented Aug 21, 2023

Copy link
Copy Markdown
Member Author
  • I think that the 'check the package is alright' step could happen via a linting run, e.g. just running twine check, which avoids the kerfuffle with uploading artefacts & to Test PyPI

There is no linter that does all the checks that PyPI does.

twine check only does a subset of the checks. I've seen packages pass twine check yet fail to upload.

It's no kerfuffle, this PR adds automation so machines take care of it for us. 🤖

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@hugovk

hugovk commented Aug 21, 2023 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Member Author
  • On Test PyPI, I'm unconvinced that we should use it for every commit -- we know that the package works, and there are limited resources on Test PyPI, even if the distribution is small, so I think we ought be conservative in what we upload.

We only know for sure the package works and can be uploaded by uploading it.

These pure Python packages are very small, especially when compared to those with C extensions which may have 60+ multi-MB wheels.

Resources are indeed limited on TestPyPI, and I doubt we'll ever reach the project limit. If we do, we can safely delete old uploads.

Disregard, removed: #148 (comment)

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@hugovk

hugovk commented Aug 21, 2023

Copy link
Copy Markdown
Member Author
  • I'm unsure what's behind hynek/build-and-inspect-python-package -- whilst I could go and look it up, it adds a layer of indirection when (to my understanding) this workflow should be replicating the release steps one would run locally -- e.g. python -m build; twine upload. Making it explicit what's going on I think is useful to observability and just having fewer moving parts in general (I'd have the same objection to pypa/gh-action-pypi-publish, save that it implements the OIDC flow for us).

I find the build and content summaries can be useful. It does lots of extra lint checks beyond twine check. It makes packages avaliable before upload, so we can test them locally if we like.

Best of all, we don't need to maintain it! We can benefit from @hynek and other contributors maintaining and improving it. We can also re-use it in other repos without duplicating all the extra checks across our repos.

(Here's the source: https://github.com/hynek/build-and-inspect-python-package)

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

hugovk and others added 4 commits August 21, 2023 08:29
@hugovk @AA-Turner
Apply suggestions from code review
1e4b09e
Co-authored-by: Adam Turner <[email protected]>
@hugovk
Tighten permissions
a1740d8
@hugovk
Don't fetch all history
5c5e94d
@hugovk
Don't upload to TestPyPI
6d82be0
@hugovk

hugovk commented Aug 21, 2023

Copy link
Copy Markdown
Member Author

I removed the TestPyPI upload because we hardcode the version and that would have meant the second and subsequent uploads would fail because you cannot reuse filenames on PyPI.

I don't want to expand the scope to propose using something like hatch-vcs or setuptools_scm in this PR, it's more important to get trusted publishing in place. 👍

AA-Turner reacted with thumbs up emoji
All reactions
  • 👍 1 reaction

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@AA-Turner AA-Turner mentioned this pull request Aug 24, 2023
Incorrectly highlighted paragraph in dark theme #149
Closed
@hugovk
Merge remote-tracking branch 'upstream/main'
9d507a1
AA-Turner
AA-Turner reviewed Aug 25, 2023
View reviewed changes
Comment thread .github/workflows/pypi-package.yml Resolved

Uh oh!

There was an error while loading. Please reload this page.

@hugovk
Simplify action triggers
719222f
@hugovk
hugovk marked this pull request as ready for review August 26, 2023 08:53
AA-Turner
AA-Turner approved these changes Aug 26, 2023
View reviewed changes
@AA-Turner

AA-Turner commented Aug 26, 2023 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Member

Thanks for your patience with my questions--excited to see trusted publishing enabled for python-docs-theme!

A

hugovk reacted with hooray emoji
All reactions
  • 🎉 1 reaction

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

ezio-melotti
ezio-melotti reviewed Aug 26, 2023
View reviewed changes
Comment thread .github/workflows/pypi-package.yml Resolved

Uh oh!

There was an error while loading. Please reload this page.

hugovk added 2 commits August 26, 2023 18:22
@hugovk
Comment when the release-pypi step is run
f15aa6d
@hugovk
Merge remote-tracking branch 'upstream/main'
ba4ecd8
pradyunsg
pradyunsg approved these changes Aug 30, 2023
View reviewed changes
@willingc

willingc commented Sep 30, 2023

Copy link
Copy Markdown
Contributor

@hugovk I'm just re-engaging with core development a bit more now that I'm semi-retired. Let me know what you need me to do re: pypi. Thanks.

hynek, hugovk, and pradyunsg reacted with hooray emoji
All reactions
  • 🎉 3 reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@hugovk

hugovk commented Sep 30, 2023

Copy link
Copy Markdown
Member Author

Congrats on the semi-retirement!

  1. Log in to pypi.org
  2. Go to https://pypi.org/manage/project/python-docs-theme/settings/publishing/
  3. Add a new publisher by filling in the form like this (leave the last env name box blank):
image

That's it!

pradyunsg reacted with heart emoji
All reactions
  • ❤️ 1 reaction

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@willingc

willingc commented Sep 30, 2023

Copy link
Copy Markdown
Contributor

@hugovk Thanks for the great instructions. It's now completed. ✨

AA-Turner and pradyunsg reacted with hooray emoji
All reactions
  • 🎉 2 reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@hugovk

hugovk commented Oct 1, 2023

Copy link
Copy Markdown
Member Author

Thank you!

Let's merge this now, and I can try out a release with #153 next week, after 3.12 is out.

willingc reacted with thumbs up emoji
All reactions
  • 👍 1 reaction

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@hugovk
hugovk merged commit 3f48c0c into python:main Oct 1, 2023
@hugovk hugovk mentioned this pull request Oct 4, 2023
Focus search box when pressing slash #153
Merged
@hugovk

hugovk commented Oct 4, 2023

Copy link
Copy Markdown
Member Author

Success!

I merged #153, updated version and changelog in #157 and followed these steps (which I'll update in CONTRIBUTING.md):

  • Update CHANGELOG.rst
  • Bump version (YYYY.MM) in pyproject.toml
  • Commit
  • Push to check tests pass on GitHub Actions
  • Go to https://github.com/python/python-docs-theme/releases
  • Click "Draft a new release"
  • Click "Choose a tag", type the next version (for example "2023.09") and select "Create new tag: 2023.9 on publish"
  • Click "Generate release notes" and amend as required
  • Click "Publish release"
  • Check the tagged GitHub Actions build has deployed to PyPI

The tag/release:

  • https://github.com/python/python-docs-theme/releases/tag/2023.09

The CI build and deploy:

Screenshot

image

Screenshot

image

  • https://github.com/python/python-docs-theme/actions/runs/6407907553

The release on PyPI:

  • https://pypi.org/project/python-docs-theme/2023.9/
hynek and ezio-melotti reacted with rocket emoji
All reactions
  • 🚀 2 reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@hugovk hugovk mentioned this pull request Oct 4, 2023
Update release checklist #158
Merged
@hugovk

hugovk commented Oct 4, 2023

Copy link
Copy Markdown
Member Author

(which I'll update in CONTRIBUTING.md):

Please see PR #158.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@Mariatta

Mariatta commented Oct 4, 2023

Copy link
Copy Markdown
Member

Great work. Thanks!

hugovk reacted with thumbs up emoji
All reactions
  • 👍 1 reaction

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@ezio-melotti ezio-melotti ezio-melotti left review comments
@pradyunsg pradyunsg pradyunsg approved these changes
@AA-Turner AA-Turner AA-Turner approved these changes

Uh oh!

There was an error while loading. Please reload this page.

Assignees

No one assigned

Labels

enhancement

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Uh oh!

There was an error while loading. Please reload this page.

6 participants

@hugovk @AA-Turner @willingc @Mariatta @pradyunsg @ezio-melotti
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
You are visitor 01402503 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.