Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add trusted publisher release workfiow #13048

Open
wants to merge 3 commits into
base: main
Choose a base branch
from
+51 −98
Open

Add trusted publisher release workfiow #13048

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
29af9e3
Add trusted publisher release workfiow
sbidoul Oct 27, 2024
a6b5aee
Improve job and step names
sbidoul Nov 19, 2024
00a38e5
Align release workflow with latest recommendations
sbidoul Dec 10, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 3 additions & 1 deletion .github/workflows/ci.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -78,7 +78,9 @@ jobs:

- run: pip install nox
- run: nox -s prepare-release -- 99.9
- run: nox -s build-release -- 99.9
- run: git checkout 99.9
- run: pipx run build
- run: pipx run twine check dist/*
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's best to catch this early:

Suggested change
- run: pipx run twine check dist/*
- run: pipx run twine check --strict dist/*

I think pypi-publish does a strict check so having it less strict might reveal unpleasant surprised too late in the process.

- run: pipx run check-manifest

vendoring:
Expand Down
46 changes: 46 additions & 0 deletions .github/workflows/release.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,46 @@
name: Publish Python 🐍 distribution 📦 to PyPI

on:
push:
tags:
- "*"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is unnecessary, it's the same by default:

Suggested change
- "*"


jobs:
build:
name: Build distribution 📦
runs-on: ubuntu-latest

steps:
- uses: actions/checkout@v4
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pin all the action steps to commit SHAs instead of git tags to avoid a source of immutability. You can use frizbee to do this for you if you'd like.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There's also https://github.com/davidism/gha-update. And Dependabot knows to update the hashes too (also bumping the human-readable tag in a comment on the same line).

- name: Set up Python
uses: actions/setup-python@v5
with:
python-version: "3.x"
- name: Build a binary wheel and a source tarball
run: pipx run build
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are you sure the syntax is correct? IIRC build is only exposed via runpy by this name (as in python -Im build). It also exposes a console script pyproject-build, I think. I'd double check the pipx invocation..

FWIW I tend to invoke this wrapped with tox. So if there's anything in nox already — use that here so the same command+env is used everywhere.

Additionally, I'd like to point out that this command is unversioned. With @sethmlarson's suggestions to pin the actions, it feels like a missing reproducibility bit when this is uncontrolled. Perhaps, put the dependencies into constraint files?

And I just remembered that I like setting the PIP_CONSTRAINT env var because this is the only way to pin the build deps (setuptools/wheel). Consider looking into pinning them and wiring into that nox command.
Generating such a lock file is possible with pip-tools — I merged the corresponding PR a little over a year ago.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are you sure the syntax is correct

Yes, pipx run build works, thank to https://github.com/pypa/build/blob/88ae8330150f253dd70170368c311d6374315632/pyproject.toml#L84-L85

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ack. Though, I'm still asking you to consider #13048 (comment).

- name: Store the distribution packages
uses: actions/upload-artifact@v4
with:
name: python-package-distributions
path: dist/

publish-to-pypi:
name: >-
Publish Python 🐍 distribution 📦 to PyPI
needs:
- build
runs-on: ubuntu-latest
environment:
name: pypi
url: https://pypi.org/p/pip/${{ github.ref_name }}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Did I suggest that /p/? This shortcut only works for unversioned URLs. Let's unbreak it:

Suggested change
url: https://pypi.org/p/pip/${{ github.ref_name }}
url: https://pypi.org/project/pip/${{ github.ref_name }}

permissions:
id-token: write # IMPORTANT: mandatory for trusted publishing

steps:
- name: Download all the dists
uses: actions/download-artifact@v4
with:
name: python-package-distributions
path: dist/
- name: Publish distribution 📦 to PyPI
uses: pypa/gh-action-pypi-publish@release/v1
7 changes: 2 additions & 5 deletions docs/html/development/release-process.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -146,11 +146,8 @@ Creating a new release
This will update the relevant files and tag the correct commit.
#. Submit the ``release/YY.N`` branch as a pull request and ensure CI passes.
Merge the changes back into ``main`` and pull them back locally.
#. Build the release artifacts using ``nox -s build-release -- YY.N``.
This will checkout the tag, generate the distribution files to be
uploaded and checkout the main branch again.
#. Upload the release to PyPI using ``nox -s upload-release -- YY.N``.
#. Push the tag created by ``prepare-release``.
#. Push the tag created by ``prepare-release``. This will trigger the release
workflow on GitHub and publish to PyPI.
#. Regenerate the ``get-pip.py`` script in the `get-pip repository`_ (as
documented there) and commit the results.
#. Submit a Pull Request to `CPython`_ adding the new version of pip
Expand Down
92 changes: 0 additions & 92 deletions noxfile.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,6 @@
"""

import argparse
import glob
import os
import shutil
import sys
Expand Down Expand Up @@ -315,94 +314,3 @@ def prepare_release(session: nox.Session) -> None:
next_dev_version = release.get_next_development_version(version)
release.update_version_file(next_dev_version, VERSION_FILE)
release.commit_file(session, VERSION_FILE, message="Bump for development")


@nox.session(name="build-release")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we keep this env so nox is still in the center of the processes and the manual ways remain in the repo? Plus, consider implementing pinning of the build env on this level per my other comments.

def build_release(session: nox.Session) -> None:
version = release.get_version_from_arguments(session)
if not version:
session.error("Usage: nox -s build-release -- YY.N[.P]")

session.log("# Ensure no files in dist/")
if release.have_files_in_folder("dist"):
session.error(
"There are files in dist/. Remove them and try again. "
"You can use `git clean -fxdi -- dist` command to do this"
)

session.log("# Install dependencies")
session.install("build", "twine")

with release.isolated_temporary_checkout(session, version) as build_dir:
session.log(
"# Start the build in an isolated, "
f"temporary Git checkout at {build_dir!s}",
)
with release.workdir(session, build_dir):
tmp_dists = build_dists(session)

tmp_dist_paths = (build_dir / p for p in tmp_dists)
session.log(f"# Copying dists from {build_dir}")
os.makedirs("dist", exist_ok=True)
for dist, final in zip(tmp_dist_paths, tmp_dists):
session.log(f"# Copying {dist} to {final}")
shutil.copy(dist, final)


def build_dists(session: nox.Session) -> List[str]:
"""Return dists with valid metadata."""
session.log(
"# Check if there's any Git-untracked files before building the wheel",
)

has_forbidden_git_untracked_files = any(
# Don't report the environment this session is running in
not untracked_file.startswith(".nox/build-release/")
for untracked_file in release.get_git_untracked_files()
)
if has_forbidden_git_untracked_files:
session.error(
"There are untracked files in the working directory. "
"Remove them and try again",
)

session.log("# Build distributions")
session.run("python", "-m", "build", silent=True)
produced_dists = glob.glob("dist/*")

session.log(f"# Verify distributions: {', '.join(produced_dists)}")
session.run("twine", "check", *produced_dists, silent=True)

return produced_dists


@nox.session(name="upload-release")
def upload_release(session: nox.Session) -> None:
version = release.get_version_from_arguments(session)
if not version:
session.error("Usage: nox -s upload-release -- YY.N[.P]")

session.log("# Install dependencies")
session.install("twine")

distribution_files = glob.glob("dist/*")
session.log(f"# Distribution files: {distribution_files}")

# Sanity check: Make sure there's 2 distribution files.
count = len(distribution_files)
if count != 2:
session.error(
f"Expected 2 distribution files for upload, got {count}. "
f"Remove dist/ and run 'nox -s build-release -- {version}'"
)
# Sanity check: Make sure the files are correctly named.
distfile_names = (os.path.basename(fn) for fn in distribution_files)
expected_distribution_files = [
f"pip-{version}-py3-none-any.whl",
f"pip-{version}.tar.gz",
]
if sorted(distfile_names) != sorted(expected_distribution_files):
session.error(f"Distribution files do not seem to be for {version} release.")

session.log("# Upload distributions")
session.run("twine", "upload", *distribution_files)
Loading