CD - Release¶
Important
The default for publish_on_pypi
has changed from true
to false
in version 2.8.0
.
To keep using the previous behaviour, set publish_on_pypi: true
in the workflow file.
This change has been introduced to push for the use of PyPI's Trusted Publisher feature, which is not yet supported by reusable/callable workflows.
See the Using PyPI's Trusted Publisher section for more information on how to migrate to this feature.
File to use: cd_release.yml
There are 2 jobs in this workflow, which run in sequence.
First, an update & publish job, which updates the version in the package's root __init__.py
file through an Invoke task.
The newly created tag (created due to the caller workflow running on.release.types.published
) will be updated accordingly, as will the publish branch (defaults to main
).
Secondly, a job to update the documentation is run, however, this can be deactivated. The job expects the documentation to be setup with either the mike+MkDocs+GitHub Pages framework or the Sphinx framework.
For more information about the specific changelog inputs, see the related changelog generator actually used, specifically the list of configuration options.
Note
Concerning the changelog generator, the specific input changelog_exclude_labels
defaults to a list of different labels if not supplied, hence, if supplied, one might want to include these labels alongside any extra labels.
The default value is given here as a help:
'duplicate,question,invalid,wontfix'
The changelog_exclude_tags_regex
is also used to remove tags in a list of tags to consider when evaluating the "previous version".
This is specifically for adding a changelog to the GitHub release body.
If used together with the Update API Reference in Documentation, please align the relative
input with the --relative
option, when running the hook.
See the proper section to understand why and how these options and inputs should be aligned.
Using PyPI's Trusted Publisher¶
PyPI has introduced a feature called Trusted Publisher which allows for a more secure way of publishing packages using OpenID Connect (OIDC).
This feature is not yet supported by reusable/callable workflows, but can be used by setting up a GitHub Action workflow in your repository that calls the cd_release.yml
workflow in one job, setting the publish_on_pypi
input to false
and the upload_distribution
input to true
, and then using the uploaded artifact to publish the package to PyPI in a subsequent job.
In this way you can still benefit from the cd_release.yml
dynamically updated workflow, while using PyPI's Trusted Publisher feature.
Info
The artifact name is statically set to dist
.
If the workflow is run multiple times, the artifact will be overwritten.
Retention time for the artifact is kept at the GitHub default (currently 90 days).
Important
The id-token:write
permission is required by the PyPI upload action for Trusted Publishers.
The following is an example of how a workflow may look that calls CD - Release and uses the uploaded built distribution artifact to publish the package to PyPI.
Note, the non-default dists
directory is chosen for the built distribution, and the artifact is downloaded to the my-dists
directory.
name: CD - Publish
on:
release:
types:
- published
jobs:
build:
name: Build distribution & publish documentation
if: github.repository == 'SINTEF/my-python-package' && startsWith(github.ref, 'refs/tags/v')
uses: SINTEF/ci-cd/.github/workflows/cd_release.yml@v2.8.3
with:
# General
git_username: "Casper Welzel Andersen"
git_email: "CasperWA@github.com"
release_branch: stable
# Build distribution
python_package: true
package_dirs: my_python_package
install_extras: "[dev,build]"
build_libs: build
build_cmd: "python -m build -o dists"
build_dir: dists
publish_on_pypi: false
upload_distribution: true
# Publish documentation
update_docs: true
doc_extras: "[docs]"
docs_framework: mkdocs
secrets:
PAT: ${{ secrets.PAT }}
publish:
name: Publish to PyPI
needs: build
runs-on: ubuntu-latest
# Using environments is recommended by PyPI when using Trusted Publishers
environment: release
# The id-token:write permission is required by the PyPI upload action for
# Trusted Publishers
permissions:
id-token: write
steps:
- name: Download distribution
uses: actions/download-artifact@v4
with:
name: dist # The artifact will always be called 'dist'
path: my-dists
- name: Publish to PyPI
uses: pypa/gh-action-pypi-publish@release/v1
with:
# The path to the distribution to upload
packages-dir: my-dists/
Updating instances of version in repository files¶
The content of repository files can be updated to use the new version where necessary.
This is done through the version_update_changes
(and version_update_changes_separator
) inputs.
To see an example of how to use the version_update_changes
(and version_update_changes_separator
) see for example the workflow used by the SINTEF/ci-cd repository calling the CD Release workflow.
Some notes to consider and respect when using version_update_changes
are:
- The value of
version_update_changes_separator
applies to all lines given inversion_update_changes
, meaning it should be a character, or series of characters, which will not be part of the actual content. -
Specifically, concerning the 'raw' Python string 'pattern' the following applies:
- Always escape double quotes (
"
). This is done by prefixing it with a backslash (\
):\"
. - Escape special bash/sh characters, e.g., back tick (
`
). - Escape special Python regular expression characters, if they are not used for their intended purpose in this 'raw' string.
See the
re
library documentation for more information.
- Always escape double quotes (
Concerning the 'replacement string' part, the package_dirs
input and full semantic version can be substituted in dynamically by wrapping either package_dir
or version
in curly braces ({}
).
Indeed, for the version, one can specify sub-parts of the version to use, e.g., if one desires to only use the major version, this can be done by using the major
attribute: {version.major}
.
The full list of version attributes are: major
, minor
, patch
, pre_release
, and build
.
More can be used, e.g., to only insert the major.minor version: {version.major}.{version.minor}
.
For the 'file path' part, package_dir
wrapped in curly braces ({}
) will also be substituted at run time with each line from the possibly multi-line package_dirs
input.
E.g., {package_dir}/__init__.py
will become ci_cd/__init__.py
if the package_dirs
input was 'ci_cd'
.
Expectations¶
This workflow should only be used for releasing a single modern Python package.
The repository contains the following:
- (required) A Python package root
__init__.py
file with__version__
defined. - (required) The workflow is run for a git tag only.
This will automatically be the case for a GitHub release.
The tag name must be a valid semantic version.
See SemVer.org for more information about semantic versioning.
It is expected, but not required, that the tag name starts with a
v
, e.g.,v1.0.0
.
Inputs¶
The following inputs are general inputs for the workflow as a whole.
Name | Description | Required | Default | Type |
---|---|---|---|---|
git_username |
A git username (used to set the 'user.name' config option). | Yes | string | |
git_email |
A git user's email address (used to set the 'user.email' config option). | Yes | string | |
release_branch |
The branch name to release/publish from. | Yes | main | string |
runner |
The runner to use for the workflow. Note, the callable workflow expects a Linux/Unix system.. | No | ubuntu-latest | string |
install_extras |
Any extras to install from the local repository through 'pip'. Must be encapsulated in square parentheses ([] ) and be separated by commas (, ) without any spaces.Example: '[dev,release]' . |
No | Empty string | string |
relative |
Whether or not to use install the local Python package(s) as an editable. | No | false |
boolean |
test |
Whether to use the TestPyPI repository index instead of PyPI as well as output debug statements in both workflow jobs. | No | false |
boolean |
pip_index_url |
A URL to a PyPI repository index. | No | https://pypi.org/simple/ |
string |
pip_extra_index_urls |
A space-delimited string of URLs to additional PyPI repository indices. | No | Empty string | string |
Inputs related to updating the version, building and releasing the Python package to PyPI.
Name | Description | Required | Default | Type |
---|---|---|---|---|
publish_on_pypi |
Whether or not to publish on PyPI.Note: This is only relevant if 'python_package' is 'true', which is the default.Important: The default has changed from true to false to push for the use of PyPI's Trusted Publisher feature.See the Using PyPI's Trusted Publisher section for more information on how to migrate to this feature. |
Yes (will be non-required in v2.9) | false |
boolean |
python_package |
Whether or not this is a Python package, where the version should be updated in the 'package_dir'/__init__.py for the possibly several 'package_dir' lines given in the package_dirs input and a build and release to PyPI should be performed. |
No | true |
boolean |
python_version_build |
The Python version to use for the workflow when building the package. | No | 3.9 | string |
package_dirs |
A multi-line string of paths to Python package directories relative to the repository directory to have its __version__ value updated.Example: 'src/my_package' .Important: This is required if 'python_package' is 'true', which is the default.See also Single vs multi-line input. |
Yes (if 'python_package' is 'true') | string | |
version_update_changes |
A multi-line string of changes to be implemented in the repository files upon updating the version. The string should be made up of three parts: 'file path', 'pattern', and 'replacement string'. These are separated by the 'version_update_changes_separator' value.The 'file path' must always either be relative to the repository root directory or absolute.The 'pattern' should be given as a 'raw' Python string.See also Single vs multi-line input. | No | Empty string | string |
version_update_changes_separator |
The separator to use for 'version_update_changes' when splitting the three parts of each string. | No | , | string |
build_libs |
A space-separated list of packages to install via PyPI (pip install ). |
No | Empty string | string |
build_cmd |
The package build command, e.g., 'flit build' or 'python -m build' . |
No | python -m build --outdir dist . |
string |
build_dir |
The directory where the built distribution is located. This should reflect the directory used in the build command or by default by the build library. | No | dist |
string |
tag_message_file |
Relative path to a release tag message file from the root of the repository.Example: '.github/utils/release_tag_msg.txt' . |
No | Empty string | string |
changelog_exclude_tags_regex |
A regular expression matching any tags that should be excluded from the CHANGELOG.md. | No | Empty string | string |
changelog_exclude_labels |
Comma-separated list of labels to exclude from the CHANGELOG.md. | No | Empty string | string |
upload_distribution |
Whether or not to upload the built distribution as an artifact.Note: This is only relevant if 'python_package' is 'true', which is the default. | No | true |
boolean |
Inputs related to building and releasing the documentation in general.
Name | Description | Required | Default | Type |
---|---|---|---|---|
update_docs |
Whether or not to also run the 'docs' workflow job. | No | false |
boolean |
python_version_docs |
The Python version to use for the workflow when building the documentation. | No | 3.9 | string |
doc_extras |
Any extras to install from the local repository through 'pip'. Must be encapsulated in square parentheses ([] ) and be separated by commas (, ) without any spaces.Note, if this is empty, 'install_extras' will be used as a fallback.Example: '[docs]' . |
No | Empty string | string |
docs_framework |
The documentation framework to use. This can only be either 'mkdocs' or 'sphinx' . |
No | mkdocs | string |
system_dependencies |
A single (space-separated) or multi-line string of Ubuntu APT packages to install prior to building the documentation.See also Single vs multi-line input. | No | Empty string | string |
Inputs related only to the MkDocs framework.
Name | Description | Required | Default | Type |
---|---|---|---|---|
mkdocs_update_latest |
Whether or not to update the 'latest' alias to point to release_branch . |
No | true |
boolean |
Finally, inputs related only to the Sphinx framework.
Name | Description | Required | Default | Type |
---|---|---|---|---|
sphinx-build_options |
Single (space-separated) or multi-line string of command-line options to use when calling sphinx-build .See also Single vs multi-line input. |
No | Empty string | string |
docs_folder |
The path to the root documentation folder relative to the repository root. | No | docs | string |
build_target_folder |
The path to the target folder for the documentation build relative to the repository root. | No | site | string |
Secrets¶
Name | Description | Required |
---|---|---|
PyPI_token |
A PyPI token for publishing the built package to PyPI.Important: This is required if both 'python_package' and 'publish_on_pypi' are 'true'. Both are 'true' by default. | Yes (if 'python_package' and 'publish_on_pypi' are 'true') |
PAT |
A personal access token (PAT) with rights to update the release_branch . This will fallback on GITHUB_TOKEN . |
No |
Usage example¶
The following is an example of how a workflow may look that calls CD - Release. It is meant to be complete as is.
name: CD - Publish
on:
release:
types:
- published
jobs:
publish:
name: Publish package and documentation
uses: SINTEF/ci-cd/.github/workflows/cd_release.yml@v2.8.3
if: github.repository == 'SINTEF/my-python-package' && startsWith(github.ref, 'refs/tags/v')
with:
# General
git_username: "Casper Welzel Andersen"
git_email: "CasperWA@github.com"
release_branch: stable
# Publish distribution
package_dirs: my_python_package
install_extras: "[dev,build]"
build_cmd: "pip install flit && flit build"
tag_message_file: ".github/utils/release_tag_msg.txt"
changelog_exclude_labels: "skip_changelog,duplicate"
publish_on_pypi: true
# Publish documentation
update_docs: true
doc_extras: "[docs]"
secrets:
PyPI_token: ${{ secrets.PYPI_TOKEN }}
PAT: ${{ secrets.PAT }}