cd: re-enable release signing

[lukpueh]

Description of issue or feature request:
#1946 adds a CD workflow to release build artifacts on PyPI and GH upon successful completion of the CI workflow for a pushed release tag.

The PR also removes instructions from RELEASE.md to gpg sign release artifacts and add them to the GitHub release assets as part of the previously manual release process. However, the installation docs still mention release signatures.

Current behavior:
No instructions / release process integration to sign release artifacts

Expected behavior:
Add instructions to sign release artifacts and integrate with release process

Ideas:

• quick-fix 1: sign in GitHub CD action
• quick-fix 2: sign locally and upload signatures to release assets manually (bonus: integrate signing/uploading with verify_release script)
• long-term fix: e.g. in-toto (Add in-toto metadata to python-tuf releases #529)

[jku]

quick-fix 1: sign in GitHub CD action

This is possible with a GPG key of course (where private key is in an environment secret) but Sigstore cosign has experimental support for this as well: https://blog.chainguard.dev/zero-friction-keyless-signing-with-github-actions/

• requires the cosign-installer action (or otherwise access to cosign tool)
• uses GitHub as the OIDC authenticator (authenticates a specific repo and the deployment environment)
• uses an ephemeral key to sign the release
• uploads the signature to a public log

We'd need to at least document how to verify this signature (as signing/verifying non-container images is not as widely used in cosign). The actual signing command is also not listed in the blog post (again as non-containers are a bit of an after thought)

It's also possible this is still a little too experimental, and we should reconsider this solution in six months.

[jku]

This space seems to move very rapidly at the moment so just documenting a few things for myself if not others:

• lukas has a hand crafted in-toto solution linked above: it's neat but IMO not very easy to understand or currently to verify
• SLSA is building a "packaged" solution in slsa-github-generator -- this is new and seems to still be under active development
• just using sigstore-python with the Github signing certificate could IMO get us 99% there with 1% of the work (Dogfood verification sigstore/sigstore-python#106) -- but sigstore-python isn't quite ready yet either: the certificate details are currently not exposed to callers so they can't be verified

These are all mostly options for build system signing (which I think is the important bit). I think the minimum we should aim for with any solution is

• easy (single command) verification for a developer who understands the basics of provenance validation
• reasonable maintenance burden

[lukpueh]

Thanks for posting this summary!

IMO there are two things that make the hand-crafted in-toto solution so hard to understand/verify:

1. Key management and metadata distribution (issue summary here) -- This is arguably easier with a sigstore-based approach with their ephemeral keys, OIDC-providers, identity/signature logs, etc.

2. Signed Policy (i.e. attestation requirements, artifact interrelation, individual signer authentication, thresholds; see example policy document here ) -- This is simply not considered by the other approaches, which of course makes their verification comparably easier.

Wrt (1) in-toto could learn from the other approaches, about (2) it should probably inspire them.

[jku]

Maybe time to discuss this one again? sigstore-python now does attestations with slsa-github-generator (this is the generic "provenance only" generator): https://github.com/sigstore/sigstore-python/blob/main/.github/workflows/release.yml#L82

I understand that that does not cover all the bases this PR does (as explained in previous comment)... but their approach seems more approachable and easier to integrate. python-tuf project obviously has the expertise to do whatever intoto magic we want but... I would like to be able to show python-tuf as an example for others who won't have that expertise: in this regard the slsa-github-generator approach seems nice.

I think I would benefit from an interactive session where we go through the various attestations we need (and the ones we want); what is provided by the slsa-github-generator approach, what isn't; what is the verification story for these different attestations, and how they affect the maintainer workflows

Maybe we can catch up on slack next week and choose a time to discuss?

[joshuagl]

I understand that that does not cover all the bases this PR does (as explained in previous comment)... but their approach seems more approachable and easier to integrate. python-tuf project obviously has the expertise to do whatever intoto magic we want but... I would like to be able to show python-tuf as an example for others who won't have that expertise: in this regard the slsa-github-generator approach seems nice.

I think this is a great goal, so long as we are not unduly reducing our security so that we're an easier example to follow.

Maybe we can catch up on slack next week and choose a time to discuss?

➕1️⃣