Description

Docker Github builder creates a strong signature proving the link between the commit from a source repository and the resulting build artifact. The artifact had to be built from the specific commit, exactly as it is fully defined in the signed provenance attestation. Even if you obtain a leaked token or have write or admin access to the source repository, you can't fake this signature.

This is great security for the build process, but it doesn't provide extra protection if the source commit was already malicious. If a malicious commit is legitimately pushed to the source repository, DGB will happily build that commit and sign the resulting artifact. In that case, your only protection is that you can always trace your way back to this malicious commit and possibly use it to revoke certain artifacts based on that afterward.

This proposal explores opportunities to detect such cases before the builder runs and stops before any artifact from a malicious commit is actually created.

In order for the builder to detect if the commit is valid and they should continue, it needs to look at some properties of the build that are hard to fake.

These would be:

• The release commit points to an annotated signed tag that is signed by a specific HSM. Assumption is that leaking repository credentials would not give access to this HSM or at least any unexpected HSM usage would be logged or trigger notifications.
• A quorum of the source repository maintainers would need to approve the commit. An attacker would need to compromise multiple maintainers for a successful attack, which is significantly harder.
• Optionally, we could consider that some specific code scanners need to run and pass on the source commit for the builder to pass.

This sort of rule we can refer to as "signing policy", meaning a policy that needs to pass on source commit for us to consider it valid for signing. Now we need to figure out how to securely define and store such policy.

Signing policy update flow

As an example lets consider Rego-style policy similar to Build Source policy:

allow = false

allow if = {
  input.tag.signer = "HSM_KEY"
}

allow if = {
  input.tag.approveQuota = 3
}

allow if = {
  input.tag.approveName = ["crazy-max", "abc"]
}

codescanned = if {
  input.
}

As the source repo owner, you first push the rego to your repo.

Then you run workflows/github-builder-signing-policy-update.yml on main branch.

The workflow has an optional environment clause if approveQuota is set. https://docs.github.com/en/actions/how-tos/deploy/configure-and-manage-deployments/manage-environments

After the environment gets confirmed, the local workflow calls a reusable workflow github-builder/github-builder-signing-policy-update (probably better to use a different repo than the main builder).

The reusable workflow calls back to the source repository to check who approved the environment. This should work fine for public repositories using the GITHUB_TOKEN in the environment, but private repositories may need something special.

If the policy matches approvals, then the file is:

• signed with the current reusable workflow sigstore scope (similar unforgeable proof as our image attestation signatures)
• then saved into a transparent storage location by the repository name(maybe internal repository ID):
  • We need to find a KV store that is publicly readable, but only our OIDC scope can write to
  • Maybe existing transparency logs could be used, but really we would need a KV storage
  • We could just use a static Github repository for storage and access policies through https://raw.githubusercontent.com/ without the need to clone full repo.

When .docker/github-builder.rego is updated, the workflow is invoked again. Reusable workflow first pulls down the last signed policy. The environment needs to match the previous policy and the new one. If everything is correct then policy in storage gets updated.

Build flow

Before every GitHub Builder build or bake flow, the signing policy is pulled down from the static storage based on the source repo name. If one does not exists then it is ignored. If it does exist then the policy is evaluated: 1) is the policy correctly signed with Sigstore 2) is the git tag signed? 2) does the current run URL have the environment approvals. If environment approvals are used, then the user needs to add an environment block into their build workflow before the call to the reusable workflow. If they don’t do this, then all builds fail.

Notes

• The Github Environments have nice UI for doing in-person approvals, but it seems that they have no support for the concept of quorum. This could be potentially worked around with two environments, so that first gathers quorum and second triggers the build to continue. But maybe there is a better alternative?
• When using quorum, the signers need to be named. If it is just a number, then if the admin credentials leak, the attacker can just make N number of new users and give them write permissions to pass the policy.
• When using some code scanner pass rule, the question would be if these run in the source repo scope or the reusable workflow scope. Source repo scope could potentially be insecure, as the source repo can likely be modified to add "ignore" exceptions to the scanner configuration. When using workflow scope, we could probably only support a couple of specific scanners with specific configuration options.