add: Sign and Verify Images from the CLI (#2043)

[iwilltry42]

Ref #2043

This PR changed quite a lot due to lots of discussions around UX, etc.

Here's the current state (20.07.2023):

The CLI-side

• new commands: acorn image sign & acorn image verify to sign and verify local/remote images
• new flags:
  • acorn image pull --verify to only pull an image if verification passed (extra flags --key and --annotation)
  • acorn image push --sign to sign an image and push it together with the new signature (extra flags --key and --annotation)

The Server Side

• ImageDetails now contain the signature digest if we could find a signature for an image
• new image/sign and image/verify subresources, where we never pass in the private key, only public keys
• Verification (from CLI or as part of IARs) now support gh:// and ac:// identifiers to pull public keys from GitHub and Acorn Manager user accounts for verification
• ImagePush and ImagePull now take care of the signature as well if there is one
• ImageCopy and RepoCopy now both consider signatures as well
• ImageDelete cleans up signatures from the registry

Discussion

Answered

• Q: Do we want to store/cache one of the following in-cluster?
  • a) Public Keys, e.g. fetched from GitHub - to lower required API requests (e.g. cache keys that matched before, as they're more likely to be used again and only fetch new ones if that one didn't match)
    • A/a: 21.06.2023 (Craig): later
  • b) Signatures as created by acorn image sign (would allow for local-only signatures, e.g. I pull an image from DockerHub, where I cannot push my signature back, but I just sign it directly in-cluster using acorn image sign to allow it to be run)
    • A/b: 21.06.2023 (Craig): for now, we'll stick with the standard of pushing to remote registries only
  • We are not going to manage private keys in Manager
  • Potentially, we can implement "keyless" signing using ephemeral keypairs, where we use the private key once for signing and only record the public key, thus connecting the Hub identity to the signature
    • In that case, users must be enabled to manage their history of signatures/public keys
    • This does not require a CA or TL (like Rekor), meaning less management burden
    • For now we'll focus on client-side signing only with existing user keys

Checklist

• The title of this PR would make a good line in Acorn's Release Note's Changelog
• The title of this PR ends with a link to the main issue being address in parentheses, like: This is a title (#1216). Here's an example
• All relevant issues are referenced in the PR description. NOTE: don't use GitHub keywords that auto-close issues
• Commits follow contributing guidance
• Automated tests added to cover the changes. If tests couldn't be added, an explanation is provided in the Verification and Testing section
• Changes to user-facing functionality, API, CLI, and upgrade impacts are clearly called out in PR description
• PR has at least two approvals before merging (or a reasonable exception, like it's just a docs change)

[g-linville]

What about acorn pull --verify?, which would only pull if verification passed? In this case we could combine it with checking IARs in the target project as well.

If it's not too much work to implement, I think this would be nice for the user. I assume (haven't tested it yet) that acorn verify works with images in remote registries, but this would be a handy way to verify and pull with just one command.

I also have a PR out for acorn copy. Once this stuff is in, I might go add some signature verification options to that as well.

[g-linville]

One other question - what does acorn image sign do if the user does not specify the push option? I assume it is not pushing signatures into the internal registry, since the CLI has no direct access to the internal registry. It is it basically a dry run?

[iwilltry42]

One other question - what does acorn image sign do if the user does not specify the push option? I assume it is not pushing signatures into the internal registry, since the CLI has no direct access to the internal registry. It is it basically a dry run?

@g-linville Yeah it's basically a dry-run. We could think about outputting the signature artifact e.g. to the filesystem.

[iwilltry42]

What about acorn pull --verify?, which would only pull if verification passed? In this case we could combine it with checking IARs in the target project as well.

If it's not too much work to implement, I think this would be nice for the user. I assume (haven't tested it yet) that acorn verify works with images in remote registries, but this would be a handy way to verify and pull with just one command.

I also have a PR out for acorn copy. Once this stuff is in, I might go add some signature verification options to that as well.

@g-linville here you go:

$ acorn pull --verify --key gh://iwilltry42 'my.reg.io/public/acorn/hello-world:v1.0.0'
  •  Verifying Image my.reg.io/public/acorn/hello-world:v1.0.0 (digest: sha256:1a6c64d2ccd0bb035f9c8196d3bfe72a7fdbddc4530dfcb3ab2a0ab8afb57eeb) using key gh://iwilltry42
  ✔  Signature verified
 [112231571/112231571] ███████████████████████████████████████████████ 100% | 2s

$ acorn pull --verify --key gh://g-linville 'my.reg.io/public/acorn/hello-world:v1.0.0'
  •  Verifying Image my.reg.io/public/acorn/hello-world:v1.0.0 (digest: sha256:1a6c64d2ccd0bb035f9c8196d3bfe72a7fdbddc4530dfcb3ab2a0ab8afb57eeb) using key gh://g-linville
  ✗  ERROR:  NOT pulling image: failed to find valid signature for my.reg.io/public/acorn/hello-world@sha256:1a6c64d2ccd0bb035f9c8196d3bfe72a7fdbddc4530dfcb3ab2a0ab8afb57eeb matching given identity and annotations using 1 loaded verifiers/keys

This is not checking against IARs right now