Feature: New check for SLSA provenance generation

[laurentsimon]

This check may replace the Signed-Release checks and verify that the releases generate SLSA provenance using the official builders from https://github.com/slsa-framework/slsa. We can:

1. Verify the presence of .intoto.jsonl file
2. Verify a workflow exists calling an official action
3. Verify the signature by calling the API exposed by https://github.com/slsa-framework/slsa

This should allow us to also provide some SLSA compliance flags for scorecard, e.g. something like --slsa=3

[azeemshaikh38]

Sounds like a good idea to me. Maybe expand the Signed-Release check to include this? The updated check could verify various methods which ensure release integrity - GitHub signatures, cosign signatures, the SLSA workflow etc?

[laurentsimon]

Provenance encompasses all the other signature schemes, since it contains a signature not only of the binary, but also of how the binary was produced ,so I'm inclined to retire the Signed-Release check to encourage users to use provenance instead - since it's a superset of all the others.

Anyone with an opinion, please chime in

[naveensrinivasan]

This is great! 👍

[azeemshaikh38]

Sure, not tied to the Signed-Release name at all. To confirm are you saying that if users sign their release (let's say with cosign), we should complain that it's not enough and instead drive them towards building provenance?

I was thinking the new check will encompass all schemes - provenance but also signatures.

[laurentsimon]

Sure, not tied to the Signed-Release name at all. To confirm are you saying that if users sign their release (let's say with cosign), we should complain that it's not enough and instead drive them towards building provenance?

yes, so long as there is tooling to achieve it.

I was thinking the new check will encompass all schemes - provenance but also signatures.

provenance also contains a signature that covers the binary and additional information (builder ID, commands, etc), so it does encompasses the signature.

[sethmlarson]

Chiming in to say I'd like to start using SLSA with urllib3 but losing out on the Signed-Releases check we're currently getting from Sigstore would be unfortunate by changing to SLSA.

Could we can get Scorecard to recognize *.intoto.jsonl the same way it currently looks for *.sig on a GitHub Release? I don't know if Scorecard is verifying the signatures we're currently publishing as the info only mentions the existence of a *.sig file so if we're going off presence of a file that'd be an easier first step than implementing full end-to-end verification?

[laurentsimon]

We're not verifying signatures at all, because it's hard t find out the correct pubic key in practice. So we can definitely look for an .intoto.jsonl.
Maybe it's enough to add this "check" within the existing Signed-Release. Wdut?

[sethmlarson]

I think that makes sense, especially as a means to convince people to start using SLSA.

[laurentsimon]

great. Anyone interested in sending a PR for this? This is the file to update https://github.com/ossf/scorecard/blob/main/checks/evaluation/signed_releases.go#L51, and we can add a unit test in https://github.com/ossf/scorecard/blob/main/checks/signed_releases_test.go

[github-actions]

This issue is stale because it has been open for 60 days with no activity.

[github-actions]

This issue is stale because it has been open for 60 days with no activity.