feat(ci): add slsa provenance for release artifacts #263

behnazh-w · 2022-09-07T07:06:44Z

The Open Source Security Foundation has been developing a standard for supply chain security called SLSA to allow users of open source projects to verify the artifacts and their build pipelines. To make that possible, the release pipelines should generate a provenance that provides verifiable information about how the software artifacts are built.

This PR adds a reusable GitHub Actions workflow to generate the provenance and publish that together with the artifacts as GitHub release artifacts. This would help users verify that the release artifacts are built from the release.yml workflow and not somewhere else. You can read more about this provenance generator here.

Looking forward to the feedback and discussions.

CLAassistant · 2022-09-07T07:06:49Z

All committers have signed the CLA.

melix · 2022-09-08T09:12:07Z

.github/workflows/release.yml

-          ./gradlew publishToSonatype closeAndReleaseSonatypeStagingRepository
+          # Publish both locally and to Sonatype.
+          # The artifacts stored locally will be used to generate the SLSA provenance.
+          ./gradlew publishToMavenLocal publishToSonatype closeAndReleaseSonatypeStagingRepository


Instead of publishing to maven local, which may (will) be polluted by many unrelated artifacts, you can use publishToBuildRepository. This will create a Maven-like repository in $ROOT/build/repo which will contain all the artifacts you want (plus the metadata files, pom, Gradle .module, version catalog, etc.).

Instead of publishing to maven local, which may (will) be polluted by many unrelated artifacts, you can use publishToBuildRepository.

Thanks @melix. When I use publishToBuildRepository locally, it tries to authenticate with ge.micronaut.io:

A build scan was not published as you have not authenticated with server 'ge.micronaut.io'. * What went wrong: For more information, please see https://gradle.com/help/gradle-authenticating-with-gradle-enterprise. Task 'publishToBuildRepository' not found in root project 'micronaut'.

Is there a way to use it locally for testing without authenticating?

Ah sorry, this error is unrelated. It says that it won't publish a build scan on ge.micronaut.io, but this isn't the cause of the problem, it's because I used the wrong task name, it's publishAllPublicationsToBuildRepository (you can also use pAPTBR).

Thanks @melix . Changed in commit bf15270.

melix · 2022-09-08T09:14:12Z

.github/workflows/release.yml

+          # The artifacts stored locally will be used to generate the SLSA provenance.
+          ./gradlew publishToMavenLocal publishToSonatype closeAndReleaseSonatypeStagingRepository
+          # Read the current version from gradle.properties.
+          VERSION=$(./gradlew properties | grep 'version:' | awk '{print $2}')


I may be wrong, but I think that if you use the publishToBuildRepository task, then you don't need to search for the group or version anymore, and you can use the build/repo directory as the upload source for GH.

I still need to read the group because I need to resolve the path to the artifacts for uploading them as artifacts. The path can be different in different repositories that use this template.

I read the version just to make sure the artifacts are collected from the right path, but that is just to be extra cautious.

behnazh-w · 2022-09-09T05:23:19Z

The final release artifacts would look something like this on micronaut-core for example:

It adds the provenance as attestation.intoto.jsonl and separate zip files for each module's artifacts. The user would need to unzip and verify the provenance against the unzipped artifacts.

@sdelamo @melix please let me know if it looks good or you have any questions.

alvarosanchez · 2022-09-09T07:04:17Z

IMO that would clutter the release assets section. Is it possible to package all the ZIPs into a single archive?

melix · 2022-09-09T12:46:20Z

.github/workflows/release.yml

+        # the provenance will attest to the artifact files and not the aggregated zip file.
+        run: |
+          ARTIFACTS=""
+          for dir in dist/* ; do


I seems a bit strange to look for the artifacts in build/repo, but then use those in dist for uploading. I think we should use build/repo for both. Also I agree with @alvarosanchez that it would be nice if we had a single zip, with possibly nested zips in it.

If it helps we could even have a Gradle task to do it.

I think we should use build/repo for both.

Sure, I'll change that.

behnazh-w · 2022-09-11T04:33:07Z

@alvarosanchez

IMO that would clutter the release assets section. Is it possible to package all the ZIPs into a single archive?

@melix

Also I agree with @alvarosanchez that it would be nice if we had a single zip, with possibly nested zips in it.

I can definitely wrap all ZIP files in a single archive. But I thought having separate archives immediately shows to the end user that for example micronaut-core is generating 35 artifacts. So it's more transparent. Do you see any value in that?

melix · 2022-09-12T06:57:54Z

I can definitely wrap all ZIP files in a single archive. But I thought having separate archives immediately shows to the end user that for example micronaut-core is generating 35 artifacts. So it's more transparent. Do you see any value in that?

I don't think users really care. In practice, you don't select each artifact separately, you'll likely depend on micronaut-core, which is going to bring a number of other artifacts transitively. Then, when you bundle the app, you can take a look at the artifacts you got and search for provenance. Ideally we should end up with tooling (e.g Gradle) doing this for us, once there's a standard way to get this information for each artifact.

behnazh-w · 2022-09-15T10:49:37Z

I can definitely wrap all ZIP files in a single archive. But I thought having separate archives immediately shows to the end user that for example micronaut-core is generating 35 artifacts. So it's more transparent. Do you see any value in that?

I don't think users really care. In practice, you don't select each artifact separately, you'll likely depend on micronaut-core, which is going to bring a number of other artifacts transitively. Then, when you bundle the app, you can take a look at the artifacts you got and search for provenance. Ideally we should end up with tooling (e.g Gradle) doing this for us, once there's a standard way to get this information for each artifact.

@melix @alvarosanchez I've packaged all the artifacts into a single ZIP file. See commit 8938a2b.

.github/workflows/release.yml

Due to an [ongoing issue](https://github.com/orgs/community/discussions/37942), Provenance regularly fails when we are publishing a release. We will remove it for now. Revert "feat(ci): add slsa provenance for release artifacts (#263)" This reverts commit 608ba93.

melix requested changes Sep 8, 2022

View reviewed changes

feat(ci): add slsa provenance for release artifacts

6c87b6b

behnazh-w force-pushed the add-slsa-provenance branch from 83a0a49 to bf15270 Compare September 9, 2022 05:07

melix reviewed Sep 9, 2022

View reviewed changes

behnazh-w added 2 commits September 15, 2022 20:32

chore: collect artifacts from local build repository

29ffa13

chore: package all the artifacts into a single ZIP file

8938a2b

behnazh-w force-pushed the add-slsa-provenance branch from bf15270 to 8938a2b Compare September 15, 2022 10:46

alvarosanchez requested a review from melix September 21, 2022 10:02

melix requested changes Sep 21, 2022

View reviewed changes

.github/workflows/release.yml Outdated Show resolved Hide resolved

chore: add .module and .toml files to artifacts

9754e69

melix approved these changes Sep 22, 2022

View reviewed changes

melix merged commit 608ba93 into micronaut-projects:master Sep 28, 2022

timyates mentioned this pull request Nov 3, 2022

Remove SLSA Provenance #287

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat(ci): add slsa provenance for release artifacts #263

feat(ci): add slsa provenance for release artifacts #263

behnazh-w commented Sep 7, 2022

CLAassistant commented Sep 7, 2022 •

edited

Loading

melix Sep 8, 2022

behnazh-w Sep 8, 2022 •

edited

Loading

melix Sep 8, 2022

behnazh-w Sep 9, 2022 •

edited

Loading

melix Sep 8, 2022

behnazh-w Sep 9, 2022

behnazh-w commented Sep 9, 2022

alvarosanchez commented Sep 9, 2022

melix Sep 9, 2022

behnazh-w Sep 11, 2022

behnazh-w commented Sep 11, 2022

melix commented Sep 12, 2022

behnazh-w commented Sep 15, 2022 •

edited

Loading

feat(ci): add slsa provenance for release artifacts #263

feat(ci): add slsa provenance for release artifacts #263

Conversation

behnazh-w commented Sep 7, 2022

CLAassistant commented Sep 7, 2022 • edited Loading

melix Sep 8, 2022

Choose a reason for hiding this comment

behnazh-w Sep 8, 2022 • edited Loading

Choose a reason for hiding this comment

melix Sep 8, 2022

Choose a reason for hiding this comment

behnazh-w Sep 9, 2022 • edited Loading

Choose a reason for hiding this comment

melix Sep 8, 2022

Choose a reason for hiding this comment

behnazh-w Sep 9, 2022

Choose a reason for hiding this comment

behnazh-w commented Sep 9, 2022

alvarosanchez commented Sep 9, 2022

melix Sep 9, 2022

Choose a reason for hiding this comment

behnazh-w Sep 11, 2022

Choose a reason for hiding this comment

behnazh-w commented Sep 11, 2022

melix commented Sep 12, 2022

behnazh-w commented Sep 15, 2022 • edited Loading

CLAassistant commented Sep 7, 2022 •

edited

Loading

behnazh-w Sep 8, 2022 •

edited

Loading

behnazh-w Sep 9, 2022 •

edited

Loading

behnazh-w commented Sep 15, 2022 •

edited

Loading