Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

overlay profiles: Fix a couple of issues with SLSA provenance stuff #1157

Merged
merged 3 commits into from
Dec 12, 2023

Conversation

krnowak
Copy link
Member

@krnowak krnowak commented Sep 20, 2023

CI: http://jenkins.infra.kinvolk.io:8080/job/container/job/packages_all_arches/2543/cldsv

  • Update to slsa 1.0. This is only partially done, as we still need to provide a proper build type. Maybe we could reuse the Github Actions Workflow (https://slsa-framework.github.io/github-actions-buildtypes/workflow/v1).

  • Stop using portageq - its use in ebuilds is banned, so eventually it would stop working. Replace it with our hack.

  • Stop trying to get a commit hash of coreos-overlay or portage-stable as if they were submodules. This setup is long gone, so a commit hash of toplevel scripts repo is enough.

  • Use zstd for compressing generated JSON files.

--

  • Changelog entries added in the respective changelog/ directory (user-facing change, bug fix, security fix, update)
  • Inspected CI output for image differences: /boot and /usr size, packages, list files for any missing binaries, kernel modules, config files, kernel modules, etc.

@krnowak krnowak marked this pull request as ready for review September 22, 2023 14:33
@krnowak krnowak requested a review from a team September 22, 2023 14:33
@krnowak
Copy link
Member Author

krnowak commented Sep 22, 2023

CI passed.

@github-actions
Copy link

github-actions bot commented Sep 22, 2023

krnowak and others added 3 commits December 6, 2023 14:03
- Update to slsa 1.0. This is only partially done, as we still need to
  provide a proper build type. Maybe we could reuse the Github Actions
  Workflow
  (https://slsa-framework.github.io/github-actions-buildtypes/workflow/v1).

- Stop using portageq - its use in ebuilds is banned, so eventually it
  would stop working. Replace it with our hack.

- Stop trying to get a commit hash of coreos-overlay or portage-stable
  as if they were submodules. This setup is long gone, so a commit
  hash of toplevel scripts repo is enough.

- Use zstd for compressing generated JSON files.
SLSA provenance generation iterates over $A (which is a subset of $SRC_URI) and
for each of those tries to find a match in $SRC_URI. That's quadratic
complexity, and the performance impact is bad because we shell out to a helper
utility (basename) for every entry. This is leading to long stalls when
generating SLSA for packages with long distfile lists, like go and rust
packages. Iterate over SRC_URI once and create a dictionary to speed up
subsequent lookups. dev-db/etcdctl is a good candidate for testing.

Signed-off-by: Jeremi Piotrowski <[email protected]>
@krnowak
Copy link
Member Author

krnowak commented Dec 6, 2023

@krnowak
Copy link
Member Author

krnowak commented Dec 7, 2023

CI passed.

Copy link
Member

@jepio jepio left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

good enough. this stuff is really hard to review unless you instrument every line separately to investigate what the input looks like

@krnowak krnowak merged commit 1c3bc8b into main Dec 12, 2023
0 of 3 checks passed
@krnowak krnowak deleted the krnowak/slsa branch December 12, 2023 11:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants