Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Use S3 and GitHub Actions for yocto build cache #443

Merged
merged 7 commits into from
Dec 13, 2024
Merged

Use S3 and GitHub Actions for yocto build cache #443

merged 7 commits into from
Dec 13, 2024

Conversation

klutchell
Copy link
Contributor

@klutchell klutchell commented Nov 1, 2024 •
edited
Loading

This change makes the legacy NFS cache mount
optional, and will not fail the build if unset
or unreachable.

Instead, if the NFS is available it will be used
as a read-only mirror for shared downloads,
but will not be updated with new files.

This allows the per-device sstate to be ephemeral
and best effort, with automatic size and expiry limits
enforced by GitHub and scoped to the repository and branch.

See: https://balena.fibery.io/Work/Project/Secure-solutions-for-GHA-yocto-shared-cache-692
Supersedes: #351

@klutchell klutchell changed the title Kyle/dl mirror Use official aws cli binaries and commands Nov 1, 2024
@klutchell klutchell temporarily deployed to balena-staging.com November 1, 2024 18:36 — with GitHub Actions Inactive
@klutchell
Use S3 sstate cache on self-hosted runners
3cd0f50 
Change-type: patch
Signed-off-by: Kyle Harding <[email protected]>
@klutchell klutchell temporarily deployed to balena-staging.com December 11, 2024 17:51 — with GitHub Actions Inactive
@klutchell klutchell temporarily deployed to balena-staging.com December 11, 2024 20:21 — with GitHub Actions Inactive
@klutchell klutchell marked this pull request as ready for review December 12, 2024 14:55
@klutchell
Include shared-downloads in local sstate cache for now
de1756c 
In a future change we can re-enable the AWS S3 upload
and mirror of shared downloads using a new bucket.

At that time we can remove shared-downloads from the local
sstate cache.

Signed-off-by: Kyle Harding <[email protected]>
@klutchell klutchell temporarily deployed to balena-staging.com December 12, 2024 15:02 — with GitHub Actions Inactive
@flowzone-app flowzone-app bot enabled auto-merge December 12, 2024 15:07
@rcooke-warwick
Copy link
Contributor

again, LGTM

@klutchell
Copy link
Contributor Author

Yes, it works with the Generic x86_64 (legacy MBR) checks and completed in 31m, so it definitely used the cache.

I've started a new run of all workflows here: balena-os/balena-raspberrypi#1186

@flowzone-app flowzone-app bot merged commit 53afeaa into master Dec 13, 2024
62 checks passed
@flowzone-app flowzone-app bot deleted the kyle/dl-mirror branch December 13, 2024 11:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@rcooke-warwick rcooke-warwick rcooke-warwick approved these changes

@vipulgupta2048 vipulgupta2048 Awaiting requested review from vipulgupta2048

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@klutchell @rcooke-warwick