Running CI on PR's - Permissions

[darkxst]

I took a brief look at this, and while tagging the container images with PR number is straight forward getting the correct permissions is a bit more challenging!

When running a workflow on an external PR, all write permissions are stripped from the GITHUB_TOKEN, thus no way to write the container image. The recommended way around this is to create a PAT with write:packages permissions but this also gets pretty broad access to the repo, that could easily be abused with a malicious PR.

So if to go the PAT approach probably want to enforce manual approval for all PR workflow runs (after code review).

@agners does this approach work for you?

Other options maybe:

• skip container build when running a PR (obviously missing any changes to the Dockerfile)
• offload the container image somewhere else unrelated to the repo (perhaps a dummy account on dockerhub?)

[agners]

So if to go the PAT approach probably want to enforce manual approval for all PR workflow runs (after code review).

I think I could life with that.

skip container build when running a PR (obviously missing any changes to the Dockerfile)

I don't really like this option.

offload the container image somewhere else unrelated to the repo (perhaps a dummy account on dockerhub?)

I think I'd try with that approach. I guess the containers don't need to be stored forever. I've used ttl.sh for ephemeral container images in the past.

[darkxst]

I've used ttl.sh for ephemeral container images in the past.

I tried this, but images maybe to big for this service. Having trouble pushing the images there...

[agners]

Hm, seems our image is huuge 😰

Hm, it doesn't seem like ttl.sh should have a size limit, but maybe we run into what is described in replicatedhq/ttl.sh#104 (comment) ? 🤔

Anyhow, the pushing the image fails for me:

$ docker push ttl.sh/silabs-firmware-builder:30m
The push refers to repository [ttl.sh/silabs-firmware-builder]
9b403a968db7: Layer already exists 
7cecbb46ee85: Layer already exists 
64c8c0830bc0: Layer already exists 
fdfdec5d9acf: Pushing [==================================================>]  6.365GB/6.365GB
a2c57b563152: Layer already exists 
61d9f0759d8e: Layer already exists 
5411d7fa654f: Layer already exists 
d79a3420a33b: Layer already exists 
f370a05d981b: Layer already exists 
received unexpected HTTP status: 500 Internal Server Error

Maybe we should only include tooling and not the whole SDK? But then, we need to download the SDK for every build. I guess it is container image download vs git lfs download... What is better/easier? 🤔

[darkxst]

Yep and its all Gecko SDK gobbling up that space!

yes buildx push hits that cloudflare issue straight up per that issue, however even docker cli push get stuck on retrying the gecko SDK layer and eventually fails with same 500 error you saw.

Cloning the SDK is a very slow operation, takes about 6-7mins (on a github runner) just for that step when no cache is involved. So its probably better to have it there in the container, with the sdk layer mostly pulled from cache for the duration of the release cycle... Not sure if this applies though when the containers are hosted off github infrastructure, may make less of a difference? I do run quite a few more builds in my fork though!

Probably the easiest thing will be to go with my original idea, create a PAT token and manually approve all PR workflows!

[agners]

#32 brings that layer down to 3.6GB, not sure if it is enough.

Probably the easiest thing will be to go with my original idea, create a PAT token and manually approve all PR workflows!

Yeah I guess.

[darkxst]

brings that layer down to 3.6GB, not sure if it is enough.

still fails to push to ttl.sh with same 500 errors

[darkxst]

I had another thought on this, we could probably just pass the container through as an artifact to the firmware jobs. That should bypass the size issues (and permission issues) for PR builds.

[darkxst]

we could probably just pass the container through as an artifact

This is completely horrible performance wise also.

[github-actions]

There hasn't been any activity on this issue recently, so we clean up some of the older and inactive issues.
Please make sure to update to the latest version and check if that solves the issue. Let us know if that works for you by leaving a comment 👍
This issue has now been marked as stale and will be closed if no further activity occurs. Thank you for your contributions.

[darkxst]

not stale, still need to clean up patches though

[github-actions]

There hasn't been any activity on this issue recently, so we clean up some of the older and inactive issues.
Please make sure to update to the latest version and check if that solves the issue. Let us know if that works for you by leaving a comment 👍
This issue has now been marked as stale and will be closed if no further activity occurs. Thank you for your contributions.

[puddly]

I've taken a slightly different approach with #54 that I think should work.

It will only build a container if it does not already exist under both the original repo and the the fork. This should allow you to make firmware code PRs (since they will reuse the old container) and also make PRs to edit the Dockerfile (since the container will have been built under your account).

[puddly]

Implemented in #54.