Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Arch specific dependencies #272

Merged
merged 2 commits into from
Jan 7, 2025
Merged

Arch specific dependencies #272

merged 2 commits into from
Jan 7, 2025

Conversation

joshwlewis
Copy link
Member

@joshwlewis joshwlewis commented Dec 17, 2024

When running pack buildpack package with a buildpack that has a docker dependency on a multi-arch image index like this:

[dependencies]
uri = "docker://heroku/buildpack-release-phase:1.0.3"

Pack defers the image resolution to docker. This works fine when packaging a buildpack for the host platform, but breaks when packaging for another architecture. The default docker image store (which pack uses) is not able to properly handle cross platform images. The error manifests like this:

ERROR: packaging dependencies (uri='docker://heroku/buildpack-release-phase:1.0.3',image=''): extracting from registry 'docker://heroku/buildpack-release-phase:1.0.3': extracting buildpacks from 'heroku/buildpack-release-phase:1.0.3': could not find label 'io.buildpacks.buildpackage.metadata'

This PR does a basic regex substitution to append the appropriate platform suffix, so the package.toml references the docker tag for the appropriate platform like this:

[dependencies]
uri = "docker://heroku/buildpack-release-phase:1.0.3_linux-arm64"

or

[dependencies]
uri = "docker://heroku/buildpack-release-phase:1.0.3_linux-amd64"

An example passing workflow that did not without this change: https://github.com/heroku/buildpacks-frontend-web/actions/runs/12381045454/job/34558845731

@joshwlewis joshwlewis marked this pull request as ready for review December 17, 2024 21:02
@joshwlewis joshwlewis requested a review from a team as a code owner December 17, 2024 21:02
Copy link
Member

@mars mars left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Confirmed, this fixes the release publishing issue with the Front-end Web buildpacks.

Copy link
Member

@edmorley edmorley left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for digging into this!

I'm presuming the reason we haven't run into this before, is that all of our other composite buildpacks have libcnb: type dependencies rather than docker image dependencies in their package.toml?

Re Pack deferring platform resolution to Docker, does that mean there is a feature-gap in Pack's multi-arch support (eg it should propagate --platforms or similar but doesn't), or is it a case of us needing to migrate the automation here (which was written before Pack supported multi-arch properly) to the new Pack feature? If the former it would be good to file a bug/feature request upstream, and if the latter then I think we should create some GUS work items/epics for tracking refactoring multi-arch in FY26 (incl ideally pushing some of the handling down into libcnb-{cargo,package}.

@joshwlewis
Copy link
Member Author

joshwlewis commented Jan 7, 2025

I'm presuming the reason we haven't run into this before, is that all of our other composite buildpacks have libcnb: type dependencies rather than docker image dependencies in their package.toml?

Yeah, indeed.

Re Pack deferring platform resolution to Docker, does that mean there is a feature-gap in Pack's multi-arch support (eg it should propagate --platforms or similar but doesn't), or is it a case of us needing to migrate the automation here (which was written before Pack supported multi-arch properly) to the new Pack feature?

There might be a gap in packaging cross-platform (not multi-platform) buildpacks with pack. I'm not sure if that's a supported workflow, though. It seems like pack supports building same-platform buildpacks OR multi-platform buildpacks, but we're doing something in between that involves an occasional cross-platform buildpack.
I do think we need to do the migration though. I've filed an epic and some basic work: https://gus.lightning.force.com/lightning/r/ADM_Epic__c/a3QEE000001SLSL2A4/view

@joshwlewis joshwlewis merged commit 9b1d7d0 into main Jan 7, 2025
2 checks passed
@joshwlewis joshwlewis deleted the jwl/arch-specific-deps branch January 7, 2025 18:08
@edmorley
Copy link
Member

edmorley commented Jan 8, 2025

Released in:
https://github.com/heroku/languages-github-actions/releases/tag/v0.9.0

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants