Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Publish Multi-Arch images using github actions #31

Merged
merged 59 commits into from
Aug 9, 2022

Conversation

wickywaka
Copy link
Contributor

I manage to publish multi-arch images using GitHub Actions and here is the merge request. There are still open questions and I was doing it while learning at the same time so please suggest what can be improved.

It uses GitHub secrets for docker hub username and token with following names DOCKER_HUB_USERNAME, DOCKER_HUB_TOKEN. Please the docker.yml file for more details.

Related Issue: #23

@wickywaka wickywaka changed the title Publish Multi-Arch images using github actions Draft: Publish Multi-Arch images using github actions Jun 1, 2022
@robinsmidsrod
Copy link
Contributor

@waqarrashid33 This is a great start! Now comes the comments. ;)

  1. Any reason why the build environment uses ubuntu 18.04 instead of the newer 20.04 or 22.04? Are the steps not compatible with the newest OS versions?
  2. I noticed you're just directly pushing the :latest tag. Is there any way to push a specific version, or push a build-artifact named after the digest? I'd like to be able to push dev versions of the images so it is easier to collaborate on testing before doing a new release.
  3. I found the secrets setup in Settings, so that part should be okay, but when does this workflow trigger? I'm assuming it triggers when code enters the master branch. How would this work for numbered releases. How about recurring rebuilds when the base ubuntu image updates?
  4. Just curious: Do you need QEMU when making an x86/x64 image? Is this just the best way to create multiarch images?

As you can probably notice, I'm not familiar with GitHub Actions, hence all the questions. I have worked some with Bitbucket Pipelines, so CI/CD is not completely unfamiliar to me.

@wickywaka
Copy link
Contributor Author

wickywaka commented Jun 1, 2022 via email

@wickywaka
Copy link
Contributor Author

Hi @robinsmidsrod ,
I haven't given it much thought consider this as an MVP. Here I will post my answers to your questions/comments again (somehow email reply got lost).

  1. Any reason why the build environment uses ubuntu 18.04 instead of the newer 20.04 or 22.04? Are the steps not compatible with the newest OS versions?

No reason, I think 20.04 should be fine. 22.04 is too new for my taste.

  1. I noticed you're just directly pushing the :latest tag. Is there any way to push a specific version, or push a build-artifact named after the digest? I'd like to be able to push dev versions of the images so it is easier to collaborate on testing before doing a new release.

From what I have read about GitHub Actions, we can change the trigger when make/push a tag in GitHub. It can also be so that it takes the tag as tag for docker image e.g docker-dhcpd:3.2 etc. There are lots of possibilities in this area. This is my first time using GitHub Actions. So definitely some work to do here.

  1. I found the secrets setup in Settings, so that part should be okay, but when does this workflow trigger? I'm assuming it triggers when code enters the master branch. How would this work for numbered releases. How about recurring rebuilds when the base ubuntu image updates?
  1. Good question, I don't know how to update when base image changes or what would be a good strategy.
  1. Just curious: Do you need QEMU when making an x86/x64 image? Is this just the best way to create multiarch images?

I think it uses qemu only for building images whose architecture is different than host. On my system the amd64 image was always finishing its build first because it was probably running natively.

As you can probably notice, I'm not familiar with GitHub Actions, hence all the questions. I have worked some with Bitbucket Pipelines, so CI/CD is not completely unfamiliar to me.

I am new to this as well but we can learn from each other.

@katbyte
Copy link

katbyte commented Jun 5, 2022

found this while trying to stand up docker dns & dhcp on a rpi, look forward to this landing on docker hub!

what you could do is have 1 job that runs on every commit and pushes to latest, and then another that runs on version tags:

on:
  push:
    tags:
      - 'v[0-9]+.[0-9]+.[0-9]+*'

and then grab the tag with ${{ steps.vars.outputs.tag }} or ${{ env.RELEASE_VERSION }} and pop that into the docker tag name (and even do multiple base versions of the os like here https://hub.docker.com/r/ubuntu/bind9/tags)

@yscialom
Copy link
Contributor

yscialom commented Jul 5, 2022

Hello, and thank your for this PR.

Tagging the result of master as :latest is a good idea. Additionally, tagging the version number would be a perfect solution.

This would require to run the workflow on git tags (1) and to extract the git's tag name to construct the docker's tag name (2).


Example for (1)

on:
  push:
    tags:
      - *

See Github's workflow syntax documentation.


Example for (2)

    steps:
      - uses: olegtarasov/[email protected]
        id: tagName
        with:
          tagRegex: "foobar-(.*)"  # Optional. Returns specified group text as tag name. Full tag string is returned if regex is not defined.
          tagRegexGroup: 1 # Optional. Default is 1.
      - name: Some other step # Output usage example
        with:
          tagname: ${{ steps.tagName.outputs.tag }}
      - name: Yet another step # Environment variabl usage example
        run: |
          docker build . --file Dockerfile --tag docker.pkg.github.com/someimage:$GIT_TAG_NAME

See actions/get-tag-name documentation.

Additionally2, it would be nice if the publishing of git tag x.y.z would publish the docker tags :x :x.y and :x.y.z.

@wickywaka
Copy link
Contributor Author

Hey guys,
Thanks for the suggestions. I will try to add them to the PR as soon as I can.

@yscialom
Copy link
Contributor

yscialom commented Jul 6, 2022

@wickywaka
Hello. I'd be happy to see this PR merged in the near future. How can I help?

@wickywaka
Copy link
Contributor Author

@wickywaka Hello. I'd be happy to see this PR merged in the near future. How can I help?

@yscialom Lets see if I can finish it up today. Anyway I have added you as collaborator to the fork so that you can make the desired changes into the merge request.

@wickywaka wickywaka changed the title Draft: Publish Multi-Arch images using github actions Publish Multi-Arch images using github actions Jul 26, 2022
@robinsmidsrod
Copy link
Contributor

robinsmidsrod commented Jul 27, 2022

@wickywaka Great work! This is coming along nicely! Good job on getting the semver version parsed correctly. I do notice two things that should probably be fixed before we merge it.

  1. Can you add a step to build and push the Dockerfile.ldap to both files as well? It would be unfortunate if that is not built with automation when the rest is. Tag it as :ldap-latest and :ldap-X.Y.Z.
  2. Since you were able to add semver parsing, whould it be possible to also build tags :1 (major semver version number) and :ldap-1? This is for people that want to use the this tag in their scenarios and avoid possibly breaking changes (that we would know about).

@wickywaka
Copy link
Contributor Author

@wickywaka Great work! This is coming along nicely! Good job on getting the semver version parsed correctly. I do notice two things that should probably be fixed before we merge it.

  1. Can you add a step to build and push the Dockerfile.ldap to both files as well? It would be unfortunate if that is not built with automation when the rest is. Tag it as :ldap-latest and :ldap-X.Y.Z.
  2. Since you were able to add semver parsing, whould it be possible to also build tags :1 (major semver version number) and :ldap-1? This is for people that want to use the this tag in their scenarios and avoid possibly breaking changes (that we would know about).
  1. I don't know much about the ldap stuff. I don't even know what it is used for, so I can use some pointers here.
  2. Tagging won't be a problem. The question I have here: Do we tag :1 with the latest one in this major release. For example: If :1 points to :1.1.1 but a new tag was pushed with 1.2.1, then we would have to update the tag :1 to :1.2.1. If the behavior you want is a simple one like if we push a tag 1.1.2 then we make three tags :1, :1.1 and :1.1.2, overwriting the previous :1 and :1.1. Is it desired behavior? I hope I was able to explain myself 😄

@robinsmidsrod
Copy link
Contributor

  1. The LDAP stuff is just another Dockerfile, so you just need to figure out how to specify something else than the default Dockerfile to the build-push step. I'm going to assume that is an option in there.
  2. I thought about pushing both :X, :X.Y and :X.Y.Z tags, but from a semver perspective it is not needed, because only major versions should introduce breaking changes. So we only need :latest, :X, and :X.Y.Z and similar with the ldap- prefix. And yes, when we tag a new version that should overwrite the :X tag on Dockerhub pointing to the last released of that major version.

@wickywaka
Copy link
Contributor Author

  1. The LDAP stuff is just another Dockerfile, so you just need to figure out how to specify something else than the default Dockerfile to the build-push step. I'm going to assume that is an option in there.

Can you check if this is what you expected:
docker pull wickywaka/docker-dhcpd:ldap-latest

@wickywaka
Copy link
Contributor Author

  1. I thought about pushing both :X, :X.Y and :X.Y.Z tags, but from a semver perspective it is not needed, because only major versions should introduce breaking changes. So we only need :latest, :X, and :X.Y.Z and similar with the ldap- prefix. And yes, when we tag a new version that should overwrite the :X tag on Dockerhub pointing to the last released of that major version.

I think its ready as well.

@wickywaka wickywaka requested a review from robinsmidsrod August 3, 2022 12:33
@robinsmidsrod robinsmidsrod merged commit ab18014 into networkboot:master Aug 9, 2022
@yscialom
Copy link
Contributor

yscialom commented Aug 9, 2022

Congrats!
image

@wickywaka
Copy link
Contributor Author

wickywaka commented Oct 11, 2022 via email

@wickywaka
Copy link
Contributor Author

wickywaka commented Oct 11, 2022 via email

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants