Multiple or per artifact tagPolicy's

[bryanlarsen]

Use Case

We're developing a service that creates Kubernetes Job's. Both the service and the job are under very active development.

Both the image for the service and the image for the job are listed in skaffold's "artifacts" section. Only the service artifact is referenced in the k8s manifests, the manifest for all jobs are dynamically generated by the service

If tagPolicy is set to gitCommit, then skaffold does a great job of keeping the service deployment up to date, rolling the deployment whenever any source changes.

If tagPolicy is set to envTemplate: "{{ IMAGE_NAME }}:latest", then skaffold does a great job of keeping the job image built. Whenever we ask the service to start a job, it starts the job running an up to date image.

The Ask

I can think of several different ways to solve my use case:

• allow multiple tagPolicy's to be set. If a user has set multiple tagPolicy's, apply multiple tags to the artifact. You'd may need some magic or an explicit flag to determine which tagPolicy to use in the manifests.
• additionally tag all artifacts with {{ IMAGE_NAME }}:latest to make it easier for us and for anybody other sorts of ad-hoc stuff. (perhaps behind a flag).
• allow tagPolicy to be set per artifact as well as per profile
• some sort of mechanism so that the service can determine what tag skaffold has created for the images it builds

Information

• Skaffold version: version 0.18.0
• Operating system: Ubuntu 18.04
• Contents of skaffold.yaml:

apiVersion: skaffold/v1alpha5
kind: Config
build:
  artifacts:
    - image: app
      docker:
        dockerfile: app/Dockerfile
    - image: files
      docker:
        dockerfile: files/Dockerfile
    - image: projects
      docker:
        dockerfile: projects/Dockerfile
    - image: users
      docker:
        dockerfile: users/Dockerfile
    - image: worker
      docker:
        dockerfile: worker/Dockerfile

profiles:
  - name: dev-shared
    build:
      artifacts:
        - image: dev
          docker:
            dockerfile: docker/dev-shared/Dockerfile

    deploy:
      kustomize:
        path: skaffold/dev-shared

  - name: pr
    build:
      tagPolicy:
        envTemplate:
          template: '{{.DOCKER_REGISTRY}}/threekit-com/{{.IMAGE_NAME}}:{{.VERSION}}'
    deploy:
      kustomize:
        path: skaffold/pr

  - name: dev-full
    deploy:
      kustomize:
        path: skaffold/dev-full

[nkubala]

Having multiple tags per artifact I think would be confusing, because Skaffold would have to choose which one to use in the manifests. I don't necessarily hate having a per-artifact tagger though: if no tag policy is set for an artifact, it could fall back to the top-level tag policy in the build stanza, and of course if that one isn't set we'd use the default.

[bryanlarsen]

@nkubala you already appear to generate multiple tags per artifact. For example, my latest container got tagged with both "a593b5ad6a6362a5cc19907a7a79ec14:latest" and "dev:20cea37-dirty-a2b9536". I'm not sure what the former tag is and why you've created that tag, but it's obviously not causing confusion in your codebase.

But as you mentioned, a per-artifact tagger is probably cleaner.

[gottfrois]

I would also be interested in tag policy per artifacts. Our use case is using multiple docker stages, I would like to tag each stages different to pull them later on new builds to benefit from them as cache:

apiVersion: skaffold/v1beta2
kind: Config
build:
  artifacts:
  - image: gottfrois/ruby-app-example
    docker:
      target: bundler
      cacheFrom:
      - gottfrois/ruby-app-example:bundler-{{.BRANCH}}
      - gottfrois/ruby-app-example:bundler
  - image: gottfrois/ruby-app-example
    docker:
      target: runner
      cacheFrom:
      - gottfrois/ruby-app-example:bundler-{{.BRANCH}}
  - image: gottfrois/ruby-app-example
    docker:
      target: release
      buildArgs:
        env: "production"
        revision: "{{.REVISION}}"
        branch: "{{.BRANCH}}"
      cacheFrom:
      - gottfrois/ruby-app-example:bundler-{{.BRANCH}}
      - gottfrois/ruby-app-example:runner-{{.BRANCH}}
      - gottfrois/ruby-app-example:{{.REVISION}}

I would like to create a tag with the branch env variable to be pulled on next build and used as cache to speed up next docker build.
My Dockerfile uses 3 different docker stages bundler, runner and release which is the final stage.
It would be great to be able to tag each stage accordingly and push them under the same repo.

[torstenwalter]

If multiple tags per image would be possible one could have a release provile which adds more specific tags and a "latest" tag at the same time.

That's quite common for images to have multiple tags. For example the tags for debian currently all point to the same image:

• jessie
• jessie-20190122
• 8.11
• 8

I understand that this cause trouble, because you have to decide which tag to use for deployment. If multiple tags are present one could be marked as default. If none has the default flag then the build could fail.

[siwyd]

I would find it very handy to be able to push multiple tags: the git commit, a sha256 tag and a tag with the branch name (I can get that from BRANCH_NAME in Jenkins for instance). That makes it easy to locate an artifact in a registry in a multitude of ways. When it comes to deploying, personally I always want the sha256 to be used so I'm 100% sure of which artifact is actually up and running at any given moment. So to me it makes sense to always use sha256 as the deployment tag.

[balopat]

Thank you for sharing. I think this is reasonable and we could start looking into it in the next coming quarter.

[rhyas]

Thank you for sharing. I think this is reasonable and we could start looking into it in the next coming quarter.

It's been over a year now since this comment, and well beyond it's milestone release, yet it's still open. Is this ever going to be a thing? Or maybe some other less documented way of doing multiple tags been implemented that should be referenced here?

[nkubala]

@rhyas this one is in our backlog milestone right now, which is our (not so clear) way of signaling that this issue is valid and would be a great addition, but we don't have the bandwidth to prioritize it right now.

I think this would be possible with our new multiple config feature which just shipped with v1.18.0 - you could create a config stanza for each artifact, each with its own tag policy. (they can even live in the same skaffold.yaml file, separated with a ---)

contributions for features like this are of course always welcome though!

[tejal29]

V3 schema has a proposal to specify additional taggers for an artifact. Lets defer this until then,

[tejal29]

#TODO @tejal29 to create a epic and add this in

[tejal29]

Relates to #5857

[afirth]

tagging latest would make cacheFrom much easier I think? Maybe I'm missing something obvious like parsing git log, but I haven't found a simple way of using a docker cache with the default tagger.

e.g. #4842 (comment)

[briandealwis]

So I'm in this situation right now with the container-debug-support images. I currently stage images to a staging repository (us-central1-docker.pkg.dev/k8s-skaffold/skaffold-staging/skaffold-debug-support) as latest. As we automate this, I want to instead push the images with an unique tag (e.g., the commit SHA), run tests, and then tag those images as latest. Once we've done some further validation, these images would be promoted to their final destination (gcr.io/k8s-skaffold/skaffold-debug-support) with the commit-SHA based tags and then as latest.

I'd been thinking it would be useful to leverage Skafold's artifacts file (skaffold build --file-output artifacts.json) to copy images with a desired tag, like

# copy the images to the new repository but keeping the same tags
skaffold copy --build-artifacts artifacts.json gcr.io/k8s-skaffold/skaffold-debug-support 

# copy the images but with a new tag of `latest`
skaffold copy --build-artifacts artifacts.json --tag latest gcr.io/k8s-skaffold/skaffold-debug-support

[rhysjtevans]

Any progress with this? I find myself with a git monorepo situation using lerna whereby the git repo consists of multiple tags and the git tag you are on isn't necessarily the tag you want to use for all artifacts as others have suggested above. E.g.

I may have mutliple packages in the git repo.

packages/package-a
packages/package-b

I then would want something like ...

artifacts:
    - image: dockerhub/package-a
      tagPolicy:
        envTemplate:
          template: "{{ .PACKAGE_A_TAG }}"
    - image: dockerhub/package-b
      tagPolicy:
        envTemplate:
          template: "{{ .PACKAGE_B_TAG }}"

Unless there's a way or a better way to achieve this now? Currently experimenting with envTemplates to attempt to achieve the above leveraging the IMAGE_NAME built in var.

[ericzzzzzzz]

Sorry, This request has been lowered in priority due to our understaffing.

[hedinasr]

any news on this topic?

[ZTaylor97]

This feature would save a lot of headaches on the project I'm working on, currently we have to run docker commands as a post build hook in order to do multiple tags