Changing 'imagePullPolicy' of all containers in all deployments #1493

matti · 2019-09-02T13:34:47Z

originally asked here #412 - but the question is still left unanswered:

following kustomization

patches:
  - path: imagepullpolicytoalways.yaml
    target:
      kind: Deployment

and

- op: replace
  path: "/spec/template/spec/containers/0/imagePullPolicy"
  value: Always

changes/adds the imagePullPolicy to first container, but how to set it to all containers? using *does not work.

The text was updated successfully, but these errors were encountered:

matti · 2019-09-02T13:37:48Z

And I can't use AlwaysPullImages AdmissionController in GKE

matti · 2019-09-02T13:40:01Z

- op: replace
  path: "/spec/template/spec/containers[]/imagePullPolicy"
  value: Always

results in doc is missing path: /spec/template/spec/containers[]/imagePullPolicy: missing value

matti · 2019-09-02T13:55:37Z

workaround:

patches:
  - path: jsonpatches/first-container-pull-policy-to-always.yaml
    target:
      kind: Deployment
  - path: jsonpatches/second-container-pull-policy-to-always.yaml
    target:
      kind: Deployment
      name: this|that

fejta-bot · 2019-12-01T14:22:12Z

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle stale

antoninbas · 2019-12-12T00:58:30Z

Any thoughts on adding this to the default images transformer?

images:
  - name: postgres
    newName: my-registry/my-postgres
    newTag: v1
    newPullPolicy: IfNotPresent

I am aware that this is not quite the ask of this issue...

antoninbas · 2019-12-12T00:59:00Z

/remove-lifecycle stale

jbmcfarlin31 · 2020-03-05T18:27:13Z

workaround:

patches:
  - path: jsonpatches/first-container-pull-policy-to-always.yaml
    target:
      kind: Deployment
  - path: jsonpatches/second-container-pull-policy-to-always.yaml
    target:
      kind: Deployment
      name: this|that

@matti , what does your patch yaml file look like for setting the imagePullPolicy? I am trying to set the imagePullPolicy values for all rendered yaml generated from kompose (which translates docker-compose into kubernetes yaml).

matti · 2020-03-05T19:43:34Z

sorry I kinda stopped using kustomize - it is too hard or impossible to have things like this.

jbmcfarlin31 · 2020-03-05T20:07:22Z

@matti I feel you. I cannot seem to get imagePullPolicy to work, at all. I either end up replacing the whole container spec or something else... thinking I might have to implement by own patching utility..

antoninbas · 2020-03-05T20:16:24Z

@jbmcfarlin31

You have to apply a patch like this one:

apiVersion: apps/v1
kind: DaemonSet
metadata:
  name: antrea-agent
spec:
  template:
    spec:
      containers:
        - name: antrea-agent
          imagePullPolicy: IfNotPresent
        - name: antrea-ovs
          imagePullPolicy: IfNotPresent
      initContainers:
        - name: install-cni
          imagePullPolicy: IfNotPresent

It is less than ideal. There should be a way to change the imagePullPolicy with the images transformer.

jbmcfarlin31 · 2020-03-05T20:19:17Z

@antoninbas do you need to have a specific patch file like that? What I mean by specific is like exact name mappings and so on?

We basically take a compose file, convert with kompose, and then want to apply kustomize patches to that rendered yaml file. The compose files we are converting aren't necessarily stuff we own, so we won't know the names of services and so on.

We ideally want something just like deployment_patch.yaml:

kind: Deployment
spec:
  templates:
    spec:
       containers:
          imagePullPolicy: Always

That is then applied to all future Deployments generated by kompose.

antoninbas · 2020-03-05T20:23:37Z

I tried that a while back but it didn't work for me. I had to enumerate all containers by name.

For your use case, it would be great if @matti's patch worked:

- op: replace
  path: "/spec/template/spec/containers/*/imagePullPolicy"
  value: Always

but the wildcard * does not work here. It is not part of the JSON patch RFC (https://tools.ietf.org/html/rfc6902) as far as I can tell, so that explains why kustomize does not support it.

It would be great if one of the kustomize developers could comment on this issue though, in case there is an alternative solution.

jbmcfarlin31 · 2020-03-05T20:29:30Z

@antoninbas man that was not the news I was hoping for lol. So as it sits currently, without the developers commenting, there currently is no way to patch through kustomize or potentially through the kubectl patch ... command all imagePullPolicy fields within deployments?

antoninbas · 2020-03-05T20:39:44Z

Not that I know of. But I have been using kustomize very lightly so I am definitely not an expert.

pre · 2020-03-06T11:28:26Z

You can deploy an admission controller webhook which mutates all the objects live on the cluster and ensures imagePullPolicy is what you need 😅 🌮

TekTimmy · 2020-04-28T17:12:29Z

We are using self build docker images in Minikube, therefore the ImagePullPolicy should be Never for local development but Always for all other environments. I did not expect this to be so hard with Kustomize 😢
Also using environment variables seems to be not possible 😢 😢

TekTimmy · 2020-04-28T17:24:00Z

Made it working with mentioned patchesStrategicMerge...
My cronjob YML:

apiVersion: batch/v1beta1
kind: CronJob
metadata:
  name: base-cronjob
spec:
  schedule: "*/1 * * * *"
  jobTemplate:
    spec:
      template:
        spec:
          containers:
            - name: base-cronjob
              image: "cronjob:latest"
              imagePullPolicy: "Never"
              args: ['python3 cronjob.py']

My kustomization.yml (important is providing the containers name):

apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
  - cronjob
patchesStrategicMerge:
  - |-
    apiVersion: batch/v1beta1
    kind: CronJob
    metadata:
      name: base-cronjob
    spec:
      schedule: "*/2 * * * *"
      jobTemplate:
        spec:
          template:
            spec:
              containers:
                - name: base-cronjob
                  image: "cronjob:dev"
                  imagePullPolicy: "Always"

fejta-bot · 2020-07-27T17:41:45Z

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle stale

agascon · 2020-07-29T14:03:01Z

sorry I kinda stopped using kustomize - it is too hard or impossible to have things like this.

Hi @matti, might I ask which other tool you moved for this kind of templating?

I'm trying similar templating like in this issue and feeling exactly the same, that either it's not possible or very diffcult. I think there should be another way.

Thanks!

matti · 2020-07-29T17:19:56Z

helm. Helm is the clear winner of these tools.

…

On 29. Jul 2020, at 17.03, agascon ***@***.***> wrote: sorry I kinda stopped using kustomize - it is too hard or impossible to have things like this. Hi @matti, might I ask which other tool you moved for this kind of templating? I'm trying similar templating like in this issue and feeling exactly the same, that either it's not possible or very diffcult. I think there should be another way. Thanks! — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or unsubscribe.

bygui86 · 2020-07-29T18:24:43Z

Helm is a great tool, but write and maintain a chart is really a pain!
Go-templating and the shitty yaml indentation are a deadly mix :(

matti · 2020-07-29T19:02:46Z

I know. That's why I tried kustomize (and kpt), but issues like these just wont work with declarative approach. Just give another try for helm, it also handles removal of resources nicely (have you tried what happens when you remove a kustomize resouce? you need to delete that manually)

bygui86 · 2020-07-29T21:56:58Z

There is already an issue about resources removal, so I think that it will be fixed soon.

I think that Kustomize offers lots of really important features and the community will add more and more within next months. Features that are completely compliant with declarative approach.

@matti can you make an example of declarative approach when Kustomize does not work?

matti · 2020-07-30T07:10:58Z

This issue? And also this "closed" issue here: #168 (comment)

matti · 2021-07-23T06:18:06Z

/remove-lifecycle stale

…

On 22. Jul 2021, at 20.00, fejta-bot ***@***.***> wrote: /remove-lifecycle stale.

ringerc · 2021-09-17T05:26:53Z

Pretty surprised to find this limitation. It's explained by the fact that a json patch specifies paths as "JSON Pointers" http://jsonpatch.com/#json-pointer which are specific identifiers for a single document-element. It is not JSONPath, there are no wildcards and there is no pattern matching.

I guess the correct answer is probably to write a custom kustomize transformer. I'm very surprised to see the need for that for a task that seems likely to be as commonplace and basic as this though.

ringerc · 2021-09-17T05:36:34Z

See also #720

k8s-triage-robot · 2021-12-16T06:14:20Z

The Kubernetes project currently lacks enough contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

After 90d of inactivity, lifecycle/stale is applied
After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

Mark this issue or PR as fresh with /remove-lifecycle stale
Mark this issue or PR as rotten with /lifecycle rotten
Close this issue or PR with /close
Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

k8s-triage-robot · 2022-01-15T06:26:26Z

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

After 90d of inactivity, lifecycle/stale is applied
After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

Mark this issue or PR as fresh with /remove-lifecycle rotten
Close this issue or PR with /close
Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle rotten

cmeury · 2022-01-20T17:23:58Z

/remove-lifecycle rotten

Just got hit by this as well, not for images, but for removing a bunch of keys from a list in a single manifest. I was frantically trying to understand why the asterisk is not supported, until I stumbled across this thread.

k8s-triage-robot · 2022-04-20T17:26:42Z

The Kubernetes project currently lacks enough contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

After 90d of inactivity, lifecycle/stale is applied
After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

Mark this issue or PR as fresh with /remove-lifecycle stale
Mark this issue or PR as rotten with /lifecycle rotten
Close this issue or PR with /close
Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

matti · 2022-04-20T17:33:48Z

just use sed and/or envsubst on kustomize yaml before apply

this will never be fixed in kustomize, because kustomize is designed to be promising, but painful.

blachniet · 2022-04-24T19:14:13Z

I had the same need and solved it by creating a plugin: PullPolicyTransformer. There's usage instructions and an example in the repository: https://github.com/blachniet/kustomize-plugins.

It's packaged as a Containerized KRM Function, which is an alpha feature at the time of this writing (see Plugin feature status).

k8s-triage-robot · 2022-05-24T19:37:27Z

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

After 90d of inactivity, lifecycle/stale is applied
After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

Mark this issue or PR as fresh with /remove-lifecycle rotten
Close this issue or PR with /close
Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle rotten

michael-db · 2022-05-30T15:04:40Z

/remove-lifecycle rotten

Vanderscycle · 2022-08-15T18:16:21Z

/remove-lifecycle rotten

@antoninbas feature would be amazing. I am sure it can be applied elsewhere too.

marculsi · 2022-11-01T18:58:30Z

This can sort of be done with raw Kustomize using replacements since it supports wildcard fieldPaths.

The caveat is that your deployments/containers need to have the imagePullPolicy already specified. But once you have it you can change all containers imagePullPolicy to the other types of policies.

There is a "create=true" option if the value is missing but you cannot use it when using wildcards to select all containers.

Error: cannot support create option in a multi-value target

rkdrnf · 2023-02-01T02:46:48Z

Instead of listing every target deployment in patches, specifying targets using labelSelector may resolve some problems.

patches:
  - path: image-pull-policy.patch.yml
    target:
      labelSelector: "image=my-image"

moatorres · 2023-10-12T16:03:35Z

I think that should work, no? 🤔

patches:
  - target:
      kind: Deployment
      name: my-deployment
    patch: |-
      - op: replace
        path: /spec/template/spec/containers/0/imagePullPolicy
        value: Always

joebowbeer · 2023-10-17T15:42:13Z

@moatorres as stated in the description with respect to /0/

changes/adds the imagePullPolicy to first container, but how to set it to all containers?

marculsi · 2023-10-17T16:30:02Z

They fixed create=true for Wildcard replacements that I mentioned earlier so if anyone is still looking for this here is an example.

Not sure which version it got added in. Im using latest Kustomize(5.1.1+) but it should be in some earlier versions.

kustomization.yaml

apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization


configMapGenerator:
- name: replacement-config
  options:
    disableNameSuffixHash: true
  literals:
    - imagePullPolicy=Always

resources:
  - "deployment.yaml"


replacements:
- source:
    kind: ConfigMap
    name: replacement-config
    fieldPath: data.imagePullPolicy
  targets:
  - select:
      kind: Deployment
      name: nginx-deployment
    options:
      create: true
    fieldPaths:
    - spec.template.spec.containers.*.imagePullPolicy

deployment.yaml

apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx-deployment
  labels:
    app: nginx
spec:
  replicas: 1
  selector:
    matchLabels:
      app: nginx
  template:
    metadata:
      labels:
        app: nginx
    spec:
      containers:
      - name: nginx
        image: nginx:1.14.2
        ports:
        - containerPort: 80
      - name: nginx2
        image: nginx:1.14.2
        ports:
        - containerPort: 80

k8s-ci-robot added the lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. label Dec 1, 2019

k8s-ci-robot removed the lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. label Dec 12, 2019

antoninbas mentioned this issue Dec 12, 2019

Enable IPSec encryption of tunnel traffic antrea-io/antrea#209

Merged

ossareh mentioned this issue Jun 23, 2020

MountVolume.SetUp failed for volume "efs-pv" kubernetes-sigs/aws-efs-csi-driver#192

Closed

jk2K mentioned this issue Jul 2, 2020

[feature request] json path support wildcard GoogleContainerTools/skaffold#4438

Open

k8s-ci-robot added the lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. label Jul 27, 2020

k8s-ci-robot added the lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. label Jul 22, 2021

k8s-ci-robot removed the lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. label Jul 23, 2021

ringerc mentioned this issue Sep 17, 2021

extend patch from one target to multiple targets #720

Closed

k8s-ci-robot added the lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. label Dec 16, 2021

k8s-ci-robot added lifecycle/rotten Denotes an issue or PR that has aged beyond stale and will be auto-closed. and removed lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. labels Jan 15, 2022

k8s-ci-robot removed the lifecycle/rotten Denotes an issue or PR that has aged beyond stale and will be auto-closed. label Jan 20, 2022

k8s-ci-robot added the lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. label Apr 20, 2022

k8s-ci-robot added lifecycle/rotten Denotes an issue or PR that has aged beyond stale and will be auto-closed. and removed lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. labels May 24, 2022

k8s-ci-robot removed the lifecycle/rotten Denotes an issue or PR that has aged beyond stale and will be auto-closed. label May 30, 2022

KnVerey mentioned this issue Nov 25, 2022

Wildcard support for creation in ReplacementTransformer #4886

Merged

k8s-ci-robot closed this as completed in #4886 Dec 6, 2022

Changing 'imagePullPolicy' of all containers in all deployments #1493

Changing 'imagePullPolicy' of all containers in all deployments #1493

Comments

matti commented Sep 2, 2019 • edited Loading

matti commented Sep 2, 2019

matti commented Sep 2, 2019

matti commented Sep 2, 2019

fejta-bot commented Dec 1, 2019

antoninbas commented Dec 12, 2019

antoninbas commented Dec 12, 2019

jbmcfarlin31 commented Mar 5, 2020

matti commented Mar 5, 2020

jbmcfarlin31 commented Mar 5, 2020

antoninbas commented Mar 5, 2020

jbmcfarlin31 commented Mar 5, 2020

antoninbas commented Mar 5, 2020

jbmcfarlin31 commented Mar 5, 2020

antoninbas commented Mar 5, 2020

pre commented Mar 6, 2020 • edited Loading

TekTimmy commented Apr 28, 2020 • edited Loading

TekTimmy commented Apr 28, 2020 • edited Loading

fejta-bot commented Jul 27, 2020

agascon commented Jul 29, 2020

matti commented Jul 29, 2020 via email

bygui86 commented Jul 29, 2020

matti commented Jul 29, 2020

bygui86 commented Jul 29, 2020

matti commented Jul 30, 2020

matti commented Jul 23, 2021 via email

ringerc commented Sep 17, 2021

ringerc commented Sep 17, 2021

k8s-triage-robot commented Dec 16, 2021

k8s-triage-robot commented Jan 15, 2022

cmeury commented Jan 20, 2022

k8s-triage-robot commented Apr 20, 2022

matti commented Apr 20, 2022

blachniet commented Apr 24, 2022

k8s-triage-robot commented May 24, 2022

michael-db commented May 30, 2022

Vanderscycle commented Aug 15, 2022

marculsi commented Nov 1, 2022 • edited Loading

rkdrnf commented Feb 1, 2023

moatorres commented Oct 12, 2023

joebowbeer commented Oct 17, 2023

marculsi commented Oct 17, 2023 • edited Loading

matti commented Sep 2, 2019 •

edited

Loading

pre commented Mar 6, 2020 •

edited

Loading

TekTimmy commented Apr 28, 2020 •

edited

Loading

TekTimmy commented Apr 28, 2020 •

edited

Loading

marculsi commented Nov 1, 2022 •

edited

Loading

marculsi commented Oct 17, 2023 •

edited

Loading