[Feature request] Ignore resources

[fungiboletus]

Summary

I would like to tell ArgoCD to completely ignore some resources.

Motivation

I have a few resources that change often, and because of that ArgoCD wastes a lot of CPU doing nothing.

I have for example a ConfigMap from KubeVirt CDI named cdi-operator-leader-election-helper that has one annotation named control-plane.alpha.kubernetes.io/leader changing all the time. The annotation is actually ignored and ArgoCD doesn't do any update (No status changes. Skipping patch) but still, it wastes a lot of CPU and network bandwidth.

I also have one EndpointSlice from a TimeScaleDB chart that has a renewTime changing often. It is ignored but still, I can see the resource usage increasing a lot when the resource is managed by ArgoCD.

I tried to add the annotation argocd.argoproj.io/hook: Skip on the resources I want to ignore, but it doesn't prevent the resource to trigger reconciliations in ArgoCD when it is updated.

Proposal

One annotation such as argocd.argoproj.io/ignore: true would work for example.

[crenshaw-dev]

Maybe related: #9558

[jessesuen]

I also have one EndpointSlice from a TimeScaleDB chart that has a renewTime changing often. It is ignored but still, I can see the resource usage increasing a lot when the resource is managed by ArgoCD.

We actually prevent Endpoints from refresh Applications because we know these objects to be of high churn. Likewise, I think we may want to do the same for EndpointSlices:

https://github.com/argoproj/gitops-engine/blob/f9456de217f8fb8893d09998bab0d8efff95f4fd/pkg/cache/cluster.go#L1050-L1054

This code was written before EndpointSlices was a thing

[sidewinder12s]

I am also wondering if some parts of DaemonSet should be ignored as well. On high churn clusters we get pretty high resource utilization on the app controller and a lot of messages in debug logs about app refreshes being triggered by status changes from DaemonSets.

Refreshing app argocd/cluster-istio for change in cluster of object istio-system/istio-cni-node of type apps/v1/DaemonSet

Setting:

data:
  resource.compareoptions: |
    ignoreResourceStatusField: all

As suggested by some in #8100 didn't appear to have an effect.

[agaudreault]

The behavior can be configured in ignoreResourceUpdates to resolve this issue.

[astralko]

@agaudreault-jive how?
I have keda that changes my replicaset replicas, adding pods, etc.
added
resource.customizations.ignoreResourceUpdates.apps_Deployment: |
jsonPointers:
- /status
- /spec/replicas
resource.customizations.ignoreResourceUpdates.apps_ReplicaSet: |
jsonPointers:
- /annotations/deployment.kubernetes.io/desired-replicas
- /annotations/deployment.kubernetes.io/max-replicas
- /status
- /spec/replicas

based on what I saw when comparing yamls,
but I still see in the controller logs:

time="2023-09-04T13:32:57Z" level=debug msg="Requesting app refresh caused by object update" api-version=apps/v1 application=argocd/customer-deployment cluster-name=cluster fields.level=1 kind=Deployment name=customer-deployment namespace=customer server="https://eks-url"
time="2023-09-04T13:32:57Z" level=debug msg="Requesting app refresh caused by object update" api-version=v1 application=argocd/customer-deployment cluster-name=cluster fields.level=0 kind=Pod name=customer-deployment-7cdd665ccb-qrtj7 namespace=customer server="https://eks-url"
time="2023-09-04T13:32:57Z" level=debug msg="Requesting app refresh caused by object update" api-version=apps/v1 application=argocd/customer-deployment cluster-name=cluster fields.level=0 kind=ReplicaSet name=customer-deployment-7cdd665ccb namespace=customer server="https://eks-url"
time="2023-09-04T13:32:57Z" level=debug msg="Requesting app refresh caused by object update" api-version=apps/v1 application=argocd/customer-deployment cluster-name=cluster fields.level=1 kind=Deployment name=customer-deployment namespace=customer server="https://eks-url"
time="2023-09-04T13:32:57Z" level=debug msg="Requesting app refresh caused by object update" api-version=cilium.io/v2 application=argocd/customer-deployment cluster-name=cluster fields.level=0 kind=CiliumEndpoint name=customer-deployment-7cdd665ccb-qrtj7 namespace=customer server="https://eks-url"
time="2023-09-04T13:32:57Z" level=debug msg="Requesting app refresh caused by object update" api-version=apps/v1 application=argocd/customer-deployment cluster-name=cluster fields.level=0 kind=ReplicaSet name=customer-deployment-7cdd665ccb namespace=customer server="https://eks-url"
time="2023-09-04T13:32:57Z" level=debug msg="Requesting app refresh caused by object update" api-version=apps/v1 application=argocd/customer-deployment cluster-name=cluster fields.level=0 kind=ReplicaSet name=customer-deployment-7cdd665ccb namespace=customer server="https://eks-url"
time="2023-09-04T13:32:58Z" level=debug msg="Requesting app refresh caused by object update" api-version=v1 application=argocd/customer-deployment cluster-name=cluster fields.level=0 kind=Pod name=customer-deployment-7cdd665ccb-qrtj7 namespace=customer server="https://eks-url"
time="2023-09-04T13:32:58Z" level=debug msg="Requesting app refresh caused by object update" api-version=v1 application=argocd/customer-deployment cluster-name=cluster fields.level=0 kind=Pod name=customer-deployment-7cdd665ccb-qrtj7 namespace=customer server="https://eks-url"
time="2023-09-04T13:32:58Z" level=debug msg="Requesting app refresh caused by object update" api-version=v1 application=argocd/customer-deployment cluster-name=cluster fields.level=0 kind=Pod name=customer-deployment-7cdd665ccb-qrtj7 namespace=customer server="https://eks-url"

[agaudreault]

@astralko You can take a look at https://argo-cd.readthedocs.io/en/stable/operator-manual/argocd-cm-yaml/ for more examples.

for the annotations, I think it would need to be specified like '.metadata.annotations."deployment.kubernetes.io/desired-replicas"'.

Also, the ignore resource updates will only be applied to tracked resources.

In the case of autoscaling, I am not sure it is the best idea to use this feature. The configurations are on the server side and will apply to all resources for any Application. Ignoring those fields would cause all your deployments and replicaSet to have potentially out-of-date status.

Maybe reducing the frequency of the autoscaler with minimum delay and cooldown period would help mitigate the issue.

[astralko]

@agaudreault-jive thanks. I tried increasing the autoscaler polling interval but it didn't help.
If this feature could be implemented for an application level that would be great, thanks anyways