Authentication when retrieving a Helm chart value files via HTTP

[fabricepipart1a]

Summary

When a Argo CD Application or ApplicationSet references a value file via HTTP the file must be accessible without any authentication. It would be preferable to be able to use some authentication for the cases when the file is stored in a repository that is protected by authentication.

Motivation

In some cases, we want to store the value files separately from the repository that generates the Helm chart. It is a different team that is in charge of maintaining the value files. Those value files pass a validation process before they are considered valid and published. As a consequence, we prefer to reference the value files by URL in Argo's Application or ApplicationSet.
Please note that referencing the value file by URL is something that is not documented but actually works ;-)

apiVersion: argoproj.io/v1alpha1
kind: ApplicationSet
metadata:
  name: my-app
  namespace: my-argo-ns
spec:
  generators:
  - list:
      elements:
      - cluster: cluster1
      - cluster: cluster2
  template:
    metadata:
      name: 'my-app-{{cluster}}'
    spec:
      project: my-project
      source:
        repoURL: 'https://repository.mycompany.net/helm-charts'
        targetRevision: 1.2.3
        helm:
          valueFiles:
            - 'https://repository.mycompany.net/helm-charts/my-app-chart/values/2.3.4/my-app-values-{{cluster}}.yaml'
        chart: my-app-chart
      destination:
        name: '{{cluster}}'
        namespace: my-app-ns

For convenience, we store the value files of a chart next to the chart itself. The problem is that credentials are not managed to download the value files. This is something we would like to see addressed to be able to secure the access to the Helm chart repository.

Proposal

The best solution IMHO would be to reuse the same authentication mechanism that is used for the Helm chart repositories. In my case, the value files would be stored alongside the Chart. So the same mechanism could apply.

Regarding the syntax, I see two possibilities. Either we keep the same yaml syntax and Argo detects that it is an existing HTTP repository path and uses the relevant credentials to connect:

...
spec:
  template:
    spec:
      source:
        helm:
          valueFiles:
            - 'https://repository.mycompany.net/helm-charts/my-app-chart/values/2.3.4/my-app-values-{{cluster}}.yaml'

Or we define an alternative syntax

spec:
  template:
    spec:
      source:
        helm:
          valueFiles:
            - valueFileURL: 'https://repository.mycompany.net/helm-charts'
               credentials: secret-name

But I have the feeling that's more complex.

[hornv]

As for me, I have the same issue. I have to have separate values set for different environments. And yes, I need to set some auth on it.
But the way @fabricepipart1a proposed probably might not work everywhere. I want to store values files in Google Cloud, so I have the service account json file for authentication, but argo-cd does not have that type of auth. Could you add that type of auth at least to values?
Thanks

[jannfis]

I was wondering whether this enhancement becomes superfluous once #2789 is implemented, which allows storing values file for Helm charts in a Git repository. This way, Argo CD would just use the credentials configured for the Git repository where the values file is stored.

[romuduck]

Indeed. Passing values via a Url is more like a workaround until 2789 gets implemented (just saw it was shifted by a release), but with no authent plus the need to regenerate the url so to "refresh" the app is a bit of a bricolage.
Going with a plugin is better.

[romuduck]

I also saw some request for enhancement on appset side to add "function"... associated with a git file generator you could cover #2789 (Loosing the commit track though)

[fabricepipart1a]

@jannfis I would say that having #2789 could unblock us. But I don't think that would fill the gap for all our needs. let me elaborate why.

As of today, we have a separate repository that contains all our environment files. This is because:

• we wanted to have them generated
• we have tens of similar environments
• those environment files are handled by ops and not by dev like the helm chart is

Since there is a templating mechanism and that value files are generated:

• You can not reference the files directly on git
• we used to build them and then upload them on a repository accessible via https (Artifactory in our case)

In some cases, we could store the value file flat in the git repository and that would already be a good start. Even if I wonder which way is the easiest for us:

• keeping the value file generation at build time and storing the file in a https repository accessible without authentication (which already works today)
• switching to storing plain values files in the git repository

[blakepettersson]

This should be addressed with #10432

[fabricepipart1a]

That's excellent news !
I could not determine if that would work in the exact same way if the source for a value file is a helm repository or a git repository.

I suppose I just need to wait for 2.6 to confirm that (we just installed 2.5 in the last days). But in any case that will unblock us 😄

Thanks for the great support

[gaspard-armagnat]

Hello @blakepettersson,

I'm a bit confused as how to implement the initial example shared by @fabricepipart1a with multi-sources applications. Can you please share a sample implementation?

As for the requirement: we want to version independently both the Helm chart and the value file(s). Any altermative solution is welcome.

Thanks!

[blakepettersson]

Hi @gaspard-armagnat does this guide help?

[gaspard-armagnat]

Hello @blakepettersson, in the initial example the value file is not from a git repository but from a DML / artifact repository.

[blakepettersson]

@gaspard-armagnat sure but in the example below that it shows an example using values files from a git repository:

apiVersion: argoproj.io/v1alpha1
kind: Application
spec:
  sources:
  - repoURL: 'https://prometheus-community.github.io/helm-charts'
    chart: prometheus
    targetRevision: 15.7.1
    helm:
      valueFiles:
      - $values/charts/prometheus/values.yaml
  - repoURL: 'https://git.example.gom/org/value-files.git'
    targetRevision: dev
    ref: values

[gaspard-armagnat]

@blakepettersson thanks for the example. But what if the value file is not in a git repository but only hosted on a web server?