WebDAV authentication using environment variables

[JojoDevel]

WebDAV authentiaction is currently only possible by modifying the dvc remote config. For CI/CD applications it would be much more convenient to configure the credentials using environment variables.

[skshetry]

@JojoDevel, Could you please share more about your usecase? This issue likely belongs to https://github.com/skshetry/webdav4.

Also, I'd prefer you use netrc rather than envvar if you must, as it should already supports it.

eg:

# ~/.netrc should only be accessible to the current user
(umask 077; echo "machine $SECRET_HOSTNAME login $SECRET_USER password $SECRET_KEY" > ~/.netrc)

Or, just set them in the global config in DVC.

[JojoDevel]

Wow, that looks super nice and works in my CI. Maybe we could just add such an example to https://dvc.org/doc/command-reference/remote/modify or https://dvc.org/doc/command-reference/remote/add so that people get aware of it :)

Thanks for the help!

[skshetry]

@JojoDevel, if you could help to document it, that'd be great. 🙂 Here's the docs repo: https://github.com/iterative/dvc.org.

Regarding this issue, I am not really a fan of environment variables for authentication, as I don't find them to be safe. Though I am open to discussion.

[JojoDevel]

Do you know a better solution than environment variables? I'm also not a fan but for me it seems to be the regular way to solve authentication in CI

[skshetry]

@JojoDevel, could you please elaborate why this is inconvenient?

dvc remote modify --global remote user $SECRET_USER
dvc remote modify --global remote password $SECRET_KEY

[JojoDevel]

Hmm, I was looking for a way to add the authentication in my CI pipeline without adding more code (just the env variables). But maybe that's not the best thinking...

[fakegermano]

I agree with @JojoDevel , adding authentication without code on CI/CD pipelines is way more user friendly. To give an example, you can setup up private pypi repositories(sources) that use either Token or HTTP-Basic authentication with poetry (https://python-poetry.org/) and you can add the credentials in two ways:

• with an equivalent to dvc remote modify ... way, requiring additional commands
• with environment variables (setting POETRY_HTTP_BASIC_SOURCE_NAME_USERNAME=$USERNAME for example, for setting the username field for the http-basic configuration for the source-name source)

The second way doesnt require the insertion of extra commands, because the env variable has the name properly set.

We could use something like

DVC_REMOTE_STORAGE_PASSWORD

just my two cents

[skshetry]

@fakegermano, I agree with what you are saying, overriding some config options with environment variables is definitely useful.

However, this issue is more specific, it's about supporting environment variable-based authentication for any WebDAV remotes with DVC.