executor: setup credentials

[dberenbaum]

Meta-issue. Please link related tickets back to this one.

User story

I want my executor to have all the credentials and configurations I need to access my data and run my code.

Requirements

• The executor has credentials to access the remote through some pattern that is documented and doesn't require manual intervention on the executor instance.
• All reasonable security precautions are taken and risks are noted for docs.

Out of scope

It's not yet necessary to handle all possible authorization patterns, although design flexibility for the future should be considered.

[dberenbaum]

Conversation between @pmrowla and me:

@pmrowla:

By credentials here, do we specifically mean a users --local DVC remote config settings?

If a user is setting credentials in an aws profile, and is only configuring the --local profile field for their S3 remote, that profile won't actually exist in the remote machine without the associated .aws/config

(but copying an entire .aws/config and the associated access keys seems like it would be beyond the scope of what DVC should be doing here)

@dberenbaum:

We need to think about all of that. How would you envision getting .aws/config and .aws/credentials set up in the executor?

@pmrowla:

So in this example, users should be running an in amazon ec2 instance configured with the right IAM role for accessing their S3 bucket (so that when the instance boots, it is automatically configured with the appropriate S3 credentials/aws profile without DVC being involved in the process at all)

AFAIK azure and google have equivalent configurations, (and all of this assumes that the user is using matching cloud storage + cloud execution platforms)

In the event that a user wants to run DVC on an azure machine but use S3 for storage, they should be required to configure a container with appropriate auth credentials themselves

[casperdcl]

FYI template which is useful for implementation: https://github.com/iterative/dvc-tpi/blob/main/run.py

[dberenbaum]

If we start with transfer over ssh, we can deprioritize this story for now.

[dberenbaum]

Needs rescoping