Introducing cache types: data, metrics and plots, run-cache and per-file

[dmpetrov]

Today we use a single remote to store all the data (which can be redefined by --remote option). However, there are different types of information in the cache:

1. Data files
2. Metrics and plots files
3. Run-cache files

There might be several reasons (data sensitivity or optimizations) to store these artifacts in different remotes:

1. Data cannot be moved outside of a company data center (outside of a country) while it's ok to use clouds (S3) for metrics and run-cache for performance or for simplifying the user experience.
2. Data needs to be duplicated among several buckets/clouds for redundancy while it is not a requirement for metrics.
3. Metrics and run-cache - tons of small files while data is large data files. Users might need to use separate storage for the optimization reason.
4. It might be required in the future when some remotes might need auto-compression remote storage auto-compression #1239 or splitting data by blocks Split data into blocks  #829 while the other files don't need that.

It would be greater to introduce "types" of remote to group some of the cache types.

Proposal ideas

This is the very first iteration on the subject and I don't have a clear proposal yet. We need to take a look at some analogs and possible solutions. How I see it for now:

$ dvc remove add --default --type data mydata ssh://some/dir
$ dvc remove add --default --type metrics,plots,run-cache mymeta s3://mybucket
$ dvc remove add --default --type data,metrics,plots --redundancy mybackup ssh://backup/dir

Related subjects

Remote per file

This use case can be extended to per-file scenarios. Sometimes a special remote is required for a specific file (one bucket for data sources, the other for the pipeline derivative data sources). An extreme case - imported data sources. It would be great to have the information about remotes and use a single command dvc push to use all the remotes instead of specifying --remove option all the time.

Change remotes globally

See #2960

External workspaces

It might be also related to #3920

Random ideas

Does it make sense to use the new dvc.yaml for storing all this information or part of these?

[pared]

Remote per file

Related to #2095

[Suor]

There is also a use case where we need to separate data as well. So on top of --type their might be some glob pattern or smth.