Explicit workflows

[kskyten]

Defining the workflows implicitly with dvc run is very convenient. However, defining the workflows explicitly adds a possibility for decoupling the workflows from specific repositories. Perhaps it is even possible to have a library of workflow templates. Pachyderm specifies the workflows in a json format, which is probably a good starting point.

[kskyten]

On second thought, the json format might not be a good idea. Luigi, which I'm not very familiar with, defines the dependency graph in Python. This is obviously more powerful and it is easier to make workflow templates in code.

[dmpetrov]

That's funny. Right now @efiop is working on yaml file for dvc run. It is going to be an alternative to implicit dvc run python script.py .... We are going to support both.

Using Python as a format is probably not the best choice for DVC since DVC is language agnostic. Also, we preferred yaml over json because yaml is more human-readable (comments, aliases). While we keep json format for internal machine generated status files.