Add repro -R, support dirs & stage files when running pull/push/repro…

[prihoda]

Added support for dvc repro --recursive.
Added support for recursive mode to accept not just directories but also dvc files:
dvc pull/push/fetch/checkout/repro --recursive someFile.dvc someDir/
for convenience (following unix cp -r mv -r conventions that support both dirs and files).

DVC issue #1989
New dvc.org docs issue: iterative/dvc.org#367

[pared]

Good stuff @prihoda!
Two things thought:

1. Would you mind writing test that repro --recursive actually gathers only stages inside targeted dir for recursive = True and isdir(target) = True?
2. I think change of checkout behaviour could be handled in separate PR. It is just a observation to be considered in future requests.

[prihoda]

I realized that there's actually an edge case with -R --single-item that will not behave the expected way (at least not deterministically).

Imagine you have the following DAG: dir/c -> dir/b -> anotherdir/a (c depends on b depends on a).
When anotherdir/a changes and we run dvc repro -R --single-item dir/, we would expect b to be reproduced and therefore c to be reproduced. However, the stages b and c are collected in order as provided by the filesystem, so it can happen that c is checked first and no changes are detected and only then b is reproduced on its own.

I will try to fix this by sorting the found stages in dfs_postorder_nodes order (this is only important in case of --single-item, otherwise the stages will be ordered properly anyway in _reproduce_stages).

[prihoda]

@pared @MrOutis We are ready for review again ;)

No longer applies

[pared]

Thorough testing!
Sorry for mentioning it only now, but recently we started to switch to pytest. Would you be willing to refactor it accordingly?

EDIT:
here is example of test refactoring:
#1953

[efiop]

Outstanding! 🥇 Thank you so much @prihoda !

[efiop]

This is breaking the backward compatibility, but it is fine since our API is not officially released yet and this change is indeed worthy.

[prihoda]

Actually this is just the internal naming, recursive argument was originally used to specify the default behaviour (recursive = not self.args.single_item), so the behavior stays the same. I just renamed the recursive method argument to single_item (and removed the negation) and added a new recursive argument that corresponds to the --recursive option.

[pared]

Great change!