Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Make dvc pull --run-cache allow targets for a granular pull process #4223

Closed
victormaricato opened this issue Jul 17, 2020 · 3 comments · Fixed by #4538
Closed

Make dvc pull --run-cache allow targets for a granular pull process #4223

victormaricato opened this issue Jul 17, 2020 · 3 comments · Fixed by #4538
Assignees
@efiop
Labels
feature request Requesting a new feature p2-medium Medium priority, should be done, but less important research

Comments

@victormaricato
Copy link

Right now, if I do

dvc pull [target]
this will pull only the target I am interested in.

However, if I do dvc pull --run-cache [targets]
the target is ignored, everything is pulled from remote. Not even a warning is thrown that --run-cache does not accept granularly pulling files.

This is based on https://discuss.dvc.org/t/dvc-pull-run-cache-target/444 discussion
@triage-new-issues triage-new-issues bot added the triage Needs to be triaged label Jul 17, 2020
@efiop efiop added the feature request Requesting a new feature label Jul 17, 2020
@triage-new-issues triage-new-issues bot removed the triage Needs to be triaged label Jul 17, 2020
@efiop
Copy link
Contributor

efiop commented Jul 17, 2020

For the record: --run-cache flag is just a temporary solution and is not meant to accept targets. We will likely just support dvc pull target that will then somehow look into the run-cache by itself and pull by itself or suggest a hint on how to do it. run-cache is really an implementation detail that should be hidden away. We need to take a closer look at this.

@efiop efiop added research p2-medium Medium priority, should be done, but less important labels Jul 17, 2020
@victormaricato
Copy link
Author

Oh, ok. Is there any kind of estimative on this being prioritized?

As described in the forum, I was planning to use --run-cache as a workaround for deploying models, but the lack of targets selection makes it impossible.

@weekly-digest weekly-digest bot mentioned this issue Jul 19, 2020
Closed
@efiop
Copy link
Contributor

efiop commented Jul 21, 2020

Hi @maricatovictor ! Sorry for the delay. We've added this task to the next sprint (starting next week).

@efiop efiop mentioned this issue Jul 22, 2020
Closed
@efiop efiop self-assigned this Sep 5, 2020
@efiop efiop mentioned this issue Sep 5, 2020
Merged
2 tasks
efiop added a commit to efiop/dvc that referenced this issue Sep 6, 2020
@efiop
run: consolidate run-cache logic
d42e3a3 
Existing lock files are just an edge case of run-cache.

Prerequisite for iterative#4223
efiop added a commit that referenced this issue Sep 6, 2020
@efiop
run: consolidate run-cache logic (#4535)
93a2983 
Existing lock files are just an edge case of run-cache.

Prerequisite for #4223
@efiop efiop mentioned this issue Sep 7, 2020
Merged
2 tasks
efiop added a commit to efiop/dvc that referenced this issue Oct 5, 2020
@efiop
repro: support --pull
2c5147b 
This option allows run-cache to pull the cache for the outputs
when restoring a stage instead of pulling everything at once during
`dvc pull --run-cache`.

Fixes iterative#4223
efiop added a commit that referenced this issue Oct 6, 2020
@efiop
repro: support --pull (#4538)
7424f22 
* repro: support --pull

This option allows run-cache to pull the cache for the outputs
when restoring a stage instead of pulling everything at once during
`dvc pull --run-cache`.

Fixes #4223

* repro: adjust --pull help
@pmrowla pmrowla mentioned this issue Mar 14, 2021
Merged
1 task
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@efiop efiop

Labels
feature request Requesting a new feature p2-medium Medium priority, should be done, but less important research
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

repro: support --pull efiop/dvc
2 participants
@efiop @victormaricato