Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

-R collects stages from all of the repo #5326

Open
skshetry opened this issue Jan 25, 2021 · 5 comments
Open

-R collects stages from all of the repo #5326

skshetry opened this issue Jan 25, 2021 · 5 comments
Labels
optimize Optimizes DVC p2-medium Medium priority, should be done, but less important performance improvement over resource / time consuming tasks

Comments

@skshetry
Copy link
Member

Bug Report

We are building repo.graph and using path to search for stages in the graph when -R instead of just reading files from the path directory.

Context

https://groups.google.com/a/iterative.ai/g/support/c/H_c36GuAsPM/m/6mLNdPIRAgAJ
@skshetry skshetry added p1-important Important, aka current backlog of things to do performance improvement over resource / time consuming tasks optimize Optimizes DVC labels Jan 25, 2021
@skshetry skshetry changed the title -R collects stage from all of the repo -R collect stages from all of the repo Jan 25, 2021
@skshetry skshetry changed the title -R collect stages from all of the repo -R collects stages from all of the repo Jan 25, 2021
@efiop
Copy link
Contributor

efiop commented Jan 25, 2021

This is by-design though, we are being paranoid in every operation. But it does make sense to revisit this in general.

@skshetry
Copy link
Member Author

skshetry commented Mar 9, 2021

Had a conversation with a user (~100k .dvcfiles, 525s to load), where this would have been really helpful.

@dberenbaum
Copy link
Collaborator

Bumping this priority down for now.

@dberenbaum dberenbaum added p2-medium Medium priority, should be done, but less important and removed p1-important Important, aka current backlog of things to do labels Feb 18, 2022
@skshetry skshetry mentioned this issue May 17, 2022
Closed
@dberenbaum

This comment was marked as off-topic.

Sign in to view
@skshetry

This comment was marked as off-topic.

Sign in to view
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
optimize Optimizes DVC p2-medium Medium priority, should be done, but less important performance improvement over resource / time consuming tasks
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@dberenbaum @efiop @skshetry