Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

exp init: create output dirs #7752

Merged
merged 1 commit into from
Jun 7, 2022
Merged

exp init: create output dirs #7752

merged 1 commit into from
Jun 7, 2022

Conversation

dberenbaum
Copy link
Collaborator

Related to #7740 (comment).

This PR creates output directories (but not any files) during dvc exp init, which accomplishes:

  1. Avoiding errors when trying to Git-ignore those outputs.
  2. Making it easier for users to generate outputs without needing to generate the parent directories.
$ dvc exp init --metrics eval.json --plots eval/scalars --models models/predict.h5 python train.py
Creating dependencies: src, data and params.yaml
Creating output directories: eval/scalars and models
Creating train stage in dvc.yaml

Ensure your experiment command creates eval.json, eval/scalars and models/predict.h5.
You can now run your experiment using "dvc exp run".

@dberenbaum dberenbaum requested a review from skshetry May 16, 2022 17:08
@dberenbaum dberenbaum requested a review from a team as a code owner May 16, 2022 17:08
@dberenbaum dberenbaum force-pushed the exp-init-mkdir-outs branch from 6bacb12 to cee35b4 Compare May 16, 2022 18:03
@dberenbaum dberenbaum force-pushed the exp-init-mkdir-outs branch from cee35b4 to 0ade8c1 Compare May 19, 2022 16:14
@dberenbaum
Copy link
Collaborator Author

@skshetry pinging you for review 🙏

Comment on lines +83 to +84
new_deps: List["Dependency"],
new_out_dirs: List[str],
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a bit inconsistent, with new_deps being List[Dependency] and others being a List[str], but don't have any good ideas. :)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe we don't need to return List[Dependency], that may be a better solution. But it's fine for now. :)

@jorgeorpinel
Copy link
Contributor

Probably needs some docs updates?

@jorgeorpinel
Copy link
Contributor

jorgeorpinel commented Jun 11, 2022

I guess we're using iterative/dvc.org#3430 (comment) to include this too.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
No open projects
Archived in project
Development

Successfully merging this pull request may close these issues.

3 participants