Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ref: exp Examples, etc. #2259

Merged
merged 24 commits into from
Mar 17, 2021
Merged

ref: exp Examples, etc. #2259

merged 24 commits into from
Mar 17, 2021

Conversation

jorgeorpinel
Copy link
Contributor

@jorgeorpinel jorgeorpinel commented Mar 3, 2021

Continuation of #2243
Also splits and improves #1926

@jorgeorpinel jorgeorpinel changed the title ref: dump exp show Examples ref: exp Examples Mar 3, 2021
@shcheklein shcheklein temporarily deployed to dvc-org-ref-exp-example-sh4p1d March 3, 2021 20:06 Inactive
@jorgeorpinel jorgeorpinel temporarily deployed to dvc-org-ref-exp-example-sh4p1d March 3, 2021 20:09 Inactive
@jorgeorpinel jorgeorpinel temporarily deployed to dvc-org-ref-exp-example-sh4p1d March 3, 2021 20:24 Inactive
@jorgeorpinel jorgeorpinel temporarily deployed to dvc-org-ref-exp-example-sh4p1d March 3, 2021 20:37 Inactive
@jorgeorpinel jorgeorpinel temporarily deployed to dvc-org-ref-exp-example-sh4p1d March 3, 2021 20:43 Inactive
@jorgeorpinel jorgeorpinel temporarily deployed to dvc-org-ref-exp-example-sh4p1d March 3, 2021 20:54 Inactive
@jorgeorpinel jorgeorpinel temporarily deployed to dvc-org-ref-exp-example-sh4p1d March 3, 2021 20:55 Inactive
@jorgeorpinel jorgeorpinel temporarily deployed to dvc-org-ref-exp-example-sh4p1d March 3, 2021 21:05 Inactive
@jorgeorpinel jorgeorpinel temporarily deployed to dvc-org-ref-exp-example-sh4p1d March 3, 2021 21:19 Inactive
@jorgeorpinel
Copy link
Contributor Author

Removing draft status. While not complete yet, it's getting big as usual...

@jorgeorpinel jorgeorpinel marked this pull request as ready for review March 3, 2021 21:39
@jorgeorpinel jorgeorpinel temporarily deployed to dvc-org-ref-exp-example-sh4p1d March 3, 2021 21:51 Inactive
@jorgeorpinel jorgeorpinel changed the base branch from ref/exp/all to master March 3, 2021 22:54
@jorgeorpinel jorgeorpinel temporarily deployed to dvc-org-ref-exp-example-khnuoo March 9, 2021 15:39 Inactive
@jorgeorpinel jorgeorpinel changed the title ref: exp Examples ref: exp Examples, etc. Mar 10, 2021
@jorgeorpinel jorgeorpinel changed the title ref: exp Examples, etc. ref: exp Examples Mar 10, 2021
@jorgeorpinel jorgeorpinel changed the title ref: exp Examples ref: exp Examples, etc. Mar 11, 2021
@jorgeorpinel jorgeorpinel temporarily deployed to dvc-org-ref-exp-example-khnuoo March 11, 2021 03:19 Inactive
@jorgeorpinel jorgeorpinel temporarily deployed to dvc-org-ref-exp-example-khnuoo March 11, 2021 04:44 Inactive
@jorgeorpinel jorgeorpinel temporarily deployed to dvc-org-ref-exp-example-khnuoo March 13, 2021 01:57 Inactive
@jorgeorpinel jorgeorpinel requested a review from dberenbaum March 13, 2021 01:57
@jorgeorpinel jorgeorpinel temporarily deployed to dvc-org-ref-exp-example-khnuoo March 14, 2021 05:02 Inactive
Copy link
Contributor

@dberenbaum dberenbaum left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like there's a few examples that need to be cleaned up, but otherwise in pretty good shape.

content/docs/command-reference/exp/apply.md Outdated Show resolved Hide resolved
content/docs/command-reference/exp/apply.md Outdated Show resolved Hide resolved
content/docs/command-reference/exp/apply.md Show resolved Hide resolved
content/docs/command-reference/exp/apply.md Outdated Show resolved Hide resolved
content/docs/command-reference/exp/branch.md Outdated Show resolved Hide resolved
content/docs/command-reference/exp/list.md Outdated Show resolved Hide resolved
You can also list experiments in a any DVC repo with `dvc exp list`:

```dvc
$ dvc exp list --all [email protected]:iterative/example-get-started.git
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we need to actually push some experiments to the repo to support this? I don't it will work right now, which could cause confusion.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe to one of the exp-specific repos you published instead (for the next iteration). Do they already have some exps up?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I haven't pushed any experiments since they need a data remote to save the experiment data. However, there is a read-only remote associated with example-get-started, so I could add some there. Let me know what you think.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yep, having some exps in there would be nice. That remote is read-only in it's HTTP form but you can use the S3 config to upload stuff as you probably already know from modifying the generator script 🙂

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sounds good, I just pushed some experiments up to https://github.com/iterative/example-get-started.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Amazing, thanks! 🙏

content/docs/command-reference/exp/run.md Outdated Show resolved Hide resolved
Comment on lines +259 to +260
> Notice that experiments run as a series don't build up on each other. They are
> all based on `HEAD`.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is meant by this? If you run experiments in your workspace (not with --temp), they will make changes that will get picked up by the subsequent experiment. For example, if you follow this example and then run dvc exp run -S prepare.split=0.3, then featurize.max_features will be 2000 in that experiment even though it is 1000 in HEAD.

content/docs/command-reference/exp/show.md Show resolved Hide resolved
@jorgeorpinel jorgeorpinel temporarily deployed to dvc-org-ref-exp-example-khnuoo March 17, 2021 00:55 Inactive
@jorgeorpinel jorgeorpinel temporarily deployed to dvc-org-ref-exp-example-khnuoo March 17, 2021 01:41 Inactive
@jorgeorpinel jorgeorpinel temporarily deployed to dvc-org-ref-exp-example-khnuoo March 17, 2021 01:42 Inactive
@jorgeorpinel jorgeorpinel temporarily deployed to dvc-org-ref-exp-example-khnuoo March 17, 2021 01:49 Inactive
@jorgeorpinel jorgeorpinel temporarily deployed to dvc-org-ref-exp-example-khnuoo March 17, 2021 01:52 Inactive
@jorgeorpinel jorgeorpinel requested a review from dberenbaum March 17, 2021 01:53
@jorgeorpinel
Copy link
Contributor Author

Hey @dberenbaum thanks for the reviews. You expressed approval in #2259 (review) and all the details have been addressed. Mind formally approving? Can't merge this otherwise 😢 Ty

@jorgeorpinel jorgeorpinel merged commit c152e3b into master Mar 17, 2021
@jorgeorpinel jorgeorpinel deleted the ref/exp/examples branch March 19, 2021 20:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants