cmd-ref: document new gc behavior #1023

skshetry · 2020-02-25T15:16:15Z

You may disregard these recommendations if you used the Edit on GitHub button from dvc.org to improve a doc in place.

❗ Please read the guidelines in the Contributing to the Documentation list if you make any substantial changes to the documentation or JS engine.

🐛 Please make sure to mention Fix #issue (if applicable) in the description of the PR. This causes GitHub to close it automatically when the PR is merged.

Please chose to allow us to edit your branch when creating the PR.

Thank you for the contribution - we'll try to review it as soon as possible. 🙏

Fixes #1013

public/static/docs/command-reference/gc.md

shcheklein

looks great! the major thing is to align option description with other similar options

public/static/docs/command-reference/gc.md

jorgeorpinel

Looking great in general, some precisions below. Not sure whether we're taking this one over (yet), please lmk @shcheklein, thanks

public/static/docs/command-reference/gc.md

jorgeorpinel

Extracted from #1023 (comment)

One last improvement is to also update the description of -a and -T options. Something like:

- `-a`, `--all-branches` - keep cached objects referenced in all Git branches
  as well as in the workspace (implies `-w`). Useful for preserving data for ...

...

- `-T`, `--all-tags` - the same as `-a` above, but applies to Git tags as well
  as in the workspace (implies `-w`). Useful if tags are used to ...

Thanks

shcheklein · 2020-03-07T01:53:52Z

@skshetry @jorgeorpinel what is the status of it? @skshetry is still assigned - are you going to move it forward?

skshetry · 2020-03-07T02:24:33Z

@shcheklein Umm, yes. I'm trying to add --all-commits on this PR as well. I'll also have to address @jorgeorpinel reviews.

shcheklein · 2020-03-07T03:29:08Z

@skshetry sure! np. Just wanted to be sure that I don't block it like it happened last time.

shcheklein · 2020-03-15T02:40:42Z

@jorgeorpinel @skshetry ping! any progress on this?

skshetry · 2020-03-16T06:49:46Z

@shcheklein, sorry, I couldn't get to the docs. I have adjusted it now and addressed most of the suggestions. 🙂

Please take a look.

public/static/docs/command-reference/gc.md

shcheklein · 2020-03-18T05:40:04Z

@skshetry @jorgeorpinel PTAL - https://dvc-landing-gc-flag-cha-stdoiq.herokuapp.com/doc/command-reference/gc - I've applied some edits to the doc

skshetry · 2020-03-18T06:59:42Z

content/docs/command-reference/gc.md

-the local cache, which is typically located on the same machine as the project
-in question. This usually helps to free up disk space.
+This command deletes (garbage collects) data files or directories that exist in
+DVC cache but are no longer needed. With `--cloud` it also removes data in


but are no longer needed.

It'd be better to express this in any other way. I don't have a better suggestion, though than the following:

Suggested change

DVC cache but are no longer needed. With `--cloud` it also removes data in

DVC cache but are no longer in use. With `--cloud` it also removes data in

Good question, I think needed was there before. I don't have a very strong opinion ... may be even needed better because it is more precise here - I just keep stuff that I need. need is more general then use. Still, need to review other place with in use/need.

Changed to always use need vs in-use. I think need and needed are actually more correct here.

jorgeorpinel

Just a few more precisions but looks good to me otherwise:

content/docs/command-reference/gc.md

jorgeorpinel · 2020-03-18T17:22:32Z

content/docs/command-reference/gc.md

+`--all-commits`, or any combination of them must be provided. Each of them
+corresponds to the current workspace _and_ a set of commits to analyze what
+files, directories and what versions are still needed and should be kept (by
+analyzing DVC-files in those commits).


Suggested change

`--all-commits`, or any combination of them must be provided. Each of them

corresponds to the current workspace _and_ a set of commits to analyze what

files, directories and what versions are still needed and should be kept (by

analyzing DVC-files in those commits).

`--all-commits`, or any combination of them must be provided. Each of them

corresponds to keeping the data for the current workspace and for a

different set of commits (determined by reading the DVC-files in them).

Question: what's the point of this option? To me it sounds like "don't gc anything" except maybe deleted parts of the repo like removed branches, rewritten commits, etc.

DVC cache contains data that is not referenced in any way in Git history. E.g. run do a lot of dvc run, or dvc repro after modifying certain files/settings. If --no-commit is not specified DVC is saving data to cache not matter you do git commit to save DVC-files or not.

We probably mention this.

mentioned a possible use case in the recent version - @jorgeorpinel pls take a look

Makes sense, thanks.

Still this suggestion wasn't applied but I'll address in regular updates.

I put a new paragraph that explains the use case, have you seen it? Or may be I didn't get you specific suggestion ... 🤔

I see it, thanks. May reword a little in regular updates but it's great!

One of the use cases for this option is to safely delete all temporary data DVC cached when dvc run and/or dvc repro were run without committing changes to DVC-files (thus potentially caching data that is not referenced from workspace or Git commits).

content/docs/command-reference/gc.md

Co-Authored-By: Jorge Orpinel <[email protected]>

…ommits

shcheklein · 2020-03-19T02:50:40Z

@jorgeorpinel @skshetry PTAL again.

skshetry

LGTM 👍

shcheklein · 2020-03-19T03:48:37Z

@jorgeorpinel merging this, feel free to do changes on top of it!

per #1023 (review)

jorgeorpinel · 2020-03-19T07:04:10Z

Great 🎉 Just added a couple more changes in 0f10925 (part of #1072)

per #1023 (comment)

skshetry added the A: docs Area: user documentation (gatsby-theme-iterative) label Feb 25, 2020

skshetry requested review from jorgeorpinel and shcheklein February 25, 2020 15:16

skshetry self-assigned this Feb 25, 2020

shcheklein temporarily deployed to dvc-landing-gc-flag-cha-j6yay1 February 25, 2020 15:16 Inactive

shcheklein reviewed Feb 26, 2020

View reviewed changes

public/static/docs/command-reference/gc.md Outdated Show resolved Hide resolved

shcheklein reviewed Feb 26, 2020

View reviewed changes

public/static/docs/command-reference/gc.md Outdated Show resolved Hide resolved

shcheklein reviewed Feb 26, 2020

View reviewed changes

public/static/docs/command-reference/gc.md Outdated Show resolved Hide resolved

shcheklein requested changes Feb 26, 2020

View reviewed changes

weekly-digest bot mentioned this pull request Mar 1, 2020

Weekly Digest (23 February, 2020 - 1 March, 2020) #1031

Closed

skshetry requested a review from shcheklein March 2, 2020 08:14

jorgeorpinel reviewed Mar 2, 2020

View reviewed changes

public/static/docs/command-reference/gc.md Outdated Show resolved Hide resolved

This comment has been minimized.

Sign in to view

jorgeorpinel suggested changes Mar 2, 2020

View reviewed changes

jorgeorpinel mentioned this pull request Mar 3, 2020

Reconsider gc implementation iterative/dvc#2325

Closed

jorgeorpinel suggested changes Mar 3, 2020

View reviewed changes

skshetry mentioned this pull request Mar 9, 2020

gc: make --all-commits flag visible iterative/dvc#3429

Merged

3 tasks

cmd-ref: document new gc behavior

21572eb

skshetry force-pushed the gc-flag-changes branch from 5611c60 to 21572eb Compare March 16, 2020 05:31

skshetry requested a review from jorgeorpinel March 16, 2020 05:33

shcheklein reviewed Mar 16, 2020

View reviewed changes

public/static/docs/command-reference/gc.md Outdated Show resolved Hide resolved

skshetry and others added 2 commits March 17, 2020 18:27

mention about -w as scope specifier for garbage collection

50156f1

Merge branch 'master' into gc-flag-changes

ab7f625

apply some corrections to the latest gc doc

e530271

shcheklein temporarily deployed to dvc-landing-gc-flag-cha-stdoiq March 18, 2020 05:28 Inactive

skshetry commented Mar 18, 2020

View reviewed changes

jorgeorpinel suggested changes Mar 18, 2020

View reviewed changes

Update content/docs/command-reference/gc.md

e4f1b3c

Co-Authored-By: Jorge Orpinel <[email protected]>

shcheklein temporarily deployed to dvc-landing-gc-flag-cha-stdoiq March 18, 2020 22:51 Inactive

Update content/docs/command-reference/gc.md

06445b8

Co-Authored-By: Jorge Orpinel <[email protected]>

shcheklein temporarily deployed to dvc-landing-gc-flag-cha-stdoiq March 18, 2020 22:52 Inactive

Update content/docs/command-reference/gc.md

f90622d

Co-Authored-By: Jorge Orpinel <[email protected]>

shcheklein temporarily deployed to dvc-landing-gc-flag-cha-stdoiq March 18, 2020 22:53 Inactive

Update content/docs/command-reference/gc.md

e2ea8a7

Co-Authored-By: Jorge Orpinel <[email protected]>

shcheklein temporarily deployed to dvc-landing-gc-flag-cha-stdoiq March 18, 2020 22:53 Inactive

address reviews feedback, always use need, mention use case for all-c…

49e4f53

…ommits

shcheklein temporarily deployed to dvc-landing-gc-flag-cha-stdoiq March 19, 2020 02:48 Inactive

shcheklein requested a review from jorgeorpinel March 19, 2020 02:49

shcheklein assigned shcheklein and unassigned skshetry Mar 19, 2020

skshetry commented Mar 19, 2020

View reviewed changes

shcheklein merged commit 0381090 into master Mar 19, 2020

skshetry deleted the gc-flag-changes branch March 19, 2020 03:55

jorgeorpinel added a commit that referenced this pull request Mar 19, 2020

cmd ref: imros to gc

0f10925

per #1023 (review)

jorgeorpinel added a commit that referenced this pull request Mar 19, 2020

cmd ref: update gc --all-commits desc

db19c60

per #1023 (comment)

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

cmd-ref: document new gc behavior #1023

cmd-ref: document new gc behavior #1023

skshetry commented Feb 25, 2020

shcheklein left a comment

This comment has been minimized.

jorgeorpinel left a comment

jorgeorpinel left a comment •

edited

Loading

shcheklein commented Mar 7, 2020

skshetry commented Mar 7, 2020

shcheklein commented Mar 7, 2020

shcheklein commented Mar 15, 2020

skshetry commented Mar 16, 2020

shcheklein commented Mar 18, 2020

skshetry Mar 18, 2020

shcheklein Mar 18, 2020

shcheklein Mar 19, 2020

jorgeorpinel left a comment

jorgeorpinel Mar 18, 2020

jorgeorpinel Mar 18, 2020

shcheklein Mar 18, 2020

shcheklein Mar 18, 2020

shcheklein Mar 19, 2020

jorgeorpinel Mar 19, 2020

jorgeorpinel Mar 19, 2020

shcheklein Mar 19, 2020

jorgeorpinel Mar 19, 2020

shcheklein commented Mar 19, 2020

skshetry left a comment

shcheklein commented Mar 19, 2020

jorgeorpinel commented Mar 19, 2020

	DVC cache but are no longer needed. With `--cloud` it also removes data in
	DVC cache but are no longer in use. With `--cloud` it also removes data in

cmd-ref: document new gc behavior #1023

cmd-ref: document new gc behavior #1023

Conversation

skshetry commented Feb 25, 2020

shcheklein left a comment

Choose a reason for hiding this comment

This comment has been minimized.

jorgeorpinel left a comment

Choose a reason for hiding this comment

jorgeorpinel left a comment • edited Loading

Choose a reason for hiding this comment

shcheklein commented Mar 7, 2020

skshetry commented Mar 7, 2020

shcheklein commented Mar 7, 2020

shcheklein commented Mar 15, 2020

skshetry commented Mar 16, 2020

shcheklein commented Mar 18, 2020

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jorgeorpinel left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

shcheklein commented Mar 19, 2020

skshetry left a comment

Choose a reason for hiding this comment

shcheklein commented Mar 19, 2020

jorgeorpinel commented Mar 19, 2020

jorgeorpinel left a comment •

edited

Loading