stack rm command

[snoyberg]

• Without arguments, lists all local GHCs and snapshots
• Takes output from previous command and deletes relevant directories
• Maybe have a --all flag?

Thoughts on this from anyone?

[snoyberg]

Can also be used to delete old GHC/Git archives (see #376, and pinging @conklech

[rvion]

Without arguments, lists all local GHCs and snapshots

do you mean: without arguments, list all local GHC and snapshots and ask the user to re-run the command with one or several of them after stack rm

Takes output from previous command and deletes relevant directories

not sure to understand

Maybe have a --all flag?

+1

[snoyberg]

Yes, that's what I meant. My idea is that the output will be something like:

Can delete:

* stack rm ghc-7.8.4
* stack rm ghc-7.10.1
* stack rm lts-2.15

Running any of those commands will delete the relevant directories

[conklech]

+1 from me, too. --all would be constructive, and ought to be mentioned alongside the installation instructions; it should be best practice to have a documented uninstallation procedure.

[radix]

I'm slightly uneasy about calling this "rm", it just doesn't seem like the right name for it. Maybe "clean" or "cleanup"?

Feel free to ignore my bikeshedding :)

[rvion]

@radix I understand it may not feel like the best name, but I think lots of people are used to this one.
I would still go for stack rm because of git rm and docker rm, that will be used by most users of stack

[conklech]

Two issues arose from #376 that are relevant here:

1. Should we give an option to delete just the GHC tarball without nuking the installation? And should stack rm ghc-7.8.4 remove or preserve the installation tarball? See Remove GHC/git tarballs after installation #376 for discussion of the rationales for each option.

2. On Windows, stack also installs its own git. That should be on the list as well.

[radix]

@rvion Funny, I had exactly those kinds of commands in mind, and it gave me the opposite reaction. Maybe I don't understand the goal of stack rm enough. But with git rm it's obvious that the thing you'd want to remove is a file. and docker rm would remove a container. But stack doesn't have any one domain that rm would obviously apply to... The idea is for stack rm to delete either GHC installations or the cached packages in a snapshot directory?

[cies]

What about stack purge? And is the --all flag the same as rm -rf ~/.stack ?

[snoyberg]

I'm open to any bikeshedding, purge sounds fine to me. And yes, I think --all would be roughly equivalent to rm -rf ~/.stack

[conklech]

Perhaps clean would be better. The idea is only to remove objects that can be mechanically recreated by downloading and/or recompiling. It's not really garbage-collecting, since we have no concept of things being "live". But it's not a potentially-irreversible operation like rm implies. The only side-effect is to increase the cost of future operations (and perhaps require stack setup).

Incidentally, that assumption would be contradicted if clean --all deleted .stack/global/stack.yaml or any other user-defined configuration files; perhaps there should be a runtime or documentation warning if such a file will be removed.

[cies]

I don't have strong feelings with either of the names. The reason I suggested purge was that (1) rm reminds me most of git rm, actually removing one or more files by path/filename, (2) clean reminds me most of make clean which deals with files created during a build of this-very-project, and (3) purge reminds me of dropping some cached objects.

[duplode]

I was going to ask about this in #115. Good to see that it is in the plans - uncontrolled proliferation of sandboxes is an annoyance with the vanilla cabal sandbox workflow; a stack purge would be the last step towards solving that issue.

[chreekat]

I like clean because of symmetry with aptitude clean: "Removes all previously downloaded .deb files from the package cache directory (usually /var/cache/apt/archives)." Your experience may vary (YEMV?)

[conklech]

stack clean is a different, already-implemented function, which I think is more like make clean.

Sorry, I thought I had posted a follow-up when I realized my clean proposal had that problem. FWIW, I'm now in the purge camp.

[ches]

Incidentally, that assumption would be contradicted if clean --all deleted .stack/global/stack.yaml or any other user-defined configuration files; perhaps there should be a runtime or documentation warning if such a file will be removed.

Agreed, slightly more subtlety required than rm -rf ~/.stack. I'd also be annoyed if it nuked ~/.local/bin, perhaps that's obvious if there's no way of knowing what each of those binaries was built with.

I'm not familiar at all with stack's internals, but no doubt this can be done programmatically with better maintainability than some hardcoded paths… but could we enumerate here what this effectively would do? This might capture the wanted PR intent and UX in simple terms, and could help for scripting a solution in the meantime (and perhaps serve as FAQ material until it's ready—e.g. "I have old snapshots eating up GBs of disk that I'm no longer using for active projects, how can I remove them safely?").

For instance, I would assume stack rm <snapshot> would effectively do:

• rm -rf ~/.stack/snapshots/<arch>/<snapshot>
• rm -rf ~/.stack/global-project/.stack-work/install/<arch>/<snapshot>
• …?

And stack rm <ghc>:

• rm -rf ~/.stack/programs/<arch>/<ghc>*
• rm -rf ~/.stack/precompiled/<arch>/<ghc>
• rm -rf ~/.stack/global-project/.stack-work/install/*/*/<ghc>
• …?

Does that appear complete? Should it operate on a local project's ./.stack-work also (I assume yes)?

Aside: FWIW with reasoning shared so far, I like stack purge.

[mgsloan]

My current thought on this is to keep a timestamped log of package, project, and database usages. Not 100% sure we want to keep track of individual package usages. It would allow for more fine-grained garbage collection, but possibly for too much overhead.

The overall idea is that build results which haven't been used for a while are less likely to be used in the future. Similarly to stack docker cleanup, we'd offer interactive selection of build results to remove.

[CMCDragonkai]

Is there no garbage collection for un-used GHC versions?

[mgsloan]

@CMCDragonkai Not currently. Could probably have something similar to stack docker gc, and track last usage times for snapshots / compilers.

[CMCDragonkai]

@mgsloan What does stack docker gc do?

[borsboom]

@CMCDragonkai I think he meant stack docker clean, which cleans up old Docker images.

[chris-martin]

The garbage collection question definitely needs to be answered in some form or another. If possible, I think I might find a documentation solution preferable to a new command. It would be great if the manual discussed the directory structure of ~/.stack and explained what directories were safe to delete.

[snoyberg]

Moved to wishlist: https://github.com/commercialhaskell/stack/wiki/Wishlist

[anka-213]

As a workaround, how would you do this manually? My last attempt at purging old ghc versions manually broke my stack installation.

[dylan-thinnes]

@snoyberg I'm considering applying using this issue to apply for Google Summer of Code, do you have any pointers to where I could go to look at the feasibility of working on a stack rm/clean, along with other features to mend space usage? If there are other performance improvements you are thinking would specifically be useful over this one, I'd be glad to discuss those too.

I've run into personal issues with stack being too large for my chromebook before, and I'm worried it would put off newcomers to stack to have it use as much space as it does, so needless to say I've been interested in this issue for a while.

[snoyberg]

As a workaround, how would you do this manually? My last attempt at purging old ghc versions manually broke my stack installation.

I've deleted GHC versions by deleting ghc-X.Y.Z* in the ~/.stack/programs/$arch-$os directory. I've never had problems with that. I'm not sure how this would have broken your installation.

do you have any pointers to where I could go to look at the feasibility of working on a stack rm/clean, along with other features to mend space usage?

I think just the issue tracker, I don't have anything in particular. It's certainly feasible to add an rm command. A more general gc command for snapshots would require more design.

[juhp]

I made a little hack today: NB this is quite experimental use at your own risk
https://github.com/juhp/stack-clean-old
Feedback is welcome - specially if you know better heuristics.

[qrilka]

@juhp I remember @snoyberg saying something like that just checking directory timestamps isn't quite enough as precompiled lib sharing could use older snapshots, not sure how to test such a scenario easily though.

[juhp]

@qrilka Yeah I believe that but it sure can save a lot of diskspace ;-)

Currently I keep 50 "snapshot" dirs per ghc-version and 4 project builds per ghc version, but it is a hack yes.

Maybe something better could be done inside stack.

edit: I dropped the snapshots cleaning part for now since that was quite damaging: it needs to be a lot smarter about tracking dependencies between hashes.

It seems to me this approach may need a global view of all .stack-work dirs really to work correctly - I need to understand how it all hangs together better.

[juhp]

(BTW stack-clean-old is now on Hackage and changed a fair a bit)