cabal install --lib <something> creates environment file that does not mention base

[jwaldmann]

With cabal-install-3.10.1.0, after cabal install --lib <something>, ghci no longer loads Prelude:

Fine, note: no environment file:

$ ghci
GHCi, version 9.6.1: https://www.haskell.org/ghc/  :? for help
ghci> False
False

Install package:

$ cabal install --lib leancheck
Resolving dependencies...
Build profile: -w ghc-9.6.1 -O1
In order, the following will be built (use -v for more details):
 - leancheck-1.0.0 (lib) (requires build)
Starting     leancheck-1.0.0 (lib)
Building     leancheck-1.0.0 (lib)
Installing   leancheck-1.0.0 (lib)
Completed    leancheck-1.0.0 (lib)

Start ghci again, note: with environment file

$ ghci
Loaded package environment from /home/waldmann/.ghc/x86_64-linux-9.6.1/environments/default
GHCi, version 9.6.1: https://www.haskell.org/ghc/  :? for help
ghci> False

<interactive>:1:1: error: [GHC-88464]
    Data constructor not in scope: False

Work-Around:

$ ghci -package base
Loaded package environment from /home/waldmann/.ghc/x86_64-linux-9.6.1/environments/default
GHCi, version 9.6.1: https://www.haskell.org/ghc/  :? for help
ghci> False
False

Indeed, the environment file (created by the install) does not mention base, while they previously did.
This is with

$ cabal --version
cabal-install version 3.10.1.0
compiled using version 3.10.1.0 of the Cabal library 

and ghc-9.6.1, also ghc-9.4.4. This is a change in behaviour from cabal-install-3.8.1.0

[ulysses4ever]

Thanks for the report! That's due to #8607 most likely. Cc @gbaz

[jwaldmann]

Thanks for the quick response. Are you saying this is intended behaviour?

I do understand that env files are evil (creating per-user cabal hell), so --lib also is?

[Mikolaj]

Yes, --lib is evil and cabal env only slightly less so. I think the idea is that you add manually whatever global packages you need, but @gbaz would need to confirm if base was intended to be covered by this rule.

[gbaz]

I would say that neither is evil, both are fine, when used right imho. But we're still figuring out the best UI for using them right. And generally I think the right way is not a global env file, but a local one per "project unit."

This is indeed a consequence of the prior PR, and maybe there can be some inbetween.

One can also just "install" base after installing another package as well:

% cabal install --package-env=foo.env --lib leancheck
Resolving dependencies...
% cabal install --package-env=foo.env --lib base     
Resolving dependencies...
% ghci ghci -package-env foo.env 
GHCi, version 9.4.1: https://www.haskell.org/ghc/  :? for help
ghci> False
False

One thing to improve the UX might be to always add base, and never warn about it, as a special case?

[jwaldmann]

always add base

seems reasonable - well, in a world where we also "always import Prelude".

I find it highly counter-intuitive that some install command removes (from visibility) packages
(relative to a freshly installed ghc without an environment file). This also holds for, e.g., containers.

[ulysses4ever]

I think base should be added automatically. (Maybe any non-reinstallable package, but that's less clear.) Not sure if it defeats the whole purpose of #8607 or will it be "mostly there". My hunch is that it should be okay. But it requires experiment, I guess.

Anyways, if anything, this shows that we should have some integration tests for this business.

[gbaz]

If we auto-add base with no other changes, then that's not enough. We also need to add it in such a way that it only gets added if it is not already there, and arguably if the env file is fresh. Otherwise we will have the same situation that PR fixed, where amending an existing env file inevitably generated an error.

[ulysses4ever]

@gbaz well, that's the thing: I don't see how it will fail "inevitably". Your PR was motivated by things like directory and process, which, when accidentally pinned, can easily screw up the plan. But base is not the same. Updating an environment usually won't bring you a new base, I think. You may run into several entries of the same base, and I just checked: it doesn't get you into trouble because, for example, ghci happily loads an environment with duplicate entries. Although you may try to ordNub the list, so that you don't run into this harmless but a bit embarassing situation.

When it can brake, I think, is when you upgrade your compiler. In that case, you may end up with two basea of different versions, and the overall plan will fail. This, I think, is a reasonable tradeoff: we say we don't support updating env files between compilers, and you must start from scratch (as in: manually remove the env file, although I advocated for autoresetting in the past #8483). In return we don't require you to cabal install --lib base.

Without base Haskell is, generally speaking, unusable (of course, there may be special cases). So, requiring cabal install --lib base is embarassing. Let me also bring up an example from another ecosystem -- that of the Julia language: it has a very similar mechanism of environments: environment is described in a text file and lists its name, a unique ID and a list of packages you want to use in this environment. Of course, the standard library, stdlib, is not mentioned there and available by default.

I challenge you to find a single example of a language ecosystem with a similar notion of package environment, where the standard library wouldn't be included by default. The closest I know is, perhaps, Agda. But then again: there's so much woes how user unfriendly that is (cf. agda/agda#5725).

---

@jwaldmann my view in a nutshell: (a) env files are fine, (b) cabal install --lib is an underdeveloped POC, for which there's no plan (or even design) to extend it to a fully-featured thing, so better avoided; (c) in its current form, cabal install --lib should refuse to update an existing environment because it can't guarantee to do so safely (or even sanely). More details on this I provide in my RFC #8483, whic failed to find any support.

[gbaz]

Your PR was motivated by things like directory and process, which, when accidentally pinned, can easily screw up the plan. But base is not the same. Updating an environment usually won't bring you a new base, I think.

It was partly motivated by that, but more importantly it was motivated by the fact that even when calling cabal install --lib with the same versions, you would still get an error. So therefore, cabal install --lib always failed when an environment file already existed, full stop. Further, before that PR, this error was not recoverable. Now, it can be overridden with the force flag. I am agreeing that base should be added. I am however noting that when we do so, we need to make sure that doing so does not trigger this behavior.

I would argue that aside from the current inconvenience of needing to manually add base if you want it, which can and should be fixed, cabal install --lib works pretty ok, and we can fix future issues as they're reported. The complaints in that RFC about not being able to update an environment are essentially what the PR fixed. That's why the RFC didn't find support. If you have further concrete issues you think need fixing, please raise issues, and they can be thought through and dealt with.

[ulysses4ever]

So therefore, cabal install --lib always failed when an environment file already existed, full stop.

This is not true. I just checked that 3.8 worked fine at least for some combinations of packages (that is, non mutually exclusive ones) E.g.

❯ cabal install --lib --package-env=. aeson
...
❯ cabal install --lib --package-env=. clock
...
❯ cat .ghc.environment.x86_64-linux-9.4.4
clear-package-db
global-package-db
package-db /home/artem/.cabal/store/ghc-9.4.4/package.db
package-id ghc-9.4.4
package-id bytestring-0.11.3.1
package-id unix-2.7.3
package-id base-4.17.0.0
package-id time-1.12.2
package-id hpc-0.6.1.0
package-id filepath-1.4.2.2
package-id process-1.6.16.0
package-id array-0.5.4.0
package-id integer-gmp-1.1
package-id containers-0.6.6
package-id ghc-boot-9.4.4
package-id binary-0.8.9.1
package-id ghc-prim-0.9.0
package-id ghci-9.4.4
package-id rts-1.0.2
package-id terminfo-0.4.1.5
package-id transformers-0.5.6.2
package-id deepseq-1.4.8.0
package-id ghc-boot-th-9.4.4
package-id pretty-1.1.3.6
package-id template-haskell-2.19.0.0
package-id directory-1.3.7.1
package-id text-2.0.1
package-id aeson-2.1.2.1-b4e99e6d85eb68b552d3bc6448013ed3c346f3f90bd1b87671bf5157dba380f7
package-id clock-0.8.3-e1cb5c09c08b802543e1c283427ef69d76d97cf2dec49fcef18840c2a8f690bf

❯ ghci                                                                                                            0 (0.002s) < 16:54:33
Loaded package environment from /home/artem/Temp/cabal-dup-env/.ghc.environment.x86_64-linux-9.4.4
GHCi, version 9.4.4: https://www.haskell.org/ghc/  :? for help
Loaded GHCi configuration from /home/artem/.ghci
λ> import Data.Aeson
λ> import System.Clock
λ> 
Leaving GHCi.

---

cabal install --lib works pretty ok

Well, it's a judgement call and so is hard to argue with. For me, it's barely usable because of two things:

• at some point it was relatively easy to get mutually exclusive packages in one environment but I fail to repro it now (it needs some creativity in which packages you sequentially cabal install --lib);
• there's no support for profiling libs: Cannot install a profiled library with cabal install --lib -p pkgname #7832.

[gbaz]

Oops, one correction. The issue with adding all the global packages was that e.g. time is such a package but it is reinstallable -- so one should be able to install a lib built against a different version of time and that was prevented by this. That's why adding base is ok, because of course base is not (as of now) reinstallable.