ghc-pkg list is not finding packages I install with cabal 3.0 and ghc 8.8.1

[GeorgeCo]

Describe the bug
ghc-pkg list is not finding packages I install with cabal 3.0 and ghc 8.8.1

To Reproduce
ghc --version
The Glorious Glasgow Haskell Compilation System, version 8.8.1
cabal --version
cabal-install version 3.0.0.0
bash-3.2$ cabal install QuickCheck
Resolving dependencies...
Up to date
Warning: You asked to install executables, but there are no executables in
target: QuickCheck. Perhaps you want to use --lib to install libraries
instead.
bash-3.2$ cabal install --lib QuickCheck
Resolving dependencies...
Up to date
bash-3.2$ ghc-pkg list|fgrep -i Quick

Expected behavior
I expect ghc-pkg list to show QuickCheck
System informataion

• Operating system
  macOS 10.14.6
• cabal, ghc versions
  cabal 3.0
  ghc 8.8.1
  Additional context
  Add any other context about the problem here.

[DanielG]

I think this is expected behaviour, at least from the cabal side of things. Version 3.0.0.0 switched to using v2-build by default so installed libraries are now registered into the "default" package environment file in .ghc/*/environments/default as well as a package-db in .cabal/store/ghc-*/package.db which GHC doesn't know about by itself.

AFAICS ghc-pkg simply doesn't have support for listing package environments which sort of kind of makes sense but also breaks this workflow. I think this should be reported as a GHC bug but I'm not sure it would even be a good idea to add pkg-env support to ghc-pkg.

[DanielG]

cc @hvr

[GeorgeCo]

Thanks Daniel I thought it might be a ghc issue but since nobody else was complaining I assumed I must have done something wrong. I can file a ghc bug if people want. Cheers George

…

On Mon, Oct 7, 2019 at 2:58 AM Daniel Gröber (dxld) < ***@***.***> wrote: cc @hvr <https://github.com/hvr> — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#6262?email_source=notifications&email_token=ABQIJ62KAVHHP7PVFRQHTB3QNLFXLA5CNFSM4I3MHL4KYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEAPDLEY#issuecomment-538850707>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABQIJ6ZESTAS3OMI6YBTN6LQNLFXLANCNFSM4I3MHL4A> .

[GeorgeCo]

I filed a ghc bug: https://gitlab.haskell.org/ghc/ghc/issues/17341
It was closed as not a bug. I updated it with:

ok, but it is a change in behaviour from the view point of an end user.
Thanks for responding; from your response, the env file, and the doc I was able to figure out that I now need to type: "ghc-pkg list -f /Users/gcolpitts/.cabal/store/ghc-8.8.1/package.db" Not sure if this change should be in the release notes.

This feels to me like a step backwards from just typing "ghc-pkg list" as the user doesn't have to know anything about package dbs or their location. Not sure how to go about improving the current state but thanks for your help.

[gbaz]

So I think there's two things. First a feature request for ghc-pkg that it become env file aware. This is not in scope for this tracker.

The thing in scope for this tracker is more documentation about how v2-build works in cabal 3.0, which is covered by this ticket among others: #6104

So going to close this out.

[GeorgeCo]

see https://gitlab.haskell.org/ghc/ghc/issues/17341 for the feature request that ghc-pkg become env file aware.

[BarneyStratford]

I worked around this for now by saying:
ln -s ~/.cabal/store/ghc-8.6.5/package.db ~/.ghc/x86_64-darwin-8.6.5/package.conf.d

EDIT:
Also say:
rm -r ~/.ghc/x86_64-darwin-8.6.5/environments

It seems to me that the problem is simply that there isn't a wildcard option to expose all visible packages in the database. With this, ghc-pkg wouldn't need to be environment-aware, as the default environment could just use the wildcard.

[weskerfoot]

I strongly feel like this issue should not have been closed, or at least document that this is broken.

[gbaz]

This is not a cabal issue. ghc-pkg is not a cabal tool, it is a ghc tool. the relevant open issue is on the ghc tracker, as posted above: https://gitlab.haskell.org/ghc/ghc/issues/17341

[BarneyStratford]

I just re-read @DanielG 's comment, saying that cabal adds the newly-installed library to the default environment, so I did some experiments and found that this is only partially true. This may be the root of the problem.

If I say cabal install xxx --lib, where xxx is a library package, then it is indeed added to the default environment. However, if I just say cabal install xxx, as is habitual, then it warns:
"Warning: You asked to install executables, but there are no executables in target: xxx. Perhaps you want to use --lib to install libraries instead." It does add the library to the package db in this case but does not add it to the default environment.

The warning message feels odd to me, as I didn't ask Cabal to install executables, I just asked it for xxx. I'd expect it to realise that the package contains only libraries and behave accordingly. So this issue feels to me like a bug, inasmuch as the behaviour I'm seeing surprises me.

[gbaz]

That behavior has nothing to do with this ticket. That's the behavior of install under 3.0 semantics. Further, the warning seems like it explains this behavior pretty clearly.

[DanielG]

@BardurArantsson I agree the --lib thing is pretty akward and I have to admit I don't really understand why we don't simply install all exes and all libs all the time, like v1-install used to do. Instead we have this ugly --lib cludge that doesn't even seem to work for all use cases.

Do we have a tracking issue for revisiting the v2-install semantics yet?

[fgaz]

@DanielG

Do we have a tracking issue for revisiting the v2-install semantics yet?

#6481

In which I believe this is explained too:

why we don't simply install all exes and all libs all the time, like v1-install used to do

[pablolob]

Hey! I am having a similar issue, I download succesfully any package with cabal install and they are stored in .cabal/store/ghc but when I import them on ghci it gives the error Module not found.
Do you know how I could fix this? Thanks

[dstibbe]

Hey! I am having a similar issue, I download succesfully any package with cabal install and they are stored in .cabal/store/ghc but when I import them on ghci it gives the error Module not found.
Do you know how I could fix this? Thanks

I just worked around it for now by using cabal v1-install {package} ... slow, but seems to do the trick.

[pablolob]

Hey! I am having a similar issue, I download succesfully any package with cabal install and they are stored in .cabal/store/ghc but when I import them on ghci it gives the error Module not found.
Do you know how I could fix this? Thanks

I just worked around it for now by using cabal v1-install {package} ... slow, but seems to do the trick.

Hey dstibbe, thank you very much. You are my savior, I was so desperate that I was ready to format the whole hardisk. Thanks :)

[1chb]

And Haskell is not widely used because?

[Mikolaj]

And Haskell is not widely used because?

@1chb: would you like to elaborate constructively? Ideally, perhaps you'd also like to contribute?

[cobordism]

would you like to elaborate constructively? Ideally, perhaps you'd also like to contribute?

The comment is indeed snarky, but it highlights the fact that Haskell tooling offers a sub-optimal experience for newcomers. The toolchain is both opaque and fragile. Furthermore, when hunting down a bug like the one in this thread, it is frustrating to be met with variations of "Not our problem" at every turn.

Just think about how this looks. ghc-pkg is not finding ghc pkgs and all we find is finger pointing.

[Mikolaj]

@cobordism: I'm afraid finger-pointing (to other issues on the tracker, to other tools, to direct and direct causes, etc.) is an unavoidable part of the development process. I'm not claiming cabal contributors are perfect or the process is perfect or the toolchain is perfect, but I won't accept discouraging contributors just because the world is not perfect and somebody likes to express his understandable disappointment in passive-aggressive way towards (mostly) innocent bystanders or, worse, people equally disappointed, but trying to actually improve the situation.

At the same time, I'm absolutely open to constructive criticism. If you have ideas how to improve the ghc-pkg situation, please do share. If you don't, please start by contributing typo-fixes, then small code patches, gradually learn the tangle of conflicting requirements and perhaps then you are likely to suggest a (partial, shallow, harming other users, whatever) solution and we'd all be grateful to consider it together.

[fgaz]

I'd also add that we do have a solution for the original problem (listing packaes added to an env) on our side: cabal env list (part of #6481). So this is is only a not-our-bug for the ghc-pkg part (maybe I should remove the label...). Please discuss cabal env list on #6481, and ghc-pkg on the ghc issue tracker

[cobordism]

That is fair. 1chb's comment was very unhelpful.

Personally, I am not a coder and I only code very rarely. When I do I like to play around with anything from Python to Lean. However I stopped using Haskell for several of my (admittedly tiny) coding problems after realising that every time I spent the better part of a day just getting the tools to work again. Every. Single. Time.

If I might make one suggestion it would be the following:
Ending up in a github thread like this is often the end-point of a long and frustrating journey for someone new to Haskell, cabal, stack, ghc-mod, and all the rest... Encountering an inexplicable error, trying, failing, searching... It would seem to me that you have a much better insight into where exactly this bug need to be fixed. Thus my suggestion would be to close such a github issue with:
"We are closing this issue because the bug is not ours to fix. The correct place to file this bug is X"
or even better:
"We are closing this issue because the bug is not ours to fix. We have filed a bug report in the correct place: see X"
That way this feels like less of a dead end instead of just a waypoint in the journey.

PS just saw your latest comment. Thanks for those links.

[Mikolaj]

@cobordism: I hear you. Thank you for sharing your (sad) user story and the bit about ending issues in a clear statement is a directly useful feedback. I hope next time you'd find Haskell tooling stable enough for casual use (I'm afraid the v1 to v2 transition was quite extreme in this regard and it's not entirely finished yet). Repeating @fgaz, because it got buried again:

I'd also add that we do have a solution for the original problem (listing packages added to an env) on our side: cabal env list (part of #6481). So this is is only a not-our-bug for the ghc-pkg part (maybe I should remove the label...). Please discuss cabal env list on #6481, and ghc-pkg on the ghc issue tracker.

[1chb]

And Haskell is not widely used because?

@1chb: would you like to elaborate constructively? Ideally, perhaps you'd also like to contribute?

Sorry, I was struggling with a similar package problem myself. Some packages were hidden, I could use them by explicitly specify -package flags to ghc (without that flags ghc was confused and suggested I should use :set ... as if I was using ghci). Also I couldn't see my hidden packages at all (not within parenthesis or in blue) when I did ghc-pkg list. Although, not as helpful, but I don't think my comment was more snarky than cabal's warning about missing --lib. As I understand it, that part worked better in earlier versions. My hidden packages didn't become visible until I removed all my packages in ~/.cabal and ~/.ghc and reinstalled ghc and the packages. So, maybe a bad state on my folder.

[Mikolaj]

@1chb: I sympathize. Indeed, this workflow (mixing cabal and ghc-pkg and generally installing packages globally instead of per-project) worked better previously. Citing links to places where we work on these precise issues, in case you'd like to help/follow/cheer:

I'd also add that we do have a solution for the original problem (listing packages added to an env) on our side: cabal env list (part of #6481). So this is is only a not-our-bug for the ghc-pkg part (maybe I should remove the label...). Please discuss cabal env list on #6481, and ghc-pkg on the ghc issue tracker.

[GeorgeCo]

would you like to elaborate constructively? Ideally, perhaps you'd also like to contribute?

The comment is indeed snarky, but it highlights the fact that Haskell tooling offers a sub-optimal experience for newcomers. The toolchain is both opaque and fragile. Furthermore, when hunting down a bug like the one in this thread, it is frustrating to be met with variations of "Not our problem" at every turn.

Just think about how this looks. ghc-pkg is not finding ghc pkgs and all we find is finger pointing.

As the original filer of this bug here is my summary of the current state of this issue which has moved from cabal to ghc which seems to be overlooked here.

1. @cobordism thanks for your comments. I feel the same.
2. As I mention in an earlier comment this issue moved to https://gitlab.haskell.org/ghc/ghc/issues/17341. Can somebody change the label at the top right of this page: "Resolution: not our bug" to "Resolution: not our bug , see https://gitlab.haskell.org/ghc/ghc/issues/17341." I would do this but don't know how. It is no longer closed but is still open as a feature request whereas it seems to me a regression from an end user view point.
3. In that bug I mention https://gitlab.haskell.org/ghc/ghc/-/issues/18933 which is a request to document the workaround.
4. So a bug that started here in Sep 2019 is, in May 2021, an open ghc feature request and an open ghc request to document a workaround.
5. Filers of bugs like this are extremely unlikely to be capable of fixing their bug so suggesting that they do so can come across as as saying go away. There is a long and difficult road to get from reporting a bug to being able to fix one.
6. As cobordism mentions this issue makes newcomers feel that GHC is not welcoming to newcomers. My past experience with other ghc bugs I have filed has generally been much better than this issue. It seems the two follow on issues would be better places to advance this. As for me, I've given up for now and have found other places to spend my energy.
7. People contribute to the extent that their ability and time allow. Filing a bug is a contribution.

[GeorgeCo]

I'd also add that we do have a solution for the original problem (listing packaes added to an env) on our side: cabal env list (part of #6481). So this is is only a not-our-bug for the ghc-pkg part (maybe I should remove the label...). Please discuss cabal env list on #6481, and ghc-pkg on the ghc issue tracker

@fgaz in what versions of ghc/Cabal do we have the solution cabal env list?

bash-3.2$ cabal env list
cabal: unrecognised command: env (try --help)
bash-3.2$ ghc-pkg list|fgrep Cabal
    Cabal-3.4.0.0
bash-3.2$ ghc --version
The Glorious Glasgow Haskell Compilation System, version 9.0.1

[Mikolaj]

Hi @GeorgeCo,

As mentioned on #6481, cabal env is still a prototype, which means it's not released yet. Regarding issue labels, I'm afraid they can't contain a free form text specific to the issue (and still be searchable, etc.)). Please let me know if I can help in any other way.

I'd also add that we do have a solution for the original problem (listing packages added to an env) on our side: cabal env list (part of #6481). So this is is only a not-our-bug for the ghc-pkg part (maybe I should remove the label...). Please discuss cabal env list on #6481, and ghc-pkg on the ghc issue tracker.