Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

-fplugin could not find module when using GHC 9.6 #6251

Open
bristermitten opened this issue Sep 21, 2023 · 25 comments
Open

-fplugin could not find module when using GHC 9.6 #6251

bristermitten opened this issue Sep 21, 2023 · 25 comments

Comments

@bristermitten
Copy link

bristermitten commented Sep 21, 2023

When trying to build a project using polysemy-plugin with GHC 9.6, the build fails with a strange error.

As far as I can tell this is a Stack issue, with the build command not including the package-db and package flags. GHC 9.4.7 works as intended

Steps to reproduce

  1. Clone this repository which contains a minimal reproducible example: https://github.com/knightzmc/psymy
  2. Run stack build
  3. It should throw an error
  4. Change the resolver to lts-21.12
  5. Build again
  6. It should work

Expected

Expected to build without errors or warnings.

Actual

The final stage of building throws an error

Installing library in /private/tmp/psymy/.stack-work/install/aarch64-osx/d648be467e8db3bbfce8aea3e8a40e40c2a4aa997e10a82332008572e1290566/9.6.2/lib/aarch64-osx-ghc-9.6.2/problematic-0.0.0.0-7Tqbt47XADvFA7jcxUQnZt
Error: Cabal-simple_6HauvNHV_3.10.1.0_ghc-9.6.2:
'/Users/alex/.ghcup/ghc/9.6.2/bin/ghc' exited with an error:
<command line>: Could not find module ‘Polysemy.Plugin’
Use -v (or `:set -v` in ghci) to see a list of the files searched for.
                           
Completed 14 action(s).    

Error: [S-7282]
       Stack failed to execute the build plan.
       
       While executing the build plan, Stack encountered the error:
       
       [S-7011]
       While building package problematic-0.0.0.0 (scroll up to its section to see the error) using:
       /Users/alex/.stack/setup-exe-cache/aarch64-osx/Cabal-simple_6HauvNHV_3.10.1.0_ghc-9.6.2 --verbose=1 --builddir=.stack-work/dist/aarch64-osx/Cabal-3.10.1.0 register
       Process exited with code: ExitFailure 1 

Stack version

stack --version
Version 2.11.1, Git revision d06506a16289c6b90e6ffc0dcc8be7f80487ac07 (dirty) aarch64 hpack-0.35.2

Method of installation

  • Via GHCup
  • Also happens with Nix

Platform

MacOS Sonoma aarch64, also reproducible on Ubuntu 22.04.2 LTS x86_64

@mpilgrem
Copy link
Member

@knightzmc, thanks for reporting. With stack --verbose build --cabal-verbosity verbose I get (extract, reformatted):

GHC 9.4.7, with Cabal 3.8.1.0 (works):

[debug - Stack] Run process within C:\Users\mikep\Documents\Code\Haskell\psymy\: 
  C:\sr\setup-exe-cache\x86_64-windows\Cabal-simple_9p6GVs8J_3.8.1.0_ghc-9.4.7.exe 
--verbose=2 
--builddir=.stack-work\dist\74a2d300 
register

[info - Cabal] "C:\Users\mikep\AppData\Local\Programs\stack\x86_64-windows\ghc-9.4.7\bin\ghc-9.4.7.exe" 
"--abi-hash" 
"-fbuilding-cabal-package" 
"-O" 
"-outputdir" ".stack-work\dist\74a2d300\build" 
"-odir" ".stack-work\dist\74a2d300\build" 
"-hidir" ".stack-work\dist\74a2d300\build" 
"-stubdir" ".stack-work\dist\74a2d300\build" 
"-i" 
"-i.stack-work\dist\74a2d300\build" 
"-isrc" 
"-i.stack-work\dist\74a2d300\build\autogen" 
"-i.stack-work\dist\74a2d300\build\global-autogen" 
"-I.stack-work\dist\74a2d300\build\autogen" 
"-I.stack-work\dist\74a2d300\build\global-autogen" 
"-I.stack-work\dist\74a2d300\build" 
"-IC:\Users\mikep\AppData\Local\Programs\stack\x86_64-windows\msys2-20230526\mingw64\include"
"-optP-include" 
"-optP.stack-work\dist\74a2d300\build\autogen\cabal_macros.h" 
"-this-unit-id" "problematic-0.0.0.0-Av9fhBRW0672BVqZNF20AQ" 
"-hide-all-packages" 
"-Wmissing-home-modules" 
"-no-user-package-db" 
"-XGHC2021" 
"-XDataKinds" 
"A" 
"-fplugin=Polysemy.Plugin" 
"-fhide-source-paths"

GHC 9.6.2, with Cabal 3.10.1.0 (fails):

[debug - Stack] Run process within C:\Users\mikep\Documents\Code\Haskell\psymy\: 
  C:\sr\setup-exe-cache\x86_64-windows\Cabal-simple_9p6GVs8J_3.10.1.0_ghc-9.6.2.exe 
--verbose=2 
--builddir=.stack-work\dist\18095c9f 
register

[info - Cabal] Running: "C:\Users\mikep\AppData\Local\Programs\stack\x86_64-windows\ghc-9.6.2\bin\ghc-9.6.2.exe" 
"--abi-hash" 
"-fbuilding-cabal-package" 
"-O" 
"-outputdir" ".stack-work\dist\18095c9f\build" 
"-odir" ".stack-work\dist\18095c9f\build" 
"-hidir" ".stack-work\dist\18095c9f\build" 
"-stubdir" ".stack-work\dist\18095c9f\build" 
"-i" 
"-i.stack-work\dist\18095c9f\build" 
"-isrc" 
"-i.stack-work\dist\18095c9f\build\autogen" 
"-i.stack-work\dist\18095c9f\build\global-autogen" 
"-I.stack-work\dist\18095c9f\build\autogen" 
"-I.stack-work\dist\18095c9f\build\global-autogen" 
"-I.stack-work\dist\18095c9f\build" 
"-IC:\Users\mikep\AppData\Local\Programs\stack\x86_64-windows\msys2-20230526\mingw64\include"
"-optP-include" 
"-optP.stack-work\dist\18095c9f\build\autogen\cabal_macros.h" 
"-this-unit-id" "problematic-0.0.0.0-7Tqbt47XADvFA7jcxUQnZt" 
"-hide-all-packages" 
"-Wmissing-home-modules" 
"-no-user-package-db" 
"-XGHC2021" 
"-XDataKinds" 
"A" 
"-fplugin=Polysemy.Plugin" 
"-fhide-source-paths"

[warn] CallStack (from HasCallStack):
[warn]   withMetadata, called at libraries\Cabal\Cabal\src\Distribution\Simple\Utils.hs:368:14 in Cabal-3.10.1.0:Distribution.Simple.Utils
[warn] Error: Cabal-simple_9p6GVs8J_3.10.1.0_ghc-9.6.2.exe:
[warn] 'C:\Users\mikep\AppData\Local\Programs\stack\x86_64-windows\ghc-9.6.2\bin\ghc-9.6.2.exe'
[warn] exited with an error:
[warn] <command line>: Could not find module ‘Polysemy.Plugin’
[warn] Use -v (or `:set -v` in ghci) to see a list of the files searched for.
[warn]

It looks to me that Stack is giving identical commands to Cabal (the tool) and it, in turn, is giving the same command to different versions of GHC. The next step is to turn on GHC's verbosity ...

@bristermitten
Copy link
Author

bristermitten commented Sep 21, 2023

Ahh sorry, I totally forgot the verbose log!
I was getting a totally unrelated error when using Cabal (the tool) which I assumed was just an issue with my machine, so perhaps this isn't a Stack issue at all. However when using Cabal (as in, cabal build) 3.10.1.0 and GHC 9.6.2 on x86_64 Linux it works as intended, whereas stack build fails.

The logs of both can be found here, https://gist.github.com/knightzmc/77e219e582242362b2d7d1370b529d28
There's a lot to break down but it definitely seems like Cabal is passing -package-id polysemy-1.9.1.2-03b9744b5a6b09461e989de3b3caa039034922d5f496217d0a114f5184ee5eed (and so on) to GHC whereas Stack is not

@mpilgrem
Copy link
Member

With GHC's verbosity turned on, I see in the GHC error message:

[warn] Locations searched:
[warn] .stack-work\dist\18095c9f\build\Polysemy\Plugin.hi
[warn] .stack-work\dist\18095c9f\build\Polysemy\Plugin.hi-boot

That does not exist in \18095c9f\build - but it does not exist in \74a2d300\build (GHC 9.4.7, Cabal 3.8.1.0) either.

@mpilgrem
Copy link
Member

Looking earlier in Stack's verbose output:

GHC 9.4.7 (extracts only):

Run process within C:\Users\mikep\Documents\Code\Haskell\psymy\: 
  C:\sr\setup-exe-cache\x86_64-windows\Cabal-simple_9p6GVs8J_3.8.1.0_ghc-9.4.7.exe
configure
--with-ghc=C:\Users\mikep\AppData\Local\Programs\stack\x86_64-windows\ghc-9.4.7\bin\ghc-9.4.7.exe
--dependency=base=base-4.17.2.0 
--dependency=polysemy=polysemy-1.9.1.1-LLER3VJ1tZ5CADZn3WfGsD 
--dependency=polysemy-plugin=polysemy-plugin-0.4.5.0-1vYmXOH4oflIZAGBHzZ4KA 

GHC 9.6.2 (extracts only):

Run process within C:\Users\mikep\Documents\Code\Haskell\psymy\: 
  C:\sr\setup-exe-cache\x86_64-windows\Cabal-simple_9p6GVs8J_3.10.1.0_ghc-9.6.2.exe 
configure 
--with-ghc=C:\Users\mikep\AppData\Local\Programs\stack\x86_64-windows\ghc-9.6.2\bin\ghc-9.6.2.exe
--dependency=base=base-4.18.0.0 
--dependency=polysemy=polysemy-1.9.1.2-KKWKK0OMtIqEc5FzcWRYKA 
--dependency=polysemy-plugin=polysemy-plugin-0.4.5.1-CCks1M1SJDYB4s9lu9bde

Stack seems to be configuring Cabal (the tool) in an identical way - save for the change in the versions of polysemy and polysemy-plugin.

@mpilgrem
Copy link
Member

mpilgrem commented Sep 21, 2023

Thinking out loud: why is GHC 9.6.2 looking for (and, of course, failing to find) Plugin.hi in .stack-work when the package polysemy-plugin is immutable? The Plugin.hi file is to be found in the snapshot.

EDIT: I am stumped. ghc-pkg 9.6.2 knows where to find Polysemy.Plugin in the Stack environment, why does GHC 9.6.2 not know?

GHC 9.4.7:

❯ stack exec -- ghc-pkg find-module Polysemy.Plugin
C:\Users\mikep\AppData\Local\Programs\stack\x86_64-windows\ghc-9.4.7\lib\package.conf.d
    (no packages)
C:\sr\snapshots\4ef858c3\pkgdb
    polysemy-plugin-0.4.5.0

C:\Users\mikep\Documents\Code\Haskell\psymy\.stack-work\install\c3501600\pkgdb
    (no packages)

GHC 9.6.2:

❯ stack exec -- ghc-pkg find-module Polysemy.Plugin
C:\Users\mikep\AppData\Local\Programs\stack\x86_64-windows\ghc-9.6.2\lib\package.conf.d
    (no packages)
C:\sr\snapshots\d12bd16e\pkgdb
    polysemy-plugin-0.4.5.1

C:\Users\mikep\Documents\Code\Haskell\psymy\.stack-work\install\d9daf07f\pkgdb
    (no packages)

@mpilgrem
Copy link
Member

I pinned some of the dependencies to rule that out as the cause:

resolver: nightly-2023-09-22 # alternatively lts-21.12

extra-deps:
- Cabal-3.8.1.0
- Cabal-syntax-3.8.1.0
- polysemy-1.9.1.2
- polysemy-plugin-0.4.5.1

@andreabedini
Copy link

It is the same issue as haskell/cabal#9375, it was fixed two weeks ago in haskell/cabal#9384.

There was an old workaround for an issue with ghc-cabal (which is now gone). The workaround was to not pass -package-db to GHC when calling --abi-hash. It used to be unnecessary but with GHC 9.6 the plugin initialisation happens earlier than before, and this means we do need to pass -package-db.

@gergoerdi
Copy link

So if this has been fixed in Cabal, what can I, as a user of Stack, do to apply that Cabal fix?

@christiaanb
Copy link

@gergoerdi as a work-around you can:

  1. add the latest 3.10 branch of cabal to the extra-deps in your stack.yaml:
 - git: https://github.com/haskell/cabal.git
   commit: a3865991986361b3a736007f620b6a8878d178e3
   subdirs:
   - Cabal
  1. Change the build-type of your package's .cabal file to Custom
  2. Add a custom-setup stanza containing:
custom-setup
  setup-depends:
    base,
    Cabal >= 3.10.2.1

Very ugly, but it works.

@mbj
Copy link
Contributor

mbj commented Feb 14, 2024

@christiaanb Thank you!

But it did not work for me, before I sink a lot of time (you probably have already spend), do you happen to have a public project here this worked for you so I can check whatever I have done differently? I checked your clash repos but did not see this specific solution (or I'm bad in searching which may very well be the case!).

@christiaanb
Copy link

@mbj we have a variant of the fix at this PR: clash-lang/clash-compiler#2665

And it’s passing CI with GHC 9.6.4: https://github.com/clash-lang/clash-compiler/actions/runs/7886581662/job/21520111671?pr=2665

@mbj
Copy link
Contributor

mbj commented Feb 14, 2024

@christiaanb ty, I cannot yet spot the difference to what I tried, but will keep digging!

@telser
Copy link

telser commented Mar 7, 2024

It is the same issue as haskell/cabal#9375, it was fixed two weeks ago in haskell/cabal#9384.

There was an old workaround for an issue with ghc-cabal (which is now gone). The workaround was to not pass -package-db to GHC when calling --abi-hash. It used to be unnecessary but with GHC 9.6 the plugin initialisation happens earlier than before, and this means we do need to pass -package-db.

This would seem to say the fix is in Cabal. @mpilgrem Are there dragons lurking in updating the dependency to 3.10.2.1? This is the blocker for getting to 9.6 right now. Would be happy to give that update a shot, unless there is some reason to hold off.

@mpilgrem
Copy link
Member

mpilgrem commented Mar 7, 2024

@telser, (EDIT: Unless a custom Setup.hs is specified) Stack uses the version of Cabal (the library) that is provided with the specified version of GHC - the boot library. I do not think that is configurable (currently) (EDIT: unless a custom Setup.hs is used). It may be work-around-able, but I would have to think about that. At the moment, there is no released version of GHC that comes with Cabal >= 3.10.2.1.

EDIT: I've not looked into the full history of that design choice (to use only GHC's Cabal boot library) but some of it is here: #4070. The reason given was 'You shouldn't generally do this [change the boot library version of Cabal], since new Cabal versions may introduce incompatibilities with package sets'.

@telser
Copy link

telser commented Mar 7, 2024

@telser, Stack uses the version of Cabal (the library) that is provided with the specified version of GHC - the boot library. I do not think that is configurable (currently). It may be work-around-able, but I would have to think about that. At the moment, there is no released version of GHC that comes with Cabal >= 3.10.2.1.

EDIT: I've not looked into the full history of that design choice (to use only GHC's Cabal boot library) but some of it is here: #4070. The reason given was 'You shouldn't generally do this [change the boot library version of Cabal], since new Cabal versions may introduce incompatibilities with package sets'.

@mpilgrem I see. If I'm understanding correctly then it really is ghc-pkg isn't working correctly since 9.6.1. Meaning ghc has been shipping with a fundamentally broken configuration for the last 2 major releases. It would seem that we need then the next point releases of 9.6 and 9.8 to bump the cabal versions.

EDIT: That is to say the situation is certainly frustrating for users. I haven't been able to find an issue for this on the ghc side, so I'll raise one there. The optics for this are frustrating because for users they can easily get the impression the issue is with the plugin(s) they use or stack or stack's integration with Cabal. With the better understanding though it feels like it is ghc-pkg. I'd love to be able to add a test case for this to ghc itself, but where/how to do that is unclear to me at this point.

@telser
Copy link

telser commented Mar 14, 2024

For anyone watching this, there is a ghc issue: https://gitlab.haskell.org/ghc/ghc/-/issues/24518

@mbj
Copy link
Contributor

mbj commented Mar 21, 2024

For anyone watching this: The cabal version with the fix has a release now. 3.10.3.0

tonosaman added a commit to tonosaman/haskell-dev-env that referenced this issue Mar 28, 2024
ghcup 0.1.22.0   latest,recommended
ghc   9.6.4      base-4.18.2.0             hls-powered
cabal 3.10.2.1   latest,recommended
stack 2.13.1     recommended

- How to start

  ```bash
  git clone https://github.com/tonosaman/haskell-dev-env
  cd haskell-dev-env
  stack new --resolver lts-22.13 clash-project clash-lang/simple
  ```

- [BUG]`stack build` is failed
  "`-fplugin` could not find module when using GHC 9.6 #6251"
  commercialhaskell/stack#6251

  Use cabal for build project
  `cabal run clash -- Example.Project --systemverilog`
@mpilgrem
Copy link
Member

GHC 9.10.0.20240328 comes with Cabal-3.11.0.0 >= 3.10.3.0, but ghc-tcplugins-extra, a direct dependency of polysemy-plugin, does not yet support GHC 9.10.

@goertzenator
Copy link

I've worked around this by just disabling the plugin (effectful-plugin in my case) for now. I had to add exactly one type annotation to get my code base to compile, so not a big deal. YMMV of course.

@gergoerdi
Copy link

@gergoerdi as a work-around you can:

  1. add the latest 3.10 branch of cabal to the extra-deps in your stack.yaml:
 - git: https://github.com/haskell/cabal.git
   commit: a3865991986361b3a736007f620b6a8878d178e3
   subdirs:
   - Cabal
  1. Change the build-type of your package's .cabal file to Custom
  2. Add a custom-setup stanza containing:
custom-setup
  setup-depends:
    base,
    Cabal >= 3.10.2.1

Very ugly, but it works.

Unfortunately, now that I've tried this, it still fails with the same Could not find module ‘GHC.TypeLits.KnownNat.Solver’ error message.

@gergoerdi
Copy link

Unfortunately, now that I've tried this, it still fails with the same Could not find module ‘GHC.TypeLits.KnownNat.Solver’ error message.

Unless... maybe this needs to be applied to all Clash-using dependencies as well, not just the main program? @christiaanb I'll try that out.

@mpilgrem
Copy link
Member

I could also get @knightzmc's minimal example to work with GHC 9.6.4 by:

  1. Adding a Setup.hs:

    import Distribution.Simple
    main = defaultMain
  2. Using a custom Setup.hs built against Cabal-3.10.3.0 and Cabal-syntax-3.10.3.0.

    package.yaml (extract):

    custom-setup:
      dependencies:
      - base
      - Cabal ==3.10.3.0
      - Cabal-syntax ==3.10.3.0

    stack.yaml:

    snapshot: lts-22.17 # GHC 9.6.4
    extra-deps:
    - Cabal-3.10.3.0
    - Cabal-syntax-3.10.3.0

@hasufell
Copy link
Contributor

If I'm understanding correctly then it really is ghc-pkg isn't working correctly since 9.6.1. Meaning ghc has been shipping with a fundamentally broken configuration for the last 2 major releases.

@telser can you expand on that? I don't see any mention of ghc-pkg in the GHC issue you linked.

@telser
Copy link

telser commented Apr 15, 2024

If I'm understanding correctly then it really is ghc-pkg isn't working correctly since 9.6.1. Meaning ghc has been shipping with a fundamentally broken configuration for the last 2 major releases.

@telser can you expand on that? I don't see any mention of ghc-pkg in the GHC issue you linked.

My, current, understanding is that ghc-pkg is what makes Cabal a boot library. But this was really my swapping of ghc-pkg and GHC in reading the following comment from earlier in this thread:

Thinking out loud: why is GHC 9.6.2 looking for (and, of course, failing to find) Plugin.hi in .stack-work when the package polysemy-plugin is immutable? The Plugin.hi file is to be found in the snapshot.

EDIT: I am stumped. ghc-pkg 9.6.2 knows where to find Polysemy.Plugin in the Stack environment, why does GHC 9.6.2 not know?

GHC 9.4.7:

❯ stack exec -- ghc-pkg find-module Polysemy.Plugin
C:\Users\mikep\AppData\Local\Programs\stack\x86_64-windows\ghc-9.4.7\lib\package.conf.d
    (no packages)
C:\sr\snapshots\4ef858c3\pkgdb
    polysemy-plugin-0.4.5.0

C:\Users\mikep\Documents\Code\Haskell\psymy\.stack-work\install\c3501600\pkgdb
    (no packages)

GHC 9.6.2:

❯ stack exec -- ghc-pkg find-module Polysemy.Plugin
C:\Users\mikep\AppData\Local\Programs\stack\x86_64-windows\ghc-9.6.2\lib\package.conf.d
    (no packages)
C:\sr\snapshots\d12bd16e\pkgdb
    polysemy-plugin-0.4.5.1

C:\Users\mikep\Documents\Code\Haskell\psymy\.stack-work\install\d9daf07f\pkgdb
    (no packages)

@mpilgrem
Copy link
Member

I could get @knightzmc's minimal example to work with GHC 9.6.5, with:

stack.yaml:

snapshot: lts-22.17 # GHC 9.6.4
compiler: ghc-9.6.5 # Comes with Cabal-3.10.3.0

# Temporary, until GHC 9.6.5 is added to Stack's default setup-info dictionary:
setup-info:
  ghc:
    windows64:
      9.6.5:
        url: https://downloads.haskell.org/ghc/9.6.5/ghc-9.6.5-x86_64-unknown-mingw32.tar.xz

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

9 participants