nix build fails on components which use ghc plugins

[spacekitteh]

While trying to build any of my libraries (or any packages which depend on those libraries) which use GHC plugins, I get the following error:

       > Error: Setup: '/build/tmp.usjCQn4s88/bin/ghc' exited with an error:
       > <command line>: Could not load module ‘Polysemy.Plugin’
       > It is a member of the hidden package ‘polysemy-plugin-0.4.5.1’.
       > Perhaps you need to add ‘polysemy-plugin’ to the build-depends in your .cabal
       > file.
       > Use -v (or `:set -v` in ghci) to see a list of the files searched for.

I obviously have it in the build-depends in the cabal file. The .cabal file looks like this:

cabal-version:      3.4
-- The cabal-version field refers to the version of the .cabal specification,
-- and can be different from the cabal-install (the tool) version and the
-- Cabal (the library) version you are using. As such, the Cabal (the library)
-- version used must be equal or greater than the version stated in this field.
-- Starting from the specification version 2.2, the cabal-version field must be
-- the first thing in the cabal file.

-- Initial package description 'cosmic-modelling' generated by
-- 'cabal init'. For further documentation, see:
--   http://haskell.org/cabal/users-guide/
--
-- The name of the package.
name:               cosmic-modelling

-- The package version.
-- See the Haskell package versioning policy (PVP) for standards
-- guiding when and how versions should be incremented.
-- https://pvp.haskell.org
-- PVP summary:     +-+------- breaking API changes
--                  | | +----- non-breaking API additions
--                  | | | +--- code changes with no API change
version:            0.1.0.0

-- A short (one-line) description of the package.
synopsis:           Domain modelling tools based on `polysemy-methodology`

-- A longer description of the package.
-- description:

-- The license under which the package is released.
license:            AGPL-3.0-or-later

-- The file containing the license text.
license-file:       LICENSE

-- The package author(s).
author:             Sophie Taylor

-- An email address to which users can send suggestions, bug reports, and patches.
maintainer:         [email protected]

-- A copyright notice.
-- copyright:
category:           Development
build-type:         Simple

-- Extra doc files to be distributed with the package, such as a CHANGELOG or a README.
extra-doc-files:    CHANGELOG.md

-- Extra source files to be distributed with the package, such as examples, or a tutorial module.
-- extra-source-files:

common commonStuff
 default-extensions: GADTs, TemplateHaskell, LambdaCase, DeriveGeneric, DeriveAnyClass, LinearTypes,
                     FlexibleContexts, BlockArguments, RecordWildCards, ScopedTypeVariables, DataKinds, KindSignatures, DerivingStrategies, DerivingVia, StandaloneDeriving, MultiParamTypeClasses, TypeApplications,
                     RankNTypes, PolyKinds, AllowAmbiguousTypes, TypeFamilies, FlexibleInstances, FunctionalDependencies, ApplicativeDo, NoFieldSelectors, TypeOperators, OverloadedStrings, ConstraintKinds, DeriveDataTypeable, 
                     UndecidableInstances, DeriveFunctor, DeriveFoldable, DeriveTraversable, MultiWayIf, ViewPatterns,
                     OverloadedLabels, QualifiedDo, RoleAnnotations, TupleSections, QuantifiedConstraints, BangPatterns, UnliftedDatatypes, DeriveLift, QuasiQuotes
 ghc-options: -Wall -fplugin=Polysemy.Plugin
 build-depends: base,polysemy, polysemy-plugin, polysemy-methodology, polysemy-process >= 0.12, polysemy-zoo

library
    import:           commonStuff

    -- Modules exported by the library.
    exposed-modules:  Development.DomainModelling

    -- Modules included in this library but not exported.
    -- other-modules:

    -- LANGUAGE extensions used by modules in this package.
    -- other-extensions:

    -- Other library packages from which modules are imported.
    -- build-depends: 

    -- Directories containing source files.
    hs-source-dirs:   lib

    -- Base language which the package is written in.
    default-language: GHC2021

test-suite cosmic-modelling-test
    import:           commonStuff

    -- Base language which the package is written in.
    default-language: GHC2021

    -- Modules included in this executable, other than Main.
    -- other-modules:

    -- LANGUAGE extensions used by modules in this package.
    -- other-extensions:

    -- The interface type and version of the test suite.
    type:             exitcode-stdio-1.0

    -- Directories containing source files.
    hs-source-dirs:   test

    -- The entrypoint to the test suite.
    main-is:          Main.hs

    -- Test dependencies.
    build-depends:    cosmic-modelling

If I enter a shell with nix develop, and then cabal build, it compiles just fine.

Haskell.nix version: 66191df
GHC version(s): 9.6.2, 9.6.3
NixOS, x86_64-linux

[hamishmack]

It looks like haskell.nix fails running Setup register during the install phase. Adding -v reveals that Setup register runs ghc like this:

Running: /private/tmp/nix-build-cosmic-modelling-lib-cosmic-modelling-0.1.0.0.drv-2/tmp.5BCLy7xIAC/bin/ghc --abi-hash -fbuilding-cabal-package -O -outputdir dist/build -odir dist/build -hidir dist/build -stubdir dist/build -i -idist/build -ilib -idist/build/autogen -idist/build/global-autogen -Idist/build/autogen -Idist/build/global-autogen -Idist/build -optP-include -optPdist/build/autogen/cabal_macros.h -this-unit-id cosmic-modelling-0.1.0.0-Ewe4Gw6q8eX2a5MsGTb26h -hide-all-packages -Wmissing-home-modules -no-user-package-db -XGHC2021 -XGADTs -XTemplateHaskell -XLambdaCase -XDeriveGeneric -XDeriveAnyClass -XLinearTypes -XFlexibleContexts -XBlockArguments -XRecordWildCards -XScopedTypeVariables -XDataKinds -XKindSignatures -XDerivingStrategies -XDerivingVia -XStandaloneDeriving -XMultiParamTypeClasses -XTypeApplications -XRankNTypes -XPolyKinds -XAllowAmbiguousTypes -XTypeFamilies -XFlexibleInstances -XFunctionalDependencies -XApplicativeDo -XNoFieldSelectors -XTypeOperators -XOverloadedStrings -XConstraintKinds -XDeriveDataTypeable -XUndecidableInstances -XDeriveFunctor -XDeriveFoldable -XDeriveTraversable -XMultiWayIf -XViewPatterns -XOverloadedLabels -XQualifiedDo -XRoleAnnotations -XTupleSections -XQuantifiedConstraints -XBangPatterns -XUnliftedDatatypes -XDeriveLift -XQuasiQuotes Development.DomainModelling -Wall '-fplugin=Polysemy.Plugin'

The Setup register is passing -hide-all-packages and so even though the ghc-pkg list does include the polysemy-plugin package, it is hidden.

I'm not sure if we can fix this in haskell.nix or if it needs to be addressed in Cabal. I did find two workarounds:

1. Add -package to the ghc-options in the .cabal file to unhide the package during Setup register:

    ghc-options: -package polysemy-plugin-0.4.5.1 -fplugin=Polysemy.Plugin
   

2. Use a pragma in the .hs files instead of setting ghc-options:

   {-# OPTIONS_GHC -fplugin=Polysemy.Plugin #-}
   

[spacekitteh]

Hmm. Both of those workarounds are incredibly unappealing, but the former can probably be automated by haskell.nix, no?

[hamishmack]

Unfortunately haskell.nix does not interact with the ghc-options in the .cabal file. It just runs Setup configure, Setup build, Setup copy and Setup register in the directory containing the .cabal file. The Cabal code in Setup knows about and uses the ghc-options field.

It is interesting that cabal build is working. I wonder if it runs Setup register. Perhaps cabal build is respecting the cabal-version: 3.4 and using Cabal 3.4 to build a Setup exe to use. Haskell.nix uses the latest Cabal version (3.10 IIRC) for default Setup (used for built-type: Simple).

[hamishmack]

I'm pretty sure cabal-version: 3.4 just indicates the version of the file format though.

[spacekitteh]

It specifies a minimum version :)

[spacekitteh]

Also, I just remembered; this used to work. I had my CI set up such that it would run various flake outputs. Now it doesn't.

[hamishmack]

It looks like the behaviour of ghc has changed (not Cabal). When running ghc --abi-hash any plugin modules passed on the command line must now be present in the package database. This was not the case will older versions of ghc.

This is easy to verify with:

$ ghc-9.4.7 --abi-hash -fplugin=Foo
921dcb31e865da25523dc30d4957f9c4
$ ghc-9.6.3 --abi-hash -fplugin=Foo
<command line>: Could not find module ‘Foo’
Use -v (or `:set -v` in ghci) to see a list of the files searched for.

@hsyl20 does ghc --abi-hash need to find the plugin modules to work correctly? If so does that mean Cabal register should be passing the configured module database arguments to ghc --abi-hash (the same way I presume Cabal build passes them to ghc --make)?

[hsyl20]

@hamishmack It looks like it's the effect of https://gitlab.haskell.org/ghc/ghc/-/commit/18a7d03d46706d2217235d26a72e6f1e82c62192#519f43ef6302b87fec67179ff5343fb9706e9748_261_266
Indeed plugins are loaded much earlier than before to be able to install hooks earlier.

If it breaks cabal register please open a Cabal ticket. I'm not sure it counts as a regression on the GHC side as nothing is really specified anyway...

[hsyl20]

@andreabedini You might be interested in this

[andreabedini]

I opened haskell/cabal#9384, let's see what the CI says.

[m4dc4p]

Is it possible this fix does not work for GHC 9.8? I'm seeing a build failure similar to the one described, and I'm using a much more recent version of haskell.nix (#0dafd67f2e18e76d413cb2fbd0c01d0d028583f3)