cabal-install >= 3.4 regression: `cabal repl -b` fails to install packages in combination with `--allow-newer` #7081

RyanGlScott · 2020-09-20T15:37:51Z

To reproduce this, pick a package that you haven't installed yet, and try the following with cabal-install-3.4:

$ /opt/cabal/3.4/bin/cabal --version
cabal-install version 3.4.0.0
compiled using version 3.4.0.0 of the Cabal library

$ /opt/cabal/3.4/bin/cabal repl -b distributive -w /opt/ghc/8.10.2/bin/ghc --allow-newer=base
Resolving dependencies...
Build profile: -w ghc-8.10.2 -O1
In order, the following will be built (use -v for more details):
 - fake-package-0 (lib) (first run)
Configuring library for fake-package-0..
Preprocessing library for fake-package-0..

Notice that distributive isn't built at all! Note that the use of the distributive and base packages aren't important here. You can substitute them with any other packages and you should still observe a similar effect.

The bug does not occur if:

--allow-newer is omitted:

$ /opt/cabal/3.4/bin/cabal repl -b distributive -w /opt/ghc/8.10.2/bin/ghc
Resolving dependencies...
Build profile: -w ghc-8.10.2 -O1
In order, the following will be built (use -v for more details):
 - base-orphans-0.8.2 (lib) (requires build)
 - cabal-doctest-1.0.8 (lib) (requires build)
 - tagged-0.8.6 (lib) (requires build)
 - distributive-0.6.2 (lib:distributive) (requires build)
 - fake-package-0 (lib) (first run)

You use install instead of repl -b:

$ /opt/cabal/3.4/bin/cabal install distributive -w /opt/ghc/8.10.2/bin/ghc --allow-newer=base
Resolving dependencies...
Build profile: -w ghc-8.10.2 -O1
In order, the following will be built (use -v for more details):
 - base-orphans-0.8.2 (lib) (requires build)
 - cabal-doctest-1.0.8 (lib) (requires build)
 - tagged-0.8.6 (lib) (requires build)
 - distributive-0.6.2 (lib:distributive) (requires build)

You use cabal-install-3.2 instead of cabal-install-3.4:

$ /opt/cabal/3.2/bin/cabal --version
cabal-install version 3.2.0.0
compiled using version 3.2.0.0 of the Cabal library

$ /opt/cabal/3.2/bin/cabal repl -b distributive -w /opt/ghc/8.10.2/bin/ghc --allow-newer=base
Resolving dependencies...
Build profile: -w ghc-8.10.2 -O1
In order, the following will be built (use -v for more details):
 - base-orphans-0.8.2 (lib) (requires build)
 - cabal-doctest-1.0.8 (lib) (requires build)
 - tagged-0.8.6 (lib) (requires build)
 - distributive-0.6.2 (lib:distributive) (requires build)
 - fake-package-0 (lib) (first run)

The text was updated successfully, but these errors were encountered:

hvr · 2020-09-20T21:02:24Z

Quick note:

I've been able to reproduce this with 5139d6e

Also, you can use :show packages inside GHCi to see which packages were provided to GHCi by cabal, and in the cases above with --allow-newer=base GHCi's :show packages would show merely base

phadej · 2020-10-02T13:01:26Z

I looked into this, from quick look, the -b flag is just a huge HACK and I'm not surprised it broke. With better extra-packages (#6972) we can (hopefully) avoid that hack, which I'm now exploring. The downside is that I'm not keen in backporting that to 3.4, I'm ready to declare repl -b pkg broken in 3.4: hack have bitrotted.

RyanGlScott · 2020-10-02T13:25:25Z

The downside is that I'm not keen in backporting that to 3.4, I'm ready to declare repl -b pkg broken in 3.4

This leaves me in a somewhat awkward spot, as cabal repl -b <...> --allow-newer=<...> is something that I use every day, and I'm not really aware of a workaround. Do you have one in mind?

phadej · 2020-10-02T15:02:09Z

@RyanGlScott I don't know. Someone can look (bisect) what caused the breakage, and see if there is small backportable change.

The code in question is in

cabal/cabal-install/src/Distribution/Client/CmdRepl.hs

Lines 214 to 231 in f6682ef

    
           (originalComponent, baseCtx') <- if null (envPackages envFlags) 
        
             then return (Nothing, baseCtx) 
        
             else 
        
               -- Unfortunately, the best way to do this is to let the normal solver 
        
               -- help us resolve the targets, but that isn't ideal for performance, 
        
               -- especially in the no-project case. 
        
               withInstallPlan (lessVerbose verbosity) baseCtx $ \elaboratedPlan _ -> do 
        
                 -- targets should be non-empty map, but there's no NonEmptyMap yet. 
        
                 targets <- validatedTargets elaboratedPlan targetSelectors 
        
                 let 
        
                   (unitId, _) = fromMaybe (error "panic: targets should be non-empty") $ safeHead $ Map.toList targets 
        
                   originalDeps = installedUnitId <$> InstallPlan.directDeps elaboratedPlan unitId 
        
                   oci = OriginalComponentInfo unitId originalDeps 
        
                   pkgId = fromMaybe (error $ "cannot find " ++ prettyShow unitId) $ packageId <$> InstallPlan.lookup elaboratedPlan unitId 
        
                   baseCtx' = addDepsToProjectTarget (envPackages envFlags) pkgId baseCtx 
        
                 return (Just oci, baseCtx')

And it just screams for "make extra-packages work", which I'd rather concentrate on.

phadej · 2020-10-09T15:29:41Z

Investigating further, making repl principled is tricky, yet not impossible.

Since Consider Installed packages when pruning the install plan #6972 is now in, extra-packages seem to work. However even we add -b packages to extra-packages
We need also invoke solver with them as a target (and currently targetSelectors don't include them)
And invoke repl further with them in scope.

In principle this is easier (but not easy) for non-project setting, as there all packages considered are loaded, not interpreted.

However in the project case this is difficult, as -b package may depend on project package, which we want to interpret.
That scenario cannot work, and we have to handle it. OTOH, there should be a mode of operation to compile project packages, and load them, then -b would work for any extra package.

At the moment I don't see a quick way out, otherwise then something close to "rewrite everything". I'll leave this for now, until the other refactoring might make tackling this easier.

hasufell · 2021-08-08T09:29:28Z

This popped up again today on IRC. Have the recent refactorings of cabal changed anything?

tomsmeding · 2021-08-08T09:32:59Z

Perhaps worth mentioning is that this behaviour doesn't actually need a non-installed package; it works with parsec too for me.

$ cabal repl -b parsec --allow-newer=base
[...]
Prelude> :show packages
active package flags:
  -package-id base-4.14.1.0
Prelude>

$ cabal repl -b parsec
[...]
Prelude> :show packages
active package flags:
  -package-id pretty-1.1.3.6
  -package-id ghc-boot-th-8.10.4
  -package-id template-haskell-2.16.0.0
  -package-id containers-0.6.2.1
  -package-id binary-0.8.8.0
  -package-id text-1.2.4.1
  -package-id transformers-0.5.6.2
  -package-id mtl-2.2.2
  -package-id array-0.5.4.0
  -package-id deepseq-1.4.4.0
  -package-id bytestring-0.10.12.0
  -package-id integer-gmp-1.0.3.0
  -package-id rts
  -package-id ghc-prim-0.6.1
  -package-id parsec-3.1.14.0
  -package-id base-4.14.1.0
Prelude>

$ cabal --version
cabal-install version 3.4.0.0
compiled using version 3.4.0.0 of the Cabal library

$ ghc --version
The Glorious Glasgow Haskell Compilation System, version 8.10.4

Mikolaj · 2021-08-09T07:11:04Z

Supposedly #6972 fixed this. Could you try with master branch?

tomsmeding · 2021-08-09T08:48:12Z

@Mikolaj Can still reproduce, but perhaps I did something wrong in compiling Cabal from master?

/large$ git clone [email protected]:haskell/cabal
[...]

/large$ cd cabal

/large/cabal$ git show
commit a8d47c3c71767414add8177b538f1783528f5cd6 (HEAD -> master, origin/master, origin/HEAD)
Merge: bd5e3f4f2 91195e844
Author: Mikolaj Konarski <[email protected]>
Date:   Sat Aug 7 23:41:46 2021 +0200

    Merge pull request #7510 from Mikolaj/master

    Set PATH_SEPARATOR=; when calling ./configure; fix #7494

/large/cabal$ cabal build --project-file=cabal.project.release cabal-install
Resolving dependencies...
cabal: Could not resolve dependencies:
[__0] trying: Cabal-3.6.0.0 (user goal)
[__1] trying: cabal-install-3.6.0.0 (user goal)
[__2] next goal: hackage-security (dependency of cabal-install)
[__2] rejecting: hackage-security-0.6.0.1 (conflict: Cabal==3.6.0.0,
hackage-security => Cabal>=1.14 && <1.26 || >=2.0 && <2.6 || >=3.0 && <3.6)
[__2] skipping: hackage-security-0.6.0.0, hackage-security-0.5.3.0,
hackage-security-0.5.2.2, hackage-security-0.5.2.1, hackage-security-0.5.2.0,
hackage-security-0.5.1.0, hackage-security-0.5.0.2, hackage-security-0.5.0.1,
hackage-security-0.5.0.0, hackage-security-0.3.0.0, hackage-security-0.2.0.0,
hackage-security-0.1.0.0 (has the same characteristics that caused the
previous version to fail: excludes 'Cabal' version 3.6.0.0)
[__2] fail (backjumping, conflict set: Cabal, cabal-install, hackage-security)
After searching the rest of the dependency tree exhaustively, these were the
goals I've had most trouble fulfilling: Cabal, cabal-install, hackage-security


/large/cabal$ cabal build --project-file=cabal.project.release cabal-install --allow-newer=Cabal
[...]
Linking /large/cabal/dist-newstyle/build/x86_64-linux/ghc-8.10.4/cabal-install-3.6.0.0/x/cabal/build/cabal/cabal ...

/large/cabal$ cd

~$ /large/cabal/dist-newstyle/build/x86_64-linux/ghc-8.10.4/cabal-install-3.6.0.0/x/cabal/build/cabal/cabal repl -b parsec --allow-newer=base
Resolving dependencies...
Build profile: -w ghc-8.10.4 -O1
In order, the following will be built (use -v for more details):
 - fake-package-0 (lib) (first run)
Configuring library for fake-package-0..
Preprocessing library for fake-package-0..
Warning: No exposed modules
GHCi, version 8.10.4: https://www.haskell.org/ghc/  :? for help
Loaded GHCi configuration from /tmp/cabal-repl.-219759/setcwd.ghci
Prelude> :show packages
active package flags:
  -package-id base-4.14.1.0
Prelude>
Leaving GHCi.

~$ /large/cabal/dist-newstyle/build/x86_64-linux/ghc-8.10.4/cabal-install-3.6.0.0/x/cabal/build/cabal/cabal repl -b parsec
Resolving dependencies...
Build profile: -w ghc-8.10.4 -O1
In order, the following will be built (use -v for more details):
 - fake-package-0 (lib) (first run)
Configuring library for fake-package-0..
Preprocessing library for fake-package-0..
Warning: No exposed modules
GHCi, version 8.10.4: https://www.haskell.org/ghc/  :? for help
Loaded GHCi configuration from /tmp/cabal-repl.-219911/setcwd.ghci
Prelude> :show packages
active package flags:
  -package-id pretty-1.1.3.6
  -package-id ghc-boot-th-8.10.4
  -package-id template-haskell-2.16.0.0
  -package-id containers-0.6.2.1
  -package-id binary-0.8.8.0
  -package-id text-1.2.4.1
  -package-id transformers-0.5.6.2
  -package-id mtl-2.2.2
  -package-id array-0.5.4.0
  -package-id deepseq-1.4.4.0
  -package-id bytestring-0.10.12.0
  -package-id integer-gmp-1.0.3.0
  -package-id rts
  -package-id ghc-prim-0.6.1
  -package-id parsec-3.1.14.0
  -package-id base-4.14.1.0
Prelude>
Leaving GHCi.

~$

EDIT: note that you need to horizontally scroll this code block to see the full command lines sometimes

Mikolaj · 2021-08-09T09:04:04Z

@tomsmeding: If I didn't break anything, if you did cabal update, you wouldn't need --allow-newer=Cabal.

But you are right, this is not fixed, I jumped to conclusions and #7081 (comment) says the merged PR was only the first step. Somebody needs to move this further...

tomsmeding · 2021-08-09T09:18:13Z

@Mikolaj Indeed, I was missing a cabal update, but as you say, that doesn't fix the inconsistency reported in this issue. Thanks for checking :)

wismill · 2023-02-05T11:51:26Z

Could you give #8732 a try?

tomsmeding · 2023-02-05T12:39:21Z

@wismill Seems to work for me! Only limited testing, though.

ulysses4ever · 2023-02-05T13:18:52Z

So, this is a dup of #6859

Mikolaj · 2023-02-06T08:41:58Z

Let me close in that case, but please kindly reopen or open a new issue if the problem persists after #8732 is merged.

hvr added cabal-install: cmd/repl type: bug type: regression labels Sep 20, 2020

phadej self-assigned this Oct 2, 2020

phadej mentioned this issue Oct 3, 2020

Using cabal install in a project or install/repl without a project, duplicates global config and rebuilds libs twice in the store #6906

Closed

jneira changed the title ~~cabal-install-3.4 regression: cabal repl -b fails to install packages in combination with --allow-newer~~ cabal-install >= 3.4 regression: cabal repl -b fails to install packages in combination with --allow-newer Mar 6, 2022

ulysses4ever unassigned phadej Feb 5, 2023

Mikolaj closed this as completed Feb 6, 2023

ulysses4ever mentioned this issue Feb 6, 2023

Fix repl discarding --build-depends after allow-newer/older #8732

Merged

3 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

cabal-install >= 3.4 regression: `cabal repl -b` fails to install packages in combination with `--allow-newer` #7081

cabal-install >= 3.4 regression: `cabal repl -b` fails to install packages in combination with `--allow-newer` #7081

RyanGlScott commented Sep 20, 2020 •

edited

Loading

hvr commented Sep 20, 2020

phadej commented Oct 2, 2020 •

edited

Loading

RyanGlScott commented Oct 2, 2020

phadej commented Oct 2, 2020 •

edited

Loading

phadej commented Oct 9, 2020

hasufell commented Aug 8, 2021

tomsmeding commented Aug 8, 2021

Mikolaj commented Aug 9, 2021 •

edited

Loading

tomsmeding commented Aug 9, 2021 •

edited

Loading

Mikolaj commented Aug 9, 2021

tomsmeding commented Aug 9, 2021

wismill commented Feb 5, 2023

tomsmeding commented Feb 5, 2023

ulysses4ever commented Feb 5, 2023

Mikolaj commented Feb 6, 2023

cabal-install >= 3.4 regression: cabal repl -b fails to install packages in combination with --allow-newer #7081

cabal-install >= 3.4 regression: cabal repl -b fails to install packages in combination with --allow-newer #7081

Comments

RyanGlScott commented Sep 20, 2020 • edited Loading

hvr commented Sep 20, 2020

phadej commented Oct 2, 2020 • edited Loading

RyanGlScott commented Oct 2, 2020

phadej commented Oct 2, 2020 • edited Loading

phadej commented Oct 9, 2020

hasufell commented Aug 8, 2021

tomsmeding commented Aug 8, 2021

Mikolaj commented Aug 9, 2021 • edited Loading

tomsmeding commented Aug 9, 2021 • edited Loading

Mikolaj commented Aug 9, 2021

tomsmeding commented Aug 9, 2021

wismill commented Feb 5, 2023

tomsmeding commented Feb 5, 2023

ulysses4ever commented Feb 5, 2023

Mikolaj commented Feb 6, 2023

cabal-install >= 3.4 regression: `cabal repl -b` fails to install packages in combination with `--allow-newer` #7081

cabal-install >= 3.4 regression: `cabal repl -b` fails to install packages in combination with `--allow-newer` #7081

RyanGlScott commented Sep 20, 2020 •

edited

Loading

phadej commented Oct 2, 2020 •

edited

Loading

phadej commented Oct 2, 2020 •

edited

Loading

Mikolaj commented Aug 9, 2021 •

edited

Loading

tomsmeding commented Aug 9, 2021 •

edited

Loading