Cabal-3.8.1.0 depends on too recent process-1.6.14, rendering it unusable with package ghc-9.2.4 and older

[andreasabel]

Lifted from #8489 (comment):

Cabal-3.8.1.0 comes with a constraint process >= 1.6.14 (released 2022) for ghc >= 8.2 (released 2017!). This looks non-sensical to me. It makes cabal trying to reinstall the ghc package, which is bound to fail. In the wild:

• Build with Cabal-3.8 and GHC 9.4 hackage-server#1141 (comment)

cabal-3.6.2.0: Could not resolve dependencies:
[__0] trying: hackage-server-0.5.1 (user goal)
[__1] rejecting: hackage-server:!test (constraint from config file, command line flag, or user target requires opposite flag selection)
[__1] trying: hackage-server:*test
[__2] trying: doctest-parallel-0.2.5 (dependency of hackage-server *test)
[__3] trying: ghc-9.2.4/installed-9.2.4 (dependency of doctest-parallel)
[__4] next goal: Cabal (dependency of hackage-server)
[__4] rejecting: Cabal-3.6.3.0/installed-3.6.3.0 (conflict: hackage-server => Cabal^>=3.8.1.0)
[__4] rejecting: Cabal-3.8.1.0 (conflict: ghc => process==1.6.13.2/installed-1.6.13.2, Cabal => process>=1.6.14.0 && <1.7)

Blame for the process >= 1.6.14 constraint goes to 95cf762. (Ping @robx.)

Sat Jul 9 14:48:25 2022 +0200
Cabal: depend on process-1.6.14.0 to fix waitForProcess error message

This is guarded to be for GHC from 8.2 only, since the process release depends on base-4.10.

This extra constraint seems to have been removed from master in the meantime.

However, it seems like a release of Cabal-3.8 is warranted to render Cabal-3.8 usable with the preinstalled ghc package below 9.4.

[andreasabel]

Reproducer: cabal build -w 9.2.4 on a .cabal with these dependencies:

    build-depends:
        base
      , Cabal == 3.8.1.0
      , ghc

This gives:

Build profile: -w ghc-9.2.4 -O1
In order, the following will be built (use -v for more details):
 - process-1.6.16.0 (lib:process) (requires download & build)
 - ghc-9.2.4 (lib) (requires build)
 - Cabal-3.8.1.0 (lib) (requires build)
...

I attach the toy project as zip file.
issue-8554.zip

[Mikolaj]

@andreasabel: I can't see this bound on https://hackage.haskell.org/package/Cabal nor on master nor 3.8 branch in the repo.

And your log says "cabal-3.6.2.0: Could not resolve dependencies".

Could you investigate some more?

[ulysses4ever]

Cabal has backwards compatibility obligation w.r.t. GHC the program, which it adheres.I don't know of any promises w.r.t. ghc the package.

[ulysses4ever]

Sorry, wrong button and premature send.

So, as it stands, a dependency (if it exists) or constraint that rules out older ghcs doesn't violate anything per se. Maybe we should consider introducing a back-compat promise for ghc (the package)? It may sound reasonable but is it worth it? Beneficiaries of this would be those who depend on both Cabal and ghc. Not a big market, I'd guess. Of course, if there are critical infrastructure packages among those, it may be justified.

[Mikolaj]

Actually, I can see the bound for process on Hackage when I look inside the .cabal file --- Hackage doesn't display the bound on the main page, because of the condition. I'm still puzzled where this bound has been removed on 3.8 branch. Blame shows nothing.

[Mikolaj]

OK, I've found it: #8342

So, it seems this is fixed already, just not released. Apologies for any inconvenience.

[Mikolaj]

Apparently, git blame does not show deletions?

[andreasabel]

@ulysses4ever wrote:

Beneficiaries of this would be those who depend on both Cabal and ghc. Not a big market, I'd guess. Of course, if there are critical infrastructure packages among those, it may be justified.

Does hackage-server qualify here? ;-) There are also doctest and friends that might be affected.

@Mikolaj wrote:

@andreasabel: I can't see this bound on https://hackage.haskell.org/package/Cabal nor on master nor 3.8 branch in the repo.

I also stumbled over this; blame goes to how hackage-server displays conditional dependencies, simply taking the union. The union is bogus, of course, as would be the intersection, the only correct display would be it is complicated.

Apparently, git blame does not show deletions?

Regrettably, yes. It only shows you how each line that survived evolution came to be... The links that died out are missing.

[andreasabel]

Would a release of the following PR, say as Cabal-3.8.1.1, fix the situation?

• Bump package process dependency version to 1.6.15.0 and move from Cabal to cabal-install #8342

[Mikolaj]

Alternatively, I could try removing the bound via a revision on Hackage, but the bound was there for a reason (and it got bumped even higher before it got moved to cabal-install). So I'd need to also add it to cabal-install instead. That's a quick and dirty solution.

[gbaz]

@andreasabel for just making things work for hackage server, we can toss allow-older into the cabal.project file :-)

[andreasabel]

Alternatively, I could try removing the bound via a revision on Hackage, but the bound was there for a reason (and it got bumped even higher before it got moved to cabal-install). So I'd need to also add it to cabal-install instead. That's a quick and dirty solution.

Revisions are subject to some limits, e.g. one cannot add or remove conditionals. Of course, changing the conditional dependency to process -any is as good as removing it.

[andreasabel]

@gbaz wrote:

@andreasabel for just making things work for hackage server, we can toss allow-older into the cabal.project file :-)

Thanks for the hint, this workaround works in any case locally. Not sure if such settings are taken into account by haskell-ci, but I'll try. In any case, such workarounds do not exist for packages pushed to hackage.

[andreasabel]

Not sure if such settings are taken into account by haskell-ci, but I'll try.

The answer to this is negative, unfortunately, allow-newer/older settings (as all settings in the cabal.project file) are ignored (i.e. overwritten) by haskell-ci.

[Mikolaj]

The answer to this is negative, unfortunately, allow-newer/older settings (as all settings in the cabal.project file) are ignored (i.e. overwritten) by haskell-ci.

Yes, I remember having to edit haskell-ci yaml by hand when adding GHC HEAD to some package's CI.

[jneira]

hmm maybe haskell-ci should be improved to allow custom cabal.project config?
btw I think hls is another example of a package depending on both cabal and ghc, so another central piece of Haskell ecosystem could be affected

[ulysses4ever]

haskell-ci doesn't depend on ghc. And the decision to not take into account project files was deliberate afaiu (e.g. it's supposed to test the workflow where you download the package from Hackage).

[jneira]

yeah and maybe it is a good default but it makes haskell-ci not usable in projects like cabal or hls
but it should belong to haskell-ci issue tracker, of course

[andreasabel]

haskell-ci ... the decision to not take into account project files was deliberate afaiu (e.g. it's supposed to test the workflow where you download the package from Hackage).

Yes, this is a stricter CI scenario than building from the repo. I come to appreciate it. It also prevent situations where a maintainer may think that adding dependency relaxations to cabal.project is sufficient---when they actually do not propagate to the Hackage release (cf. coot/ghc-tags-plugin#50 (comment)).

[andreasabel]

Suggested course of action (#8342 (comment)):

• revise the said conditional constraint in Cabal-3.8.1.0 to process -any
• release a cabal-install-3.8.1.1 with just this conditional constraint added over 3.8.1.0

[andreasabel]

Plan in

• Revise process in Cabal-3.8.1.0 and cabal-install-3.8.1.0 #8653

[gbaz]

Just to note that today's discussion suggests "relax the constraint in Cabal, tighten the constraint in cabal-install, just revisions, no releases necessary"

[andreasabel]

Indeed, see my summary at #8653 (comment).

[andreasabel]

This is now fixed (hopefully). I made the respective Hackage revisions, further info at:

• Move process >= 1.16.4 constraint from Cabal-3.8.1.0 to cabal-install-3.8.1.0 #8802

[Mikolaj]

Wow, so it was possible after all? Is it confirmed in the field by now? E.g., do the compilations that failed previously in the wild succeed now in the wild?

[andreasabel]

Well, at least for my use case I was successful: hackage-server can now use doctest-parallel testing even at GHC 9.2 and below.

• Bump CI: GHC 9.2.5 -> 9.2.6, reenable tests for GHC < 9.4 hackage-server#1185

Originally, I didn't think one could revise if impl(ghc >= 8.2) build-depends: process >= 0.16.4. But this was my mental limitation: I didn't consider that one can leave the if in place but make its content trivial, like process >= 0.

[Mikolaj]

Splendid! The dev chats turn out useful in many way.