Task: make cabal-testsuite use non-bleeding lib:Cabal for its Setup.hs

[hvr]

A recurring problem everytime we have to bump the major version in cabal is cabal-testsuite's dependency on the lib:Cabal version used for its setup script; in fact, the .cabal file contains the note:

custom-setup
  -- It's important that we pick the exact same version of lib:Cabal
  -- both here and for cabal-tests itself. Without this constraint,
  -- the solver would pick the in-tree Cabal for cabal-tests's
  -- lib:Cabal dependency, and some stable lib:Cabal version for its
  -- custom-setup's one (due to 'setupMaxCabalVersionConstraint' in
  -- 'D.C.ProjectPlanning').
  setup-depends: Cabal == 3.0.0.0,

The problem here is that in order to use a lib:Cabal we also need a corresponding cabal executable that's capable of talking to such a bleeding lib:Cabal-based Setup.hs, and in general, it's not guaranteed that e.g. cabal-install-2.0 is able to able to communicate with a lib:Cabal-3.0-based Setup.hs.

Moreover, it's not possible to perfectly synchronise as cabal treats setup components as qualified goals and we can easily end up e.g. picking a subtly different instance of lib:Cabal than the one that the library component was linked against.

The actual reason for requiring the setup component and the testsuite library/executable component is primarily so that library can read the LocalBuildInfo in order to construct a ScriptEnv,

cabal/cabal-testsuite/Test/Cabal/Script.hs

Lines 34 to 41 in 0c97920

	data ScriptEnv = ScriptEnv
	{ runnerProgramDb :: ProgramDb
	, runnerPackageDbStack :: PackageDBStack
	, runnerVerbosity :: Verbosity
	, runnerPlatform :: Platform
	, runnerCompiler :: Compiler
	, runnerPackages :: [(OpenUnitId, ModuleRenaming)]
	}

and this is constructed via a LBI which was written out by the Setup.hs, at

cabal/cabal-testsuite/Test/Cabal/Monad.hs

Lines 243 to 245 in 0c97920

	lbi <- getPersistBuildConfig dist_dir
	let verbosity = normal -- TODO: configurable
	senv <- mkScriptEnv verbosity lbi

and

cabal/cabal-testsuite/main/cabal-tests.hs

Lines 113 to 116 in e8a8287

	lbi <- getPersistBuildConfig dist_dir
	-- Get ready to go!
	senv <- mkScriptEnv verbosity lbi

and in fact, this is also explained in the Setup.hs file:

import Distribution.Simple
main :: IO ()
main = defaultMain

-- Although this looks like the Simple build type, it is in fact vital that
-- we use this Setup.hs because we need to compile against the very same
-- version of the Cabal library that the test suite will be compiled
-- against.  When this happens, it will mean that we'll be able to
-- read the LocalBuildInfo of our build environment, which we will
-- subsequently use to make decisions about PATHs etc.  Important!

How to resolve the issue

However, if we avoid requiring binary format compatibility for the binary encoded LBI, and instead dump out the minimum needed information to construct the ScriptEnv structure we can avoid needing to synchronise the Cabal version of the setup component with the rest.

[phadej]

@hvr i.e. you suggest that Setup.hs would dump needed LBI as text? and cabal-tests would read that?

[hvr]

@phadej something like that yeah, could even be just via Show/Read; and we only need a very small part of the LBI

See also

cabal/cabal-testsuite/Test/Cabal/Script.hs

Lines 50 to 64 in 0c97920

	-- | Create a 'ScriptEnv' from a 'LocalBuildInfo' configured with
	-- the GHC that we want to use.
	mkScriptEnv :: Verbosity -> LocalBuildInfo -> IO ScriptEnv
	mkScriptEnv verbosity lbi = do
	package_db <- mapM canonicalizePackageDB (withPackageDB lbi)
	return $ ScriptEnv
	{ runnerVerbosity = verbosity
	, runnerProgramDb = withPrograms lbi
	, runnerPackageDbStack = package_db
	, runnerPlatform = hostPlatform lbi
	, runnerCompiler = compiler lbi
	-- NB: the set of packages available to test.hs scripts will COINCIDE
	-- with the dependencies on the cabal-testsuite library
	, runnerPackages = cabalTestsPackages lbi
	}

IOW, the main challenge here is how to encode the ProgrammDB and the PackageDbStack in a portable format. Maybe we don't even need actual ProgrammDB/PackageDBStacks in the testsuite code and might get away with simpler data.

[typedrat]

PackageDbStack is a subset of GHC environment files we already have syntax for and ProgramDB could just be a prog:PATH entry per line, couldn't it?

[hvr]

Actually, I just remembered that the IMO easiest robust approach would in fact be to use Show in order to autogen-modules: a .hs module. That way we avoid the need to clutter the src-dir with autogenerated data nor do we need to come up with heuristic to locate the "dist" folder where the data was generated into.

[phadej]

@hvr yes, that's essentially what cabal-doctest does for solving similar problem.