From 5d3618a144991cbe04126abac4e68e7ce1b56df3 Mon Sep 17 00:00:00 2001 From: Kristen Kozak Date: Wed, 10 Jan 2018 00:02:16 -0800 Subject: [PATCH] Rerun dependency solver to generate a better error message (issue #4823). This commit changes the way that the solver generates the summarized log that it displays at normal verbosity. Previously, the solver saved the full log from the start to the first backjump. Then it filtered the log using the conflict set from the node where the first backjump occurred, i.e., it removed all lines from the log that did not relate to variables in the conflict set. The solver also printed the final conflict set at the end of the log. This approach had several problems: 1. It was possible for the conflicts at the first backjump to be unrelated to the final conflict set (issue #941). The conflicts in the summarized log could be irrelevant to the failure, for example, if they were caused by only a single version of a dependency, which the solver could skip, and the real problem was a different dependency that was missing from the index. Even if the summarized log was relevant, showing two different explanations for the same failure could be confusing. 2. Filtering the full log was error prone and could remove the wrong lines. It caused bugs mentioned in #2853 and #4154. 3. The conflict set at the first backjump contains the variables directly involved in the conflicts at that level and the variables that introduced them, but it doesn't contain the whole chain of variables starting with the user targets (issue #4792). When the log is filtered with that conflict set, it can be unclear why the solver needed to choose the conflicting packages in the first place. This commit creates the summarized log by rerunning the solver with a backjump limit of zero and using the full log. Using an unfiltered log avoids (2) and (3). However, it is also important to shorten the log by only showing choices that are relevant to conflicts. This commit uses different approaches for the two types of solver failures. No solution: This commit makes the solver prefer variables from the first run's final conflict set when choosing goals in the second run. This means that the log to the first backjump is more likely to be relevant to the final failure, because it only mentions packages, flags, and stanzas from the final conflict set. Backjump limit reached: There is no final conflict set in this case, since the solver did not traverse the whole tree. This commit tries to create a final conflict set by rerunning the solver with a subtree of the original search tree that contains the path to the first backjump. Then it uses the final conflict set from that run to generate a log message, as in the case where the solver found that there was no solution. Here is an example of the differences between the new and old logs, using the command from issue #4792 and GHC 8.2.1: Before: $ cabal install --dry-run --index-state=2018-01-04T21:05:55Z thorn Resolving dependencies... cabal: Could not resolve dependencies: trying: base-4.10.0.0/installed-4.1... (dependency of thorn) next goal: profunctors (dependency of thorn) rejecting: profunctors-5.2.1, profunctors-5.2, profunctors-5.1.2, profunctors-5.1.1, profunctors-5.1, profunctors-5.0.1, profunctors-5.0.0.1, profunctors-5 (conflict: thorn => profunctors<5) trying: profunctors-4.4.1 next goal: transformers (dependency of profunctors) rejecting: transformers-0.5.2.0/installed-0.5..., transformers-0.5.5.0, transformers-0.5.4.0, transformers-0.5.2.0, transformers-0.5.1.0, transformers-0.5.0.1, transformers-0.5.0.0 (conflict: profunctors => transformers>=0.2 && <0.5) rejecting: transformers-0.4.3.0, transformers-0.4.2.0 (conflict: base==4.10.0.0/installed-4.1..., transformers => base>=2 && <4.9) rejecting: transformers-0.4.1.0, transformers-0.3.0.0, transformers-0.2.2.1 (conflict: base==4.10.0.0/installed-4.1..., transformers +/-applicativeinbase => base>=1.0 && <4.8) rejecting: transformers-0.2.1.0, transformers-0.2.0.0 (conflict: base==4.10.0.0/installed-4.1..., transformers +/-applicativeinbase => base>=1.0 && <4.3) rejecting: transformers-0.1.4.0, transformers-0.1.3.0, transformers-0.1.1.0, transformers-0.1.0.1, transformers-0.0.1.0, transformers-0.0.0.0, transformers-0.5.3.1, transformers-0.5.3.0, transformers-0.5.0.2 (conflict: profunctors => transformers>=0.2 && <0.5) rejecting: transformers-0.4.0.0 (conflict: base==4.10.0.0/installed-4.1..., transformers +/-applicativeinbase => base>=1.0 && <4.8) rejecting: transformers-0.2.2.0 (conflict: base==4.10.0.0/installed-4.1..., transformers +/-applicativeinbase => base>=1.0 && <4.6) rejecting: transformers-0.1.0.0 (conflict: profunctors => transformers>=0.2 && <0.5) After searching the rest of the dependency tree exhaustively, these were the goals I've had most trouble fulfilling: transformers, contravariant, base, thorn After: $ cabal install --dry-run --index-state=2018-01-04T21:05:55Z thorn Resolving dependencies... cabal: Could not resolve dependencies: [__0] trying: thorn-0.2 (user goal) [__1] next goal: contravariant (dependency of thorn) [__1] rejecting: contravariant-1.4, contravariant-1.3.3, contravariant-1.3.2, contravariant-1.3.1.1, contravariant-1.3.1, contravariant-1.3, contravariant-1.2.2.1, contravariant-1.2.2, contravariant-1.2.1, contravariant-1.2.0.1, contravariant-1.2, contravariant-1.1, contravariant-1.0 (conflict: thorn => contravariant<1) [__1] trying: contravariant-0.6.1.1 [__2] next goal: transformers (dependency of contravariant) [__2] rejecting: transformers-0.5.2.0/installed-0.5..., transformers-0.5.5.0, transformers-0.5.4.0, transformers-0.5.2.0, transformers-0.5.1.0, transformers-0.5.0.1, transformers-0.5.0.0 (conflict: contravariant => transformers>=0.2 && <0.5) [__2] trying: transformers-0.4.3.0 [__3] next goal: base (dependency of thorn) [__3] rejecting: base-4.10.0.0/installed-4.1... (conflict: transformers => base>=2 && <4.9) [__3] rejecting: base-4.10.1.0, base-4.10.0.0, base-4.9.1.0, base-4.9.0.0, base-4.8.2.0, base-4.8.1.0, base-4.8.0.0, base-4.7.0.2, base-4.7.0.1, base-4.7.0.0, base-4.6.0.1, base-4.6.0.0, base-4.5.1.0, base-4.5.0.0, base-4.4.1.0, base-4.4.0.0, base-4.3.1.0, base-4.3.0.0, base-4.2.0.2, base-4.2.0.1, base-4.2.0.0, base-4.1.0.0, base-4.0.0.0, base-3.0.3.2, base-3.0.3.1 (constraint from non-upgradeable package requires installed instance) After searching the rest of the dependency tree exhaustively, these were the goals I've had most trouble fulfilling: transformers, contravariant, base, thorn Differences: - The new summary has level numbers, like the full log. - The conflicts are different. The old log mentions thorn, base, profunctors, and transformers, and the new log mentions the four packages from the conflict set: thorn, contravariant, transformers, and base. - The new log starts with the solver choosing a user goal, thorn. The solver continues to display the conflicts at the first backjump when it reaches the backjump limit, i.e, it shows profunctors instead of contravariant: Before: $ cabal install --dry-run --index-state=2018-01-04T21:05:55Z thorn --max-backjumps=10 Resolving dependencies... cabal: Could not resolve dependencies: trying: base-4.10.0.0/installed-4.1... (dependency of thorn) next goal: profunctors (dependency of thorn) rejecting: profunctors-5.2.1, profunctors-5.2, profunctors-5.1.2, profunctors-5.1.1, profunctors-5.1, profunctors-5.0.1, profunctors-5.0.0.1, profunctors-5 (conflict: thorn => profunctors<5) trying: profunctors-4.4.1 next goal: transformers (dependency of profunctors) rejecting: transformers-0.5.2.0/installed-0.5..., transformers-0.5.5.0, transformers-0.5.4.0, transformers-0.5.2.0, transformers-0.5.1.0, transformers-0.5.0.1, transformers-0.5.0.0 (conflict: profunctors => transformers>=0.2 && <0.5) rejecting: transformers-0.4.3.0, transformers-0.4.2.0 (conflict: base==4.10.0.0/installed-4.1..., transformers => base>=2 && <4.9) rejecting: transformers-0.4.1.0, transformers-0.3.0.0, transformers-0.2.2.1 (conflict: base==4.10.0.0/installed-4.1..., transformers +/-applicativeinbase => base>=1.0 && <4.8) rejecting: transformers-0.2.1.0, transformers-0.2.0.0 (conflict: base==4.10.0.0/installed-4.1..., transformers +/-applicativeinbase => base>=1.0 && <4.3) rejecting: transformers-0.1.4.0, transformers-0.1.3.0, transformers-0.1.1.0, transformers-0.1.0.1, transformers-0.0.1.0, transformers-0.0.0.0, transformers-0.5.3.1, transformers-0.5.3.0, transformers-0.5.0.2 (conflict: profunctors => transformers>=0.2 && <0.5) rejecting: transformers-0.4.0.0 (conflict: base==4.10.0.0/installed-4.1..., transformers +/-applicativeinbase => base>=1.0 && <4.8) rejecting: transformers-0.2.2.0 (conflict: base==4.10.0.0/installed-4.1..., transformers +/-applicativeinbase => base>=1.0 && <4.6) rejecting: transformers-0.1.0.0 (conflict: profunctors => transformers>=0.2 && <0.5) Backjump limit reached (currently 10, change with --max-backjumps or try to run with --reorder-goals). After: $ cabal install --dry-run --index-state=2018-01-04T21:05:55Z thorn --max-backjumps=10 Resolving dependencies... cabal: Could not resolve dependencies: [__0] trying: thorn-0.2 (user goal) [__1] next goal: profunctors (dependency of thorn) [__1] rejecting: profunctors-5.2.1, profunctors-5.2, profunctors-5.1.2, profunctors-5.1.1, profunctors-5.1, profunctors-5.0.1, profunctors-5.0.0.1, profunctors-5 (conflict: thorn => profunctors<5) [__1] trying: profunctors-4.4.1 [__2] next goal: transformers (dependency of profunctors) [__2] rejecting: transformers-0.5.2.0/installed-0.5..., transformers-0.5.5.0, transformers-0.5.4.0, transformers-0.5.2.0, transformers-0.5.1.0, transformers-0.5.0.1, transformers-0.5.0.0 (conflict: profunctors => transformers>=0.2 && <0.5) [__2] trying: transformers-0.4.3.0 [__3] next goal: base (dependency of thorn) [__3] rejecting: base-4.10.0.0/installed-4.1... (conflict: transformers => base>=2 && <4.9) [__3] rejecting: base-4.10.1.0, base-4.10.0.0, base-4.9.1.0, base-4.9.0.0, base-4.8.2.0, base-4.8.1.0, base-4.8.0.0, base-4.7.0.2, base-4.7.0.1, base-4.7.0.0, base-4.6.0.1, base-4.6.0.0, base-4.5.1.0, base-4.5.0.0, base-4.4.1.0, base-4.4.0.0, base-4.3.1.0, base-4.3.0.0, base-4.2.0.2, base-4.2.0.1, base-4.2.0.0, base-4.1.0.0, base-4.0.0.0, base-3.0.3.2, base-3.0.3.1 (constraint from non-upgradeable package requires installed instance) Backjump limit reached (currently 10, change with --max-backjumps or try to run with --reorder-goals). One downside of this change is that the solver may reach the backjump limit when generating the summarized log, if the backjump limit is very low: $ cabal install --dry-run --index-state=2018-01-04T21:05:55Z thorn --max-backjumps=1 Resolving dependencies... cabal: Backjump limit reached (currently 1, change with --max-backjumps or try to run with --reorder-goals). Failed to generate a summarized dependency solver log due to low backjump limit. Another downside is the performance impact of rerunning the solver. It looks like there isn't a big change in run time when the solver finds a solution or fails after an exhaustive search. However, rerunning the solver to the first backjump after it reaches the backjump limit can take a significant amount of time. The worst case I could find was acme-everything with GHC 7.10.3, where that step took 13 seconds. The difference was normally small, though. I ran hackage-benchmark on packages from Hackage to try to find packages where the run time changed by more than a few percent. I stopped it after all packages starting with "b" (That includes all uppercase packages). compiler: GHC 8.2.1 index state: 2018-01-04T21:05:55Z parameters: --min-run-time-percentage-difference-to-rerun=1 --pvalue=0.01 --trials=20 --print-skipped-packages Out of 2219 packages, 1064 were skipped because the run times in the first trial were within 1%, 1065 differed by more than 1% in the first trial but did not show a significant difference in run time in 20 trials, and 90 did show a significant difference in run time. Here are the counts of packages for different ranges of speedup, for those 90 packages: speedup (master avg. run time / branch avg. run time) package count [0.93, 0.94) 1 [0.94, 0.95) 0 [0.95, 0.96) 0 [0.96, 0.97) 1 [0.97, 0.98) 7 [0.98, 0.99) 29 [0.99, 1.00) 47 [1.00, 1.01) 3 [1.01, 1.02) 2 The package with the biggest percentage change was bittorrent, which ran for 3.85 seconds on master and 4.12 seconds on this branch. It reached the backjump limit. --- .../Distribution/Client/Dependency.hs | 4 +- cabal-install/Distribution/Solver/Modular.hs | 125 ++++++++++++++++-- .../Distribution/Solver/Modular/Log.hs | 53 +++----- .../Distribution/Solver/Modular/Preference.hs | 12 ++ .../Distribution/Solver/Modular/Solver.hs | 51 ++++--- .../Distribution/Solver/Modular/Tree.hs | 1 + .../Solver/Modular/WeightedPSQ.hs | 11 ++ .../Distribution/Solver/Modular/Solver.hs | 16 +-- .../Includes2/setup-external.cabal.out | 8 +- .../use-local-version-of-package.out | 6 +- .../use-local-package-as-setup-dep.out | 10 +- .../ConfigureComponent/Exe/setup.cabal.out | 6 +- .../BuildDependsBad/setup.cabal.out | 4 +- 13 files changed, 213 insertions(+), 94 deletions(-) diff --git a/cabal-install/Distribution/Client/Dependency.hs b/cabal-install/Distribution/Client/Dependency.hs index 8506849a446..e3eff395779 100644 --- a/cabal-install/Distribution/Client/Dependency.hs +++ b/cabal-install/Distribution/Client/Dependency.hs @@ -64,7 +64,7 @@ module Distribution.Client.Dependency ( ) where import Distribution.Solver.Modular - ( modularResolver, SolverConfig(..) ) + ( modularResolver, SolverConfig(..), PruneAfterFirstSuccess(..) ) import Distribution.Simple.PackageIndex (InstalledPackageIndex) import qualified Distribution.Simple.PackageIndex as InstalledPackageIndex import Distribution.Client.SolverInstallPlan (SolverInstallPlan) @@ -719,7 +719,7 @@ resolveDependencies platform comp pkgConfigDB solver params = $ runSolver solver (SolverConfig reordGoals cntConflicts indGoals noReinstalls shadowing strFlags allowBootLibs maxBkjumps enableBj - solveExes order verbosity) + solveExes order verbosity (PruneAfterFirstSuccess False)) platform comp installedPkgIndex sourcePkgIndex pkgConfigDB preferences constraints targets where diff --git a/cabal-install/Distribution/Solver/Modular.hs b/cabal-install/Distribution/Solver/Modular.hs index 21de9cb57dc..b7e2d98430a 100644 --- a/cabal-install/Distribution/Solver/Modular.hs +++ b/cabal-install/Distribution/Solver/Modular.hs @@ -1,5 +1,5 @@ module Distribution.Solver.Modular - ( modularResolver, SolverConfig(..)) where + ( modularResolver, SolverConfig(..), PruneAfterFirstSuccess(..)) where -- Here, we try to map between the external cabal-install solver -- interface and the internal interface that the solver actually @@ -9,25 +9,39 @@ module Distribution.Solver.Modular -- and finally, we have to convert back the resulting install -- plan. -import Data.Map as M - ( fromListWith ) +import Data.Map (Map) +import qualified Data.Map as M +import Data.Set (Set) +import Data.Ord import Distribution.Compat.Graph ( IsNode(..) ) +import Distribution.Compiler + ( CompilerInfo ) import Distribution.Solver.Modular.Assignment - ( toCPs ) + ( Assignment, toCPs ) import Distribution.Solver.Modular.ConfiguredConversion ( convCP ) +import qualified Distribution.Solver.Modular.ConflictSet as CS +import Distribution.Solver.Modular.Dependency +import Distribution.Solver.Modular.Flag +import Distribution.Solver.Modular.Index import Distribution.Solver.Modular.IndexConversion ( convPIs ) import Distribution.Solver.Modular.Log - ( logToProgress ) + ( SolverFailure(..), logToProgress ) import Distribution.Solver.Modular.Package ( PN ) import Distribution.Solver.Modular.Solver - ( SolverConfig(..), solve ) + ( SolverConfig(..), PruneAfterFirstSuccess(..), solve ) +import Distribution.Solver.Types.DependencyResolver import Distribution.Solver.Types.LabeledPackageConstraint import Distribution.Solver.Types.PackageConstraint -import Distribution.Solver.Types.DependencyResolver +import Distribution.Solver.Types.PackagePath +import Distribution.Solver.Types.PackagePreferences +import Distribution.Solver.Types.PkgConfigDb + ( PkgConfigDb ) +import Distribution.Solver.Types.Progress +import Distribution.Solver.Types.Variable import Distribution.System ( Platform(..) ) import Distribution.Simple.Utils @@ -38,9 +52,8 @@ import Distribution.Simple.Utils -- solver. Performs the necessary translations before and after. modularResolver :: SolverConfig -> DependencyResolver loc modularResolver sc (Platform arch os) cinfo iidx sidx pkgConfigDB pprefs pcs pns = - fmap (uncurry postprocess) $ -- convert install plan - logToProgress (solverVerbosity sc) (maxBackjumps sc) $ -- convert log format into progress format - solve sc cinfo idx pkgConfigDB pprefs gcs pns + fmap (uncurry postprocess) $ -- convert install plan + solve' sc cinfo idx pkgConfigDB pprefs gcs pns where -- Indices have to be converted into solver-specific uniform index. idx = convPIs os arch cinfo (shadowPkgs sc) (strongFlags sc) (solveExecutables sc) iidx sidx @@ -58,3 +71,95 @@ modularResolver sc (Platform arch os) cinfo iidx sidx pkgConfigDB pprefs pcs pns -- Helper function to extract the PN from a constraint. pcName :: PackageConstraint -> PN pcName (PackageConstraint scope _) = scopeToPackageName scope + +-- | Run 'D.S.Modular.Solver.solve' and then produce a summarized log to display +-- in the error case. +-- +-- When there is no solution, we produce the error message by rerunning the +-- solver but making it prefer the goals from the final conflict set from the +-- first run. We also set the backjump limit to 0, so that the log stops at the +-- first backjump and is relatively short. Preferring goals from the final +-- conflict set increases the probability that the log to the first backjump +-- contains package, flag, and stanza choices that are relevant to the final +-- failure. The solver shouldn't need to choose any packages that aren't in the +-- final conflict set. (For every variable in the final conflict set, the final +-- conflict set should also contain the variable that introduced that variable. +-- The solver can then follow that chain of variables in reverse order from the +-- user target to the conflict.) However, it is possible that the conflict set +-- contains unnecessary variables. +-- +-- Producing an error message when the solver reaches the backjump limit is more +-- complicated. There is no final conflict set, so we create one for the minimal +-- subtree containing the path that the solver took to the first backjump. This +-- conflict set helps explain why the solver reached the backjump limit, because +-- the first backjump contributes to reaching the backjump limit. Additionally, +-- the solver is much more likely to be able to finish traversing this subtree +-- before the backjump limit, since its size is linear (not exponential) in the +-- number of goal choices. We create it by pruning all children after the first +-- successful child under each node in the original tree, so that there is at +-- most one valid choice at each level. Then we use the final conflict set from +-- that run to generate an error message, as in the case where the solver found +-- that there was no solution. +-- +-- Using the full log from a rerun of the solver ensures that the log is +-- complete, i.e., it shows the whole chain of dependencies from the user +-- targets to the conflicting packages. +solve' :: SolverConfig + -> CompilerInfo + -> Index + -> PkgConfigDb + -> (PN -> PackagePreferences) + -> Map PN [LabeledPackageConstraint] + -> Set PN + -> Progress String String (Assignment, RevDepMap) +solve' sc cinfo idx pkgConfigDB pprefs gcs pns = + foldProgress Step createErrorMsg Done (runSolver sc) + where + runSolver :: SolverConfig + -> Progress String SolverFailure (Assignment, RevDepMap) + runSolver sc' = + logToProgress (solverVerbosity sc') (maxBackjumps sc') $ -- convert log format into progress format + solve sc' cinfo idx pkgConfigDB pprefs gcs pns + + createErrorMsg :: SolverFailure + -> Progress String String (Assignment, RevDepMap) + createErrorMsg (NoSolution cs msg) = + Fail $ rerunSolverForErrorMsg cs msg + createErrorMsg (BackjumpLimitReached msg) = + Step ("Backjump limit reached. Rerunning dependency solver to generate " + ++ "a final conflict set for the search tree containing the " + ++ "first backjump.") $ + foldProgress Step f Done $ + runSolver sc { pruneAfterFirstSuccess = PruneAfterFirstSuccess True } + where + f :: SolverFailure -> Progress String String (Assignment, RevDepMap) + f (NoSolution cs _) = Fail $ rerunSolverForErrorMsg cs msg + f (BackjumpLimitReached _) = + -- This case is possible when the number of goals involved in + -- conflicts is greater than the backjump limit. + Fail $ msg ++ "Failed to generate a summarized dependency solver " + ++ "log due to low backjump limit." + + rerunSolverForErrorMsg :: ConflictSet -> String -> String + rerunSolverForErrorMsg cs finalMsg = + let sc' = sc { + goalOrder = Just (preferGoalsFromConflictSet cs) + , maxBackjumps = Just 0 + } + in unlines ("Could not resolve dependencies:" : messages (runSolver sc')) + ++ finalMsg + + messages :: Progress step fail done -> [step] + messages = foldProgress (:) (const []) (const []) + +-- | Goal ordering that chooses goals contained in the conflict set before +-- other goals. +preferGoalsFromConflictSet :: ConflictSet + -> Variable QPN -> Variable QPN -> Ordering +preferGoalsFromConflictSet cs = + comparing $ \v -> not $ CS.member (toVar v) cs + where + toVar :: Variable QPN -> Var QPN + toVar (PackageVar qpn) = P qpn + toVar (FlagVar qpn fn) = F (FN qpn fn) + toVar (StanzaVar qpn sn) = S (SN qpn sn) diff --git a/cabal-install/Distribution/Solver/Modular/Log.hs b/cabal-install/Distribution/Solver/Modular/Log.hs index f80352e7406..50d554e076d 100644 --- a/cabal-install/Distribution/Solver/Modular/Log.hs +++ b/cabal-install/Distribution/Solver/Modular/Log.hs @@ -1,13 +1,12 @@ module Distribution.Solver.Modular.Log ( Log , logToProgress + , SolverFailure(..) ) where import Prelude () import Distribution.Solver.Compat.Prelude -import Data.List as L - import Distribution.Solver.Types.Progress import Distribution.Solver.Modular.Dependency @@ -23,20 +22,22 @@ import Distribution.Verbosity -- Parameterized over the type of actual messages and the final result. type Log m a = Progress m (ConflictSet, ConflictMap) a -messages :: Progress step fail done -> [step] -messages = foldProgress (:) (const []) (const []) +data Exhaustiveness = Exhaustive | BackjumpLimit -data Exhaustiveness = Exhaustive | BackjumpLimitReached +-- | Information about a dependency solver failure. It includes an error message +-- and a final conflict set, if available. +data SolverFailure = + NoSolution ConflictSet String + | BackjumpLimitReached String -- | Postprocesses a log file. Takes as an argument a limit on allowed backjumps. -- If the limit is 'Nothing', then infinitely many backjumps are allowed. If the -- limit is 'Just 0', backtracking is completely disabled. -logToProgress :: Verbosity -> Maybe Int -> Log Message a -> Progress String String a +logToProgress :: Verbosity -> Maybe Int -> Log Message a -> Progress String SolverFailure a logToProgress verbosity mbj l = - let es = proc (Just 0) l -- catch first error (always) - ms = proc mbj l - in go es es -- trace for first error - (showMessages (const True) True ms) -- run with backjump limit applied + let ms = proc mbj l + mapFailure f = foldProgress Step (Fail . f) Done + in mapFailure finalError (showMessages (const True) True ms) -- run with backjump limit applied where -- Proc takes the allowed number of backjumps and a 'Progress' and explores the -- messages until the maximum number of backjumps has been reached. It filters out @@ -48,30 +49,15 @@ logToProgress verbosity mbj l = proc _ (Fail (cs, cm)) = Fail (Exhaustive, cs, cm) proc mbj' (Step x@(Failure cs Backjump) xs@(Step Leave (Step (Failure cs' Backjump) _))) | cs == cs' = Step x (proc mbj' xs) -- repeated backjumps count as one - proc (Just 0) (Step (Failure cs Backjump) _) = Fail (BackjumpLimitReached, cs, mempty) -- No final conflict map available + proc (Just 0) (Step (Failure cs Backjump) _) = Fail (BackjumpLimit, cs, mempty) -- No final conflict map available proc (Just n) (Step x@(Failure _ Backjump) xs) = Step x (proc (Just (n - 1)) xs) proc mbj' (Step x xs) = Step x (proc mbj' xs) - -- The first two arguments are both supposed to be the log up to the first error. - -- That's the error that will always be printed in case we do not find a solution. - -- We pass this log twice, because we evaluate it in parallel with the full log, - -- but we also want to retain the reference to its beginning for when we print it. - -- This trick prevents a space leak! - -- - -- The third argument is the full log, ending with either the solution or the - -- exhaustiveness and final conflict set. - go :: Progress Message (Exhaustiveness, ConflictSet, ConflictMap) b - -> Progress Message (Exhaustiveness, ConflictSet, ConflictMap) b - -> Progress String (Exhaustiveness, ConflictSet, ConflictMap) b - -> Progress String String b - go ms (Step _ ns) (Step x xs) = Step x (go ms ns xs) - go ms r (Step x xs) = Step x (go ms r xs) - go ms (Step _ ns) r = go ms ns r - go ms (Fail (_, cs', _)) (Fail (exh, cs, cm)) = Fail $ - "Could not resolve dependencies:\n" ++ - unlines (messages $ showMessages (L.foldr (\ v _ -> v `CS.member` cs') True) False ms) ++ + finalError :: (Exhaustiveness, ConflictSet, ConflictMap) -> SolverFailure + finalError (exh, cs, cm) = case exh of Exhaustive -> + NoSolution cs $ "After searching the rest of the dependency tree exhaustively, " ++ "these were the goals I've had most trouble fulfilling: " ++ showCS cm cs @@ -79,14 +65,9 @@ logToProgress verbosity mbj l = showCS = if verbosity > normal then CS.showCSWithFrequency else CS.showCSSortedByFrequency - BackjumpLimitReached -> + BackjumpLimit -> + BackjumpLimitReached $ "Backjump limit reached (" ++ currlimit mbj ++ "change with --max-backjumps or try to run with --reorder-goals).\n" where currlimit (Just n) = "currently " ++ show n ++ ", " currlimit Nothing = "" - go _ _ (Done s) = Done s - go _ (Done _) (Fail _) = Fail $ - -- Should not happen: Second argument is the log up to first error, - -- third one is the entire log. Therefore it should never happen that - -- the second log finishes with 'Done' and the third log with 'Fail'. - "Could not resolve dependencies; something strange happened." diff --git a/cabal-install/Distribution/Solver/Modular/Preference.hs b/cabal-install/Distribution/Solver/Modular/Preference.hs index 74816f51f09..e7665ba528a 100644 --- a/cabal-install/Distribution/Solver/Modular/Preference.hs +++ b/cabal-install/Distribution/Solver/Modular/Preference.hs @@ -14,6 +14,7 @@ module Distribution.Solver.Modular.Preference , preferReallyEasyGoalChoices , requireInstalled , sortGoals + , pruneAfterFirstSuccess ) where import Prelude () @@ -351,6 +352,17 @@ sortGoals variableOrder = trav go varToVariable (F (FN qpn fn)) = FlagVar qpn fn varToVariable (S (SN qpn stanza)) = StanzaVar qpn stanza +-- | Reduce the branching degree of the search tree by removing all choices +-- after the first successful choice at each level. The returned tree is the +-- minimal subtree containing the path to the first backjump. +pruneAfterFirstSuccess :: Tree d c -> Tree d c +pruneAfterFirstSuccess = trav go + where + go (PChoiceF qpn rdm gr ts) = PChoiceF qpn rdm gr (W.takeUntil active ts) + go (FChoiceF qfn rdm gr w m d ts) = FChoiceF qfn rdm gr w m d (W.takeUntil active ts) + go (SChoiceF qsn rdm gr w ts) = SChoiceF qsn rdm gr w (W.takeUntil active ts) + go x = x + -- | Always choose the first goal in the list next, abandoning all -- other choices. -- diff --git a/cabal-install/Distribution/Solver/Modular/Solver.hs b/cabal-install/Distribution/Solver/Modular/Solver.hs index acfe4def487..b2b717ca4f6 100644 --- a/cabal-install/Distribution/Solver/Modular/Solver.hs +++ b/cabal-install/Distribution/Solver/Modular/Solver.hs @@ -6,6 +6,7 @@ module Distribution.Solver.Modular.Solver ( SolverConfig(..) , solve + , PruneAfterFirstSuccess(..) ) where import Data.Map as M @@ -53,20 +54,25 @@ import Debug.Trace.Tree.Assoc (Assoc(..)) -- | Various options for the modular solver. data SolverConfig = SolverConfig { - reorderGoals :: ReorderGoals, - countConflicts :: CountConflicts, - independentGoals :: IndependentGoals, - avoidReinstalls :: AvoidReinstalls, - shadowPkgs :: ShadowPkgs, - strongFlags :: StrongFlags, - allowBootLibInstalls :: AllowBootLibInstalls, - maxBackjumps :: Maybe Int, - enableBackjumping :: EnableBackjumping, - solveExecutables :: SolveExecutables, - goalOrder :: Maybe (Variable QPN -> Variable QPN -> Ordering), - solverVerbosity :: Verbosity + reorderGoals :: ReorderGoals, + countConflicts :: CountConflicts, + independentGoals :: IndependentGoals, + avoidReinstalls :: AvoidReinstalls, + shadowPkgs :: ShadowPkgs, + strongFlags :: StrongFlags, + allowBootLibInstalls :: AllowBootLibInstalls, + maxBackjumps :: Maybe Int, + enableBackjumping :: EnableBackjumping, + solveExecutables :: SolveExecutables, + goalOrder :: Maybe (Variable QPN -> Variable QPN -> Ordering), + solverVerbosity :: Verbosity, + pruneAfterFirstSuccess :: PruneAfterFirstSuccess } +-- | Whether to remove all choices after the first successful choice at each +-- level in the search tree. +newtype PruneAfterFirstSuccess = PruneAfterFirstSuccess Bool + -- | Run all solver phases. -- -- In principle, we have a valid tree after 'validationPhase', which @@ -97,15 +103,18 @@ solve sc cinfo idx pkgConfigDB userPrefs userConstraints userGoals = detectCycles = traceTree "cycles.json" id . detectCyclesPhase heuristicsPhase = let heuristicsTree = traceTree "heuristics.json" id - in case goalOrder sc of - Nothing -> goalChoiceHeuristics . - heuristicsTree . - P.deferSetupChoices . - P.deferWeakFlagChoices . - P.preferBaseGoalChoice - Just order -> P.firstGoal . - heuristicsTree . - P.sortGoals order + sortGoals = case goalOrder sc of + Nothing -> goalChoiceHeuristics . + heuristicsTree . + P.deferSetupChoices . + P.deferWeakFlagChoices . + P.preferBaseGoalChoice + Just order -> P.firstGoal . + heuristicsTree . + P.sortGoals order + PruneAfterFirstSuccess prune = pruneAfterFirstSuccess sc + in sortGoals . + (if prune then P.pruneAfterFirstSuccess else id) preferencesPhase = P.preferLinked . P.preferPackagePreferences userPrefs validationPhase = traceTree "validated.json" id . diff --git a/cabal-install/Distribution/Solver/Modular/Tree.hs b/cabal-install/Distribution/Solver/Modular/Tree.hs index 91c2f9becc4..a7ee060e043 100644 --- a/cabal-install/Distribution/Solver/Modular/Tree.hs +++ b/cabal-install/Distribution/Solver/Modular/Tree.hs @@ -13,6 +13,7 @@ module Distribution.Solver.Modular.Tree , para , trav , zeroOrOneChoices + , active ) where import Control.Monad hiding (mapM, sequence) diff --git a/cabal-install/Distribution/Solver/Modular/WeightedPSQ.hs b/cabal-install/Distribution/Solver/Modular/WeightedPSQ.hs index 91c37a4e6ae..21216358ac5 100644 --- a/cabal-install/Distribution/Solver/Modular/WeightedPSQ.hs +++ b/cabal-install/Distribution/Solver/Modular/WeightedPSQ.hs @@ -1,4 +1,5 @@ {-# LANGUAGE DeriveFunctor, DeriveFoldable, DeriveTraversable #-} +{-# LANGUAGE ScopedTypeVariables #-} module Distribution.Solver.Modular.WeightedPSQ ( WeightedPSQ , fromList @@ -11,6 +12,7 @@ module Distribution.Solver.Modular.WeightedPSQ ( , mapWithKey , mapWeightsWithKey , union + , takeUntil ) where import qualified Data.Foldable as F @@ -76,6 +78,15 @@ mapWithKey f (WeightedPSQ xs) = WeightedPSQ $ union :: Ord w => WeightedPSQ w k v -> WeightedPSQ w k v -> WeightedPSQ w k v union (WeightedPSQ xs) (WeightedPSQ ys) = fromList (xs ++ ys) +-- | /O(N)/. Return the prefix of values ending with the first element that +-- satisfies p, or all elements if none satisfy p. +takeUntil :: forall w k v. (v -> Bool) -> WeightedPSQ w k v -> WeightedPSQ w k v +takeUntil p (WeightedPSQ xs) = WeightedPSQ (go xs) + where + go :: [(w, k, v)] -> [(w, k, v)] + go [] = [] + go (y : ys) = y : if p (triple_3 y) then [] else go ys + triple_1 :: (x, y, z) -> x triple_1 (x, _, _) = x diff --git a/cabal-install/tests/UnitTests/Distribution/Solver/Modular/Solver.hs b/cabal-install/tests/UnitTests/Distribution/Solver/Modular/Solver.hs index 23e0b11f455..9907a61683e 100644 --- a/cabal-install/tests/UnitTests/Distribution/Solver/Modular/Solver.hs +++ b/cabal-install/tests/UnitTests/Distribution/Solver/Modular/Solver.hs @@ -291,9 +291,9 @@ tests = [ solverFailure $ isInfixOf $ -- The solver reports the version conflict when a version conflict -- and an executable conflict apply to the same package version. - "rejecting: H:bt-pkg:exe.bt-pkg-4.0.0 (conflict: H => H:bt-pkg:exe.bt-pkg (exe exe1)==3.0.0)\n" - ++ "rejecting: H:bt-pkg:exe.bt-pkg-3.0.0 (does not contain executable exe1, which is required by H)\n" - ++ "rejecting: H:bt-pkg:exe.bt-pkg-2.0.0, H:bt-pkg:exe.bt-pkg-1.0.0 (conflict: H => H:bt-pkg:exe.bt-pkg (exe exe1)==3.0.0)" + "[__1] rejecting: H:bt-pkg:exe.bt-pkg-4.0.0 (conflict: H => H:bt-pkg:exe.bt-pkg (exe exe1)==3.0.0)\n" + ++ "[__1] rejecting: H:bt-pkg:exe.bt-pkg-3.0.0 (does not contain executable exe1, which is required by H)\n" + ++ "[__1] rejecting: H:bt-pkg:exe.bt-pkg-2.0.0, H:bt-pkg:exe.bt-pkg-1.0.0 (conflict: H => H:bt-pkg:exe.bt-pkg (exe exe1)==3.0.0)" , runTest $ chooseExeAfterBuildToolsPackage True "choose exe after choosing its package - success" @@ -923,9 +923,9 @@ db18 = [ commonDependencyLogMessage :: String -> SolverTest commonDependencyLogMessage name = mkTest db name ["A"] $ solverFailure $ isInfixOf $ - "trying: A-1.0.0 (user goal)\n" - ++ "next goal: B (dependency of A +/-flagA)\n" - ++ "rejecting: B-2.0.0 (conflict: A +/-flagA => B==1.0.0 || ==3.0.0)" + "[__0] trying: A-1.0.0 (user goal)\n" + ++ "[__1] next goal: B (dependency of A +/-flagA)\n" + ++ "[__1] rejecting: B-2.0.0 (conflict: A +/-flagA => B==1.0.0 || ==3.0.0)" where db :: ExampleDb db = [ @@ -1434,8 +1434,8 @@ chooseExeAfterBuildToolsPackage shouldSucceed name = requireConsistentBuildToolVersions :: String -> SolverTest requireConsistentBuildToolVersions name = mkTest db name ["A"] $ solverFailure $ isInfixOf $ - "rejecting: A:B:exe.B-2.0.0 (conflict: A => A:B:exe.B (exe exe1)==1.0.0)\n" - ++ "rejecting: A:B:exe.B-1.0.0 (conflict: A => A:B:exe.B (exe exe2)==2.0.0)" + "[__1] rejecting: A:B:exe.B-2.0.0 (conflict: A => A:B:exe.B (exe exe1)==1.0.0)\n" + ++ "[__1] rejecting: A:B:exe.B-1.0.0 (conflict: A => A:B:exe.B (exe exe2)==2.0.0)" where db :: ExampleDb db = [ diff --git a/cabal-testsuite/PackageTests/Backpack/Includes2/setup-external.cabal.out b/cabal-testsuite/PackageTests/Backpack/Includes2/setup-external.cabal.out index 38102e2778f..e91346a7b78 100644 --- a/cabal-testsuite/PackageTests/Backpack/Includes2/setup-external.cabal.out +++ b/cabal-testsuite/PackageTests/Backpack/Includes2/setup-external.cabal.out @@ -99,10 +99,10 @@ Registering library for src-0.1.0.0.. Resolving dependencies... Warning: solver failed to find a solution: Could not resolve dependencies: -trying: exe-0.1.0.0 (user goal) -next goal: src (dependency of exe) -rejecting: src-/installed-... (conflict: src => mylib==0.1.0.0/installed-0.1..., src => mylib==0.1.0.0/installed-0.1...) -fail (backjumping, conflict set: exe, src) +[__0] trying: exe-0.1.0.0 (user goal) +[__1] next goal: src (dependency of exe) +[__1] rejecting: src-/installed-... (conflict: src => mylib==0.1.0.0/installed-0.1..., src => mylib==0.1.0.0/installed-0.1...) +[__1] fail (backjumping, conflict set: exe, src) After searching the rest of the dependency tree exhaustively, these were the goals I've had most trouble fulfilling: exe (2), src (2) Trying configure anyway. Configuring exe-0.1.0.0... diff --git a/cabal-testsuite/PackageTests/BuildTargets/UseLocalPackage/use-local-version-of-package.out b/cabal-testsuite/PackageTests/BuildTargets/UseLocalPackage/use-local-version-of-package.out index 3515de864da..f975325ec42 100644 --- a/cabal-testsuite/PackageTests/BuildTargets/UseLocalPackage/use-local-version-of-package.out +++ b/cabal-testsuite/PackageTests/BuildTargets/UseLocalPackage/use-local-version-of-package.out @@ -13,7 +13,7 @@ local pkg-1.0 # cabal new-build Resolving dependencies... cabal: Could not resolve dependencies: -next goal: pkg (user goal) -rejecting: pkg-2.0 (constraint from user target requires ==1.0) -rejecting: pkg-1.0 (constraint from command line flag requires ==2.0) +[__0] next goal: pkg (user goal) +[__0] rejecting: pkg-2.0 (constraint from user target requires ==1.0) +[__0] rejecting: pkg-1.0 (constraint from command line flag requires ==2.0) After searching the rest of the dependency tree exhaustively, these were the goals I've had most trouble fulfilling: pkg (3) diff --git a/cabal-testsuite/PackageTests/BuildTargets/UseLocalPackageForSetup/use-local-package-as-setup-dep.out b/cabal-testsuite/PackageTests/BuildTargets/UseLocalPackageForSetup/use-local-package-as-setup-dep.out index c9c06ce0dbf..db1c3e94a60 100644 --- a/cabal-testsuite/PackageTests/BuildTargets/UseLocalPackageForSetup/use-local-package-as-setup-dep.out +++ b/cabal-testsuite/PackageTests/BuildTargets/UseLocalPackageForSetup/use-local-package-as-setup-dep.out @@ -3,11 +3,11 @@ Downloading the latest package list from test-local-repo # cabal new-build Resolving dependencies... cabal: Could not resolve dependencies: -trying: pkg-1.0 (user goal) -next goal: setup-dep (user goal) -rejecting: setup-dep-2.0 (conflict: pkg => setup-dep==1.*) -rejecting: setup-dep-1.0 (constraint from user target requires ==2.0) -fail (backjumping, conflict set: pkg, setup-dep) +[__0] trying: pkg-1.0 (user goal) +[__1] next goal: setup-dep (user goal) +[__1] rejecting: setup-dep-2.0 (conflict: pkg => setup-dep==1.*) +[__1] rejecting: setup-dep-1.0 (constraint from user target requires ==2.0) +[__1] fail (backjumping, conflict set: pkg, setup-dep) After searching the rest of the dependency tree exhaustively, these were the goals I've had most trouble fulfilling: setup-dep (3), pkg (2) # pkg my-exe Main.hs: setup-dep from repo diff --git a/cabal-testsuite/PackageTests/ConfigureComponent/Exe/setup.cabal.out b/cabal-testsuite/PackageTests/ConfigureComponent/Exe/setup.cabal.out index 9284a1ad2c5..90439a1fdc6 100644 --- a/cabal-testsuite/PackageTests/ConfigureComponent/Exe/setup.cabal.out +++ b/cabal-testsuite/PackageTests/ConfigureComponent/Exe/setup.cabal.out @@ -2,9 +2,9 @@ Resolving dependencies... Warning: solver failed to find a solution: Could not resolve dependencies: -trying: Exe-0.1.0.0 (user goal) -unknown package: totally-impossible-dependency-to-fill (dependency of Exe) -fail (backjumping, conflict set: Exe, totally-impossible-dependency-to-fill) +[__0] trying: Exe-0.1.0.0 (user goal) +[__1] unknown package: totally-impossible-dependency-to-fill (dependency of Exe) +[__1] fail (backjumping, conflict set: Exe, totally-impossible-dependency-to-fill) After searching the rest of the dependency tree exhaustively, these were the goals I've had most trouble fulfilling: Exe (2), totally-impossible-dependency-to-fill (1) Trying configure anyway. Configuring executable 'goodexe' for Exe-0.1.0.0.. diff --git a/cabal-testsuite/PackageTests/InternalVersions/BuildDependsBad/setup.cabal.out b/cabal-testsuite/PackageTests/InternalVersions/BuildDependsBad/setup.cabal.out index 32067ddeab2..815f6399b4a 100644 --- a/cabal-testsuite/PackageTests/InternalVersions/BuildDependsBad/setup.cabal.out +++ b/cabal-testsuite/PackageTests/InternalVersions/BuildDependsBad/setup.cabal.out @@ -2,8 +2,8 @@ Resolving dependencies... Warning: solver failed to find a solution: Could not resolve dependencies: -next goal: build-depends-bad-version (user goal) -rejecting: build-depends-bad-version-0.1.0.0 (conflict: build-depends-bad-version==0.1.0.0, build-depends-bad-version => build-depends-bad-version>=2) +[__0] next goal: build-depends-bad-version (user goal) +[__0] rejecting: build-depends-bad-version-0.1.0.0 (conflict: build-depends-bad-version==0.1.0.0, build-depends-bad-version => build-depends-bad-version>=2) After searching the rest of the dependency tree exhaustively, these were the goals I've had most trouble fulfilling: build-depends-bad-version (2) Trying configure anyway. Configuring build-depends-bad-version-0.1.0.0...