1.5.1 breaks Cabal test suites

[geekosaur]

We've been using tasty 1.5, but 1.5.1 was just released today and now our CI gets:

tests/UnitTests/Distribution/Solver/Modular/QuickCheck/Utils.hs:34:25: error: [GHC-83865]
Error:     • Couldn't match expected type: (Test.QuickCheck.Random.QCGen, Int)
                  with actual type: Maybe a1
    • In the pattern: Just override
      In the pattern: QuickCheckReplay (Just override)
      In a case alternative:
          QuickCheckReplay (Just override) -> return override
   |
34 |       QuickCheckReplay (Just override) -> return override
   |                         ^^^^^^^^^^^^^

tests/UnitTests/Distribution/Solver/Modular/QuickCheck/Utils.hs:35:24: error: [GHC-83865]
Error:     • Couldn't match expected type: (Test.QuickCheck.Random.QCGen, Int)
                  with actual type: Maybe a0
    • In the pattern: Nothing
      In the pattern: QuickCheckReplay Nothing
      In a case alternative:
          QuickCheckReplay Nothing -> getStdRandom random
   |
35 |       QuickCheckReplay Nothing -> getStdRandom random
   |                        ^^^^^^^

tests/UnitTests/Distribution/Solver/Modular/QuickCheck/Utils.hs:37:39: error: [GHC-83865]
Error:     • Couldn't match expected type: (Test.QuickCheck.Random.QCGen, Int)
                  with actual type: Maybe a1
    • In the first argument of ‘QuickCheckReplay’, namely
        ‘(Just replay)’
      In the first argument of ‘setOption’, namely
        ‘(QuickCheckReplay (Just replay))’
      In the first argument of ‘run’, namely
        ‘(setOption (QuickCheckReplay (Just replay)) options)’
    • Relevant bindings include
        replay :: a1
          (bound at tests/UnitTests/Distribution/Solver/Modular/QuickCheck/Utils.hs:33:5)
   |
37 |     run (setOption (QuickCheckReplay (Just replay)) options) test progress
   |                                       ^^^^^^^^^^^

While the errors involve QuickCheck, it hasn't changed recently, while the build log says Test.Tasty.Providers did change and forced the rebuild.

[geekosaur]

Confirmed: changing our tasty constraint from <1.6 to <1.5.1 our tests build. Shouldn't this kind of change not happen in minor releases?

[Bodigrim]

Isn't this caused by tasty-quickcheck-0.11, which is a major release?

[Bodigrim]

You can probably replace Nothing with QuickCheckReplayNone and Just with QuickCheckReplayLegacy. Or put tasty-quickcheck < 0.11 if time is short.

[Bodigrim]

tasty/quickcheck/Test/Tasty/QuickCheck.hs

Lines 189 to 195 in 298f69b

	(intSeed, replaySeed) <- case quickCheckReplay of
	QuickCheckReplayNone -> do
	intSeed <- getStdRandom (randomR (1,999999))
	return (intSeed, (mkQCGen intSeed, 0))
	QuickCheckReplayLegacy intSeed -> return (intSeed, (mkQCGen intSeed, 0))
	-- The intSeed is not used when the new form of replay seed is used.
	QuickCheckReplay replaySeed -> return (0, replaySeed)

...and then show (intSeed, replaySeed) instead of show replay.

---

It would be better to fix the root cause for Cabal's workaround by implementing #359 (comment). Should be pretty straightforward.

[geekosaur]

We're trying to get a release out, so I'm pushing the constraint. I just got a successful validate run.

[Bodigrim]

Cf. haskell/cabal#10136