Shrinking takes an enormous amount of time

[dnadales]

When a counterexample is found, the shrinking process can take up to 10 minutes. This is not acceptable. We should try to tweak the generators so that the shrinking becomes more efficient. Alternatively, we might have to change our testing approach so that we use state-machine-based testing, instead of generating the whole abstract trace at once.

The first thing to try is to replace the use of frequency:

      Gen.frequency [ (5, return acc)
                    -- The probability of continue with the recursion depends
                    -- on the size parameter of the generator. Here the
                    -- constant factor is determined ad-hoc.
                    , (unSize d * 2, do
                          mStSig <- genSigSt @s env sti aSigGen
                          case mStSig of
                            Nothing ->
                              go d sti acc
                            Just (stNext, sig) ->
                              go d stNext ((stNext, sig): acc)
                      )
                    ]

Since this will generate a very large shrink three. An alternative suggested by @dcoutts is to generate an arbitrary number that determines the trace length, and then generate a trace of that size.

The other thing to try, is generating only valid signals, so that the chance of failure is smaller when composing large generators.

Finally, we might need to incorporate more knowledge of the operational rules into our generators.

[dnadales]

This will be solved by the work @ruhatch carried out, combining hedgehog state machine models with our STS framework.

[ruhatch]

Should be helped by PR #451

[ruhatch]

@dnadales Maybe we should reopen this as you're going to look into Trace shrinking further?

[dnadales]

I don't think so. What we will be doing is experimental. The issue was solved by using hedgehog state machine, so whatever proposal results from our experiments with an improved traces generator should take shrinking time into consideration. Does that make sense?