Regression in nuget >=2.7.1: Execution engine crash or tests hang

[blumu]

Upgrading FSharpLu.Tests from FsCheck 2.4.0 to >=2.7.1 causes some tests to fail or timeout.
After bisecting the history of nuget packages I could determine that the regression was introduced in FsCheck 2.7.1 (It does not repro on FsCheck <=2.7.0)

Repro

1 - Get the repo at https://github.com/microsoft/fsharplu
2 - Build and run the test from VS 2017. Observe that all tests pass
3 - Upgrade the FsCheck nuget to 2.8.0
4 - Rebuild and run the test.
5 - Observe that the test execution engine fails with the following error

[8/26/2017 12:49:29 PM Informational] ------ Run test started ------
[8/26/2017 12:50:46 PM Error] The active Test Run was aborted because the execution process exited unexpectedly. The test execution process crashed while running the tests. To investigate further, open file:///C:/Users/wiblum/AppData/Local/CrashDumps/vstest.executionengine.x86.exe.11708.dmp file in Visual Studio and choose "Debug in mixed mode".

6 - The behavior varies by version. For instance with version 2.7.1 the execution engine does not crash but 4 of the 19 unit tests hang (on 2.7.0 they all pass in a few seconds).

[kurtschelfthout]

Probably because of a change to recursive type generation in 2.7.1: 923f9d3.

Are you generating record and/or union types that are recursive? If so, can you give me one or more examples of such types for which it fails?

[blumu]

Yes, the following types are used in those tests, which include recursive records and union types:

type WithFields = SomeField of int * int
type SimpleDu = Foo | FooBar | Bar
type ComplexDu = ComplexDu of WithFields | SimpleDU | AString of string
type 'a RecursiveList = RecListLeaf of 'a | RecListCons of 'a RecursiveList
type OptionOfBase = int option
type OptionOfDu = SimpleDu option
type Color = Red | Blue
type Shape = Circle of int * int | Rectangle
type 'a Tree = Leaf of 'a | Node of 'a Tree * 'a Tree
type 'a Test = Case1 | Case2 of int | Case3 of int * string * 'a
type MapType = Map<string,Color>
type 'a NestedOptions = 'a option option option option

type 'a Wrapper = { WrappedField : 'a }
type NestedStructure = { subField : int }
type NestedOptionStructure = { field : NestedStructure option }

See source file at https://github.com/Microsoft/fsharplu/blob/master/FSharpLu.Tests/JsonTests.fs

[kurtschelfthout]

I think I have this fixed, re-opening because not released yet, but hopefully later today. Some housekeeping I want to take a look at first.

[kurtschelfthout]

Released in 2.9.2. Give it a go and re-open if not fixed. Thanks!

[blumu]

I confirm that 2.9.2 fixes it. Thanks @kurtschelfthout for the quick fix!