F# 6 takes fewer tailcalls in debug code when explicit /tailcalls+ is specified

[dsyme]

This is an interesting problem I saw when investigating #12322

Context: Debug mode has "/tailcalls-" by default. However, users can turn on "/tailcalls+" explicitly for debug code, in which case we do expect tailcalls to be taken.

Now, for F# 6, fewer tailcalls are taken for Debug code "/tailcalls+". Consider the following continuation calling code:

    let g x cont = (x+2) |> cont

• In F# 5 with "/tailcalls+", the "|> cont" was a tailcall
• In F# 6 with "/tailcalls+", the "|> cont" is not a tailcall, I presume because of pipeline debugging on by default

Does this matter? I guess it does since Debug code now takes fewer tailcalls even when /tailcalls+ is specified.

Possible solutions

• We could disable pipeline debugging for pipelines in tailcall-position when "/tailcalls+" is specified, or
• We could completely disable pipeline debugging when "/tailcalls+" is specified.

The first is a little more work.

Symptom

Stack overflow in debug code when using F# 6 where it doesn't occur when using F# 5.

Workarounds

As workaround, you can enable optimizations in debug code which has "/tailcalls+" enabled. This suppresses pipeline debugging. However it makes debugging much worse.

Context

As context: I saw an instance of this in FSharp.Data when activating "/tailcalls+" in a design-time type provider - the overall projects compiled with F# 5 toolchain but didn't with F# 6 toolchain. However this only affected Debug code, including the Debug instance of the type provider. Turning optimizations on for the design-time type provider code would have disabled pipeline debugging and the tailcalls would ahve been recovered.

[dsyme]

Fixed