Cabal: Use -flink-rts when available

[bgamari]

Previously Cabal did quite some headstands to link against libHSrts.
Note only this is complex but it couples very tightly to GHC's
implementation. Thankfully, as of GHC 9.0 GHC provides a -flink-rts
flag for precisely this purpose. Use it when available.

Fixes #7763.

---

Please include the following checklist in your PR:

• Patches conform to the coding conventions.
• Any changes that could be relevant to users have been recorded in the changelog.
• The documentation has been updated, if necessary.

Please also shortly describe how you tested your change. Bonus points for added tests!

[bgamari]

This appears to fix the original ticket on Windows. However, my testing on Linux suggests that this feature is actually slightly broken on that platform. Specifically, Cabal must ensure that objects linked into a shared object are built with -fPIC (which may mean using the dyn way, since all transitive objects in the final link must be position independent). It's quite unclear how the original design intended this to be ensured.

[Mikolaj]

Is it broken with GHC 8.10 as well? I assume it was broken for a long time?

[Mikolaj]

@bgamari: you mentioned Windows and Linux. Could this affect M1 as well, as in #7837? If so, we can ask @vmchale to test once the PR is complete.

[bgamari]

Is it broken with GHC 8.10 as well? I assume it was broken for a long time?

Indeed, as far as I can tell this is just a hole in the design. Specifically, one can only expect to build a foreign library on most platforms in the dynamic way. Perhaps Cabal should throw an error to catch this case?

[Mikolaj]

My suspicion is that Cabal should either implicitly enable --enable-dynamic-libraries' when building foreign library`s on some platforms or refuse to build them if it's disabled.

Sounds reasonable. I think, as long as the error messages are clear, it's less surprising when stuff refuses to build and asks to enable --enable-dynamic-libraries than when it's enabled implicitly, the user doesn't notice and is utterly confused why the build products are not static as requested.

[bgamari]

Unfortunately the issue is even more confused than I initially thought. It appears that, in an attempt to mitigate the issue described above, GHC.Distribution.Simple.GHC.gbuild explicitly enables -fPIC when building a foreign library. Meanwhile, it passes -static, a combination that makes very little sense and which causes GHC with this patch to link against the the vanilla RTS, which isn't PIC.

Ultimately GHC doesn't make things easier here as it defines -dynamic to serve a dual-purpose: it both tells GHC to link against Haskell code dynamically and it tells GHC to produce PIC code. the ghcOptDynLinkMode name given to this flag inCabal's GHC flag type seems to suggest that Cabal is under the impression that it only did the former, which is likely the reason for the confusion here.

[bgamari]

It appears that the -static flag arises from Distribution.Simple.GHC.gbuildNeedDynamic. Specifically, that function returns False if the library type is standalone. This seems to be quite intentional but it's not clear to me why it would do this and, as far as I can tell, it's wrong.

[bgamari]

It appears that the -static flag arises from Distribution.Simple.GHC.gbuildNeedDynamic. Specifically, that function returns False if the library type is standalone. This seems to be quite intentional but it's not clear to me why it would do this and, as far as I can tell, it's wrong.

Thinking about this a bit more, I suspect that this is due to the fact that the author wanted a "standalone" library to have no dynamic dependencies. That is, they expected GHC to link against static Haskell dependencies. However, it's unclear to me how this could have ever worked on platforms that require dynamic libraries to be PIC since, while Cabal does pass -fPIC when to the GHC invocation used to compile the objects of the foreign-library, its (static) dependencies will have been built as non-PIC.

[hsyl20]

Ultimately GHC doesn't make things easier here as it defines -dynamic to serve a dual-purpose: it both tells GHC to link against Haskell code dynamically and it tells GHC to produce PIC code.

Couldn't we use -fexternal-dynamic-refs instead?

[Mikolaj]

@bgamari: any news?

[Mikolaj]

@Bodigrim points our attention to https://gitlab.haskell.org/ghc/ghc/-/issues/20520#note_411402, citing:

Unfortunately, this is at its core a Cabal bug. I have tried to fix this bug in Cabal #7764 but have run into inconsistencies in the current implementation of foreign-library that I can't currently explain. Specifically, Cabal passes -static to GHC when building a type: standalone library. However, this is inconsistent with the users intent to build a dynamic library. I'm afraid the fix for this will need to be bumped to %9.2.3 at the very least.

@bgamari: do you have any vague suggestions how to fix that in Cabal?

[gbaz]

I think based on discussion that this is a strict improvement, but there remain cases that make no sense in general with the static/dynamic issue. So we should merge this, but also open a new ticket trying to specify what the semantics actually are.

[Bodigrim]

This does not seem to resolve the reproducer in https://gitlab.haskell.org/ghc/ghc/-/issues/20520, and I'm not qualified to jusge about other benefits.

[Mikolaj]

This does not seem to resolve the reproducer in https://gitlab.haskell.org/ghc/ghc/-/issues/20520, and I'm not qualified to jusge about other benefits.

Oh dear, I thought it does, based on "This appears to fix the original ticket on Windows."

@bgamari: do we need to do anything more to fix the original issue? Remove the --static that cabal inserts? Add -fexternal-dynamic-refs @hsyl20 is suggesting?

[Bodigrim]

Sorry for confusion, I tested a wrong branch. This indeed fixes the reproducer.

[Mikolaj]

@Mergifyio rebase

[mergify]

rebase

✅ Branch has been successfully rebased

[jneira]

This should had included a changelog entry, added in #8111