Standard library reform: Split base

[Ericson2314]

Issues with the standard library are holding back the Haskell ecosystem. The problems and solutions are multifaceted, and so the Haskell Foundation in its "umbrella organization" capacity is uniquely suited to coordinate the fixing of them.

Here is proposed a first step in this process, splitting base.
Future work addressing a larger set of goals is also included to situate splitting base in context.

Rendered

[Bodigrim]

I greatly appreciate the amount of work put into the proposal, and I'd love to fall in love with it. This is an awesome idea, but IMHO (and I hate to say it!) wrong time: too late or too early.

In the current state we don't have enough resources.

Firstly, I'm not confident that the disruption on the way to the eventual goal is sustainable for the ecosystem. Indeed, we Haskellers tend to brush off such temporary concerns for the greater good of eternal shining. However, even if we accept such school of thought, I claim that even skipping ahead several years of churn we would not be able to maintain the split base in the long term.

Currently we support only 1 major version of base at a time, which needs to be compatible with exactly 1 version of compiler, and most of daily maintenance is done by GHC developers themselves. Under the proposal, to achieve the goal "upgrading GHC does not require bumping base", someone'd have to support N major series of base, each of them compatible with N versions of compiler, and do it all on their own (because GHC developers will be tasked with their internal ghc-base), maintaining N^2 instances of CI and test infrastructure.

The thing is that we don't have resources for any N > 1. So in practice, once a couple of burnt-out maintainers leave the frustrated community, we'll get back to N = 1 and status quo.

---

With regards to Step 1A: Task the CLC with defining new standard libraries, currently I don't see this happening. Ten person-years and one million dollars later? Maybe. We can barely cope with a never-ending flood of community proposals business-as-usual. There is no bandwidth for anything else unless we are full-time occupied.

[Ericson2314]

I greatly appreciate the amount of work put into the proposal, and I'd love to fall in love with it.

❤️ this is still the most positive feedback I have heard you on this sort of thing, so yes, I will take it! Thank you.

---

Firstly, I'm not confident that the disruption on the way to the eventual goal is sustainable for the ecosystem.

Well there are no actual breaking changes proposed here, base stays base in perpetuity. The new stuff is also supposed to be more stable. There is disruption on a meta-level in that this is a long marathon of work that will take up people's braincells, but I don't think more code will be broken at all.

Under the proposal, to achieve the goal "upgrading GHC does not require bumping base", someone'd have to support N major series of base, each of them compatible with N versions of compiler, and do it all on their own (because GHC developers will be tasked with their internal ghc-base), maintaining N^2 instances of CI and test infrastructure.

The thing is that we don't have resources for any N > 1. So in practice, once a couple of burnt-out maintainers leave the frustrated community, we'll get back to N = 1 and status quo.

OK a few things to take a part.

• Firstly, when people hear "N^2", they usually think asymptotic complexity. Just so we are all clear, n the length of the sliding support window, and so n is bounded at, say, 2 or 3. n^2 = 4 or 9 isn't such a large number. cabal-install has about that many CI jobs, for example.

• instances of CI and test infrastructure.

  Computers are cheap! I would gladly CI an order of magnitude more, say 100 combinations! Compared to the volunteer person-hours put into Haskell this is nothing. Maybe the dev-ops of managing that much testing is large, but if so that is simply a sign that our CI needs to be more reliable.

• someone'd have to support N major series of base, each of them compatible with N versions of compiler, and do it all on their own (because GHC developers will be tasked with their internal ghc-base)

  Considering the amount of time we waste individually (human individuals and firms) trying to upgrade our projects. I think this is less work overall. And pooling our resources so we don't inefficiently suffer in isolation is the exactly why the Haskell Foundation exists in the first place.

  And the more we rationalize the libraries the easier it should get. We have lots of friction / maintainship overhead precisely because the division of labor is so bad between these libraries today. This is why base takes a whole committee, meanwhile @RyanGlScott can maintain a fuckton of libraries almost all by himself.

  I am banking on the productivity gains here swamping the fixed initial investment; I don't think that is pie-in-the-sky thinking.

---

With regards to Step 1A: Task the CLC with defining new standard libraries, currently I don't see this happening. Ten person-years and one million dollars later? Maybe. We can barely cope with a never-ending flood of community proposals business-as-usual. There is no bandwidth for anything else unless we are full-time occupied.

So I too think front-loading the CLC bike-shedding is silly and will waste CLC time! But the feedback I got in haskell/core-libraries-committee#105 was repeatedly "let's not do anything tell we have a design of where we want to end up" . I would much rather do the ghc-base split to de-risk and start locking into those productivity gains before tasking the CLC with The Big Bikeshed, but I didn't want to write the proposal in a way people had specifically told me not too.

If there is consensus that it does make sense to explore the the behind the scene stuff first after all, I will gladly change this.

Similarly, I would much rather just straight up commit to the idea that there should be an IO-free 100% portable standard library that the IO-full ones build upon. If we can commit to that, I would also delay wrestling with what the Browser and WASI IO interfaces should look like until later --- in fact we can simply experiment with designs on top of the IO-free design and then have the CLC ratify something that is implemented vs sketch something tentative out from a blank slate.

If there is consensus that that too sounds good --- also in the past I got feedback that having too many libraries would piss off users --- I will also gladly rearrange the steps to reflect this.

---

Finally (and I should put this in the proposal), I know this proposal is a big lift --- not because it is technically challenge, but because it is administratively challenging --- but I think that can be a good thing. This can be the marquee project that the Haskell Foundation takes on, something super user-visibility and impactful, that can drive a lot of interest and fundraising.

As the saying goes, "you have to spend money to make money": I think if the Haskell Foundation can rise to the occasion and pull this up, we'll grow our resources and administrative capacity to meet the needs, and with such momentum be in a better place to tackle whatever comes next.

[hasufell]

Regarding the concern of CI complexity and testing: we have a full time engineer working on that already, employed by HF.

Afaiu their scope currently lies on fixing GHC tests and CI stability. However this work seems very much in alignment to the required work of this proposal.

@chreekat

[nomeata]

Very nice and thoughtful write up, thanks a lot! I hope it'll happen, one way or another.

[simonpj]

Thank you for this thoughtful writeup John. I appreciate it. It is not easy to navigate the best path given the differing needs of our users, and limited resources. But debating and (I hope) agreeing a North Star destination would be reallly helpful, even if it takes us a while to get there.

So I'd argue for not getting enmeshed too quickly in "we can't afford it". (Having said which, "we can never afford it" is a reasonable argument. e.g. It's a waste of time to debate which particular exoplanet we want to colonise when we have no feasible way to get to any of them.)

[simonpj]

 New Goal: Split Base

I wonder if you could elaborate the proposal to explain why splitting base will help? After all, if we become 100% clear about what is

• The external API of base (subject to PVP, subject to CLC, strong bias towards stability)
• The internal mechanism of base (not subject to PVP, nor CLC, can change at will)
  then it's not clear to me what spllitting that package into two parts achieves. Can you elaborate (in the proposal, I mean)?

[chreekat]

@hasufell , thanks for the mention. :) Yes, I'm here, and in fact you could say my current mission is to increase the GHC team's bandwidth. I trust that will have far-reaching positive effects on topics such as this one.

This is definitely an interesting topic and a cool development. I look forward to watching it progress.

I echo Simon's request for more information about how splitting base will help. As a matter of fact, I do have a couple guesses, but they are only guesses! It would be good to see the reasons fleshed out in the proposal.

[Ericson2314]

Good point. Splitting base is done to address Problem 4 without the maintenance burden explosion @Bodigrim warms of. However this is indeed not yet described well. I will update the "New Goal: Split Base" section to make the connection between these 3 things (Problem 4, maintenance cost control, and split base) clear.

[Ericson2314]

Right now we don't make any distinction about internal or not when it comes to the proposal process. There's only exposed API.

There is still ghc-prim. So we do already have the dependency transitive closure of base stuff including non-base stuff.

Once we have a package split and are working towards moving GHC internals away and possibly deprecating them from base, we'll have a disadvantage in foreseeing or understanding how GHC internals may affect exposed API.

I think that is a conversation better to have if and when we un-expose items --- not now.

Any time something is moved to ghc-internals and reexported, the situation is exactly like today: exposed items in base depend on things which themselves are also exposed in base. (Whether those definitions ultimately live is immaterial, as @simonpj says.)

The only material difference is when stuff in the closure of exposed things becomes internal (i.e. moved to ghc-internals and not also exposed in base), but as I said at above that is a separate conversation.

[hasufell]

I think that is a conversation better to have if and when we un-expose items --- not now.

I don't agree. We're trying to hash out how the future communication and collaboration is going to look like. We're not making decisions on specific items or modules (that should all be postponed).

I feel these concerns are being dismissed with "but that problem already exists", which I personally find insufficient.

A further split will make collaboration more challenging. I think that should be absolutely clear to all parties.

Postponing the discussion about these collaboration practices until someone writes a CLC proposal is not going to go well. Parties will be overinvested and it's going to blow up.

I do expect that we have absolute clarity about the collaboration process going forward, even if it touches already existing problems.

[Ericson2314]

I say this not to merely punt, but also because it genuinely depends on what the change in question is.

I would expect most things that are truly internal (not reexported in base), to have performance characteristics that don't obviously relate to base's. So in that case it is probably better to just performance test the base stuff, and not try to "pre-reason" what the effect is and constrain the performance of the ghc-internals stuff.

Indeed, my experience with GHC itself is that micro-benchmarks that do not obviously relate to real-world concerns are incredibly annoying!

On the flip side, the long term goal is to eventually have base be implementation agnostic, in the sense that it should only use "standardized" items from ghc-internals. For example, GHC may depend Int in all sorts of magic aways, but base just effects there to be some Int type with a Num instance, etc. (Ideally would formalize these interfaces with Backpack signatures or something similar.)

In that case, we should do performance tests, precisely because even though the implementation is GHC-specific, the interface is implementation agnostic. It therefore makes sense to test as a natural "narrow waist" between the wide expanse of misc GHC internals, and the wide expanse of implementation-agnostic code in base.

Also it is also not a coincidence that many of these "input interfaces" from the Haskell implementation to base are also reexported as-is as "output interfaces" of base. So per what I said, they would squarely be in the CLC's remit anyways.

[hasufell]

So in that case it is probably better to just performance test the base stuff

That's what I proposed. Was that not clear?

Imagine:

1. exposed internal function is proposed to be moved (the implementation) from base to ghc-internals
2. another exposed base function (something non-internal like foldr) depends on it

This now requires a performance regression test to be added for foldr. Because that is under CLC purview. Given that we may also be more lax on proposals of still exposed internals wrt haskell/core-libraries-committee#146 I believe the test should be added right then, not just when the internal part is finally unexposed from base.

This was my proposal after @Bodigrim raised further concerns. Is that unreasonable?

[goldfirere]

As a more-removed party from this discussion, I have to say I sympathise with e.g. @hasufell's concerns here. While it's true that performance changes might happen accidentally in base today (because there is no comprehensive CI testing perf) -- and that the ghc-prim loophole exists -- I see the change proposed here as something that will make CLC monitoring of base perf changes harder. It's not a binary change from "excellent perspective" to "no perspective", but I can see why it would be qualitatively different after this proposal.

I don't have a good concrete proposal for how to close this gap. @hasufell's suggested added perf tests are a fine idea in theory, but I see how they could become a blocker in practice.

But, lacking a good concrete proposal, let me put forth a bad abstract one. (It's bad because it's hard to action, not a bad idea.) Proposed: Work to increase trust. The key problem around which the struggle in this thread has hinged is what I perceive to be a low level of trust from the CLC toward the GHC developers. Furthermore, I think this low level of trust is based on concrete evidence, not on arbitrary prejudice. To my eyes, it boils down to a difference in goals and values: the CLC's remit (and the individuals on the committee) is about maintaining an excellent base. This requires rock-solid stability. On the other hand, GHC continues to strive to be an excellent compiler, with lots of room for experimentation. While not eschewing stability, GHC has generally been more accepting of breaking changes. Given a handful of non-backward-compatible changes that GHC has adopted recently (e.g. simplified subsumption, the change to instance solving that had to be backed out at the last minute, the very high churn in the GHC API, the changes in performance of compiled text, possibly more), I can see why the CLC is worried.

I'm not suggesting that anyone is acting (at all!) in bad faith or that anyone is making poor decisions. Instead, I'm suggesting that different people have different levels of tolerance for breakage and different values around how to maintain a language and its library. Diversity on this point is essential for Haskell -- too much in any one direction would destroy what has made Haskell great.

All that said, I think it's important to highlight this difference in approach so that everyone can work toward structures that can help to bridge this gap. @bgamari's automated check for API changes and @hasufell's suggested performance tests serve this goal very nicely. That is, they replace trust -- hard to achieve, hard to maintain, hard to transfer -- with verification. (Which is, I suppose, also hard to achieve and hard to maintain, but perhaps more in this group's wheelhouse.)

Perhaps another high-level point worth making here: no one is going to get everything they want. Which is probably also best for Haskell, given the forces tugging at different directions here.

[Ericson2314]

@hasufell

OK. I am sorry, I did miss which item you intended to test. Testing foldr is indeed more reasonable. But again condition 1 should not be based on merely moving the code, but just moving code that is also won't be reexported. Only newly internalized, not newly moved code should fall under any such distinction.

There is a larger point here that I think the overall social goal of free and open source software is to organize code based on what it is / what it does, not who controls it. In other words, it is to defeat Conway's law.

I started writing this before he posted, but it actually dovetails with what @goldfirere says very nicely. It is precisely in low trust situations, which is indeed what we have year, that the urge to organize code by ownership is strongest --- to hunker down in ones fortress, whether that is a repo, or a directory within a repo, or anything else. But this is not a good solution firstly because the low trust problem in and of itself is worth solving, as @goldfirere points out, but because the costs of a bad architecture based on groups of people not engineering concerns intrinsic to the task at hand is also a cost, not worth paying, and one that itself exacerbates trust issues ---- the boundaries between Conways-laws components rarely have simple, well-defined interfaces that actually enable good delegation!

---

Going back to the foldr example. Suppose:

• a1 a2 a3 are functions in base which depend on b.
• b is a function which in turn depends on c; both are also currently in base. The as do not use c directly, only through b.
• b and c are slated to be moved to ghc-internals, because their implementations are both GHC-specific.
• b will be reexported by base because while its implementation is GHC-specific, its interface is GHC-agnostic. (Or, because it is widely used and grandfathered in, or any other hypothetical reason.)

Also suppose we are agreeing to performance or other testing as a trust-building exercise.

Per the original condition 1 based purpose on code motion, a1, a2 and a3 must each individually be tested. This is potentially a lot of work, as the number of as increases.

Per my adjusted condition 1, since b is reexpported, and since the use of c by stuff in base is only via b, it suffices to just test b. This means means we need to write fewer tests, assuming there are more as than bs, which should be a hallmark of good interface design anyways.

@hasufell's suggested added perf tests are a fine idea in theory, but I see how they could become a blocker in practice.

Right, and so for any version of that to be workable, per the above example I think it is crucial that we don't penalize reexported moves, but allow testing them directly, and don't need to additionally test other stuff whose used of truly internal stuff merely "factors through" the reexported functions.

Going broad again, we should always strive to make choices that gets us closer to the "narrow waist" ideal where we have minimal, scoped and standardize interface and contract between ghc-internals and base. And allowing for factored-through usages to count per any "verification" requirement is crucial for that.

[simonpj]

Right now we don't make any distinction about internal or not when it comes to the proposal process. There's only exposed API.

That's right. And I'm not proposing any change whatsover there.

This changes status quo.

Can you elaborate on what would change from the status quo? We are already committed to consulting CLC on user-visible changes to the base API. No change there.

There are already many hidden modules in base, any of which could in principle change the semantics or performance of the exposed base API. We should consult if these change performance.

Likewise ghc-prim and ghc-bignum.

All of this is the status quo.

Perhaps you mean that members of CLC might pro-actively monitor the GHC repository (in which base lives), looking for changes to base? I did not know that you did that, but you of course are most welcome to do so. And you'd be free to continue to do so, monitoring ghc-internals as well which will also be part of the GHC repo, along with ghc-prim and ghc-bignum. No more code is involved than you are monitoring today, if indeed you are doing so.

Does that make sense?

[Bodigrim]

A second point:

• Stuff in ghc-experimental is intended to have its ultimate home in base, if validated by experience.
• Stuff in ghc-internals is not intended ever to be in base. (Exception: if something is so useful that users cry out for a stable API to it.)

Fully in agreement.

MIght we benefit from separating these goals to different layers?
ghc-internals, base, ghc-experimental

So I can see the logic here. I am agnostic about whether it is worth the additional huff and puff.

My concerns are two-fold:

• Reduce amount of stuff, which is not internal per se, but required to implement ghc-internals. Re-exports are harmful.
• Remove as much incentive to use ghc-internals as possible. It should not be OK to reach out for it to play with dependent types or label selectors.

Hyrum's Law states that people will abuse ghc-internals in every possible way. We should not assume that they won't just because documentation advises against. Any potentially useful helper, exported from ghc-internals, will be depended upon. And ten years later it would not really matter anymore whether there was a disclaimer or no, the only thing which would matter is that half of your ecosystem transitively depends on some GHC.Internals.foo and you absolutely do not want to break it. We've seen this process of ossification happening in ghc-prim and in GHC.* namespace of base, and it makes absolutely no sense to introduce ghc-internals only to end up in the very same place couple of years later again.

That's basically why I'm opposed to the original proposal. Re-exporting entire base from ghc-base would just mean that very soon everyone depends on both, and my work as a Hackage trustee will be even more nightmarish than it is.

Let me re-iterate. Unless people are actively disincentivised from using ghc-internals by all possible technical and social means, they will rely on it massively. Couple of years down the line they will start complaining that there is too much breakage. Five years later the community will insist on installing an Internal Libraries Committee to overview and stabilise ghc-internals.

---

I keep saying that re-exports are bad. It's true that this is not a completely new problem, but it does not give us a leeway to aggravate it recklessly.

1. Re-exports make import choices difficult both for humans and for tooling. This is already challenging even within the same package. Which module am I supposed to export to access foldl1'? GHC.List, GHC.OldList or Data.List? I know the answer, but newcomers don't and, even worse, tooling such as HLS doesn't. Autocompletion will autoimport any of the modules above.

   Spreading re-exports over several packages makes situation exponentially worse, especially if HLS Cabal plugin learns how to add new dependencies automatically. Type when in any project with mtl<2.3 and ask HLS to autocomplete imports: there are 17 (!) suggested modules.

2. There is a PVP-related issue with re-exports. Namely: you can control re-exported functions and classes, but not instances, which are implicitly re-exported in full. If package A defines a type class X, re-exported from package B, then package B must use tight bounds A >= X.Y.Z && < X.Y.(Z+1), because even minor releases of A can introduce new instances, leaking through the API of the very same version of B. A new release of B is required to allow A >= X.Y.(Z+1) && < X.Y.(Z+2), and so on. Both packages must evolve and be released in lockstep.

   This is a very difficult property to maintain and verify across packages, even on much smaller scale than proposed.

---

I think that is a conversation better to have if and when we un-expose items --- not now.

Reading HF TWG meeting notes, there seems to be a rush to ram the proposal asap ("It will be worrying if we don't get this done by next meeting..."), which leaves me puzzled and bewildered, as I do not possess such notion of urgency. Paraphrasing a repeated argument here about re-exports, "this is not a new problem". Unless there is a burning need to tick the box soon, I would strongly suggest to spend as much time as needed to think at least a few steps ahead.

It seems likely to me that a proper solution requires more control mechanisms than are available right now. It seems wise to design and develop them before, not after.

[bgamari]

That's basically why I'm opposed to the original proposal. Re-exporting entire base from ghc-base would just mean that very soon everyone depends on both, and my work as a Hackage trustee will be even more nightmarish than it is.

I am sympathetic to this argument. We could easily implement a new warning mechanism, -Wpackage-import=<pkg>, which would throw a warning if the user attempts to import a module from the given package. We would then add -Wpackage-import=ghc-internals to the default flag set. In the future we could add -Wpackage-import=ghc-prim to this set as well.

1. Re-exports make import choices difficult both for humans and for tooling. This is already challenging even within the same package. Which module am I supposed to export to access foldl1'? GHC.List, GHC.OldList or Data.List? I know the answer, but newcomers don't and, even worse, tooling such as HLS doesn't. Autocompletion will autoimport any of the modules above.

This sounds to me like a technical problem in need of a technical solution. I can think of a few possible approaches:

• tooling (and GHC) could respect existing {-# HADDOCK not-home #-} pragmas when suggesting imports
• we could introduce another module-level, non-Haddock-centric pragma which accomplishes the same end as above (e.g. {-# NOT_HOME #-} module GHC.List ...)
• we could introduce a per-declaration pragma which could be used to specify the recommended "home" module of a given declaration (e.g. {-# HOME foldl' Data.List #-})

2. There is a PVP-related issue with re-exports. Namely: you can control re-exported functions and classes, but not instances, which are implicitly re-exported in full. If package A defines a type class X, re-exported from package B, then package B must use tight bounds A >= X.Y.Z && < X.Y.(Z+1), because even minor releases of A can introduce new instances, leaking through the API of the very same version of B. A new release of B is required to allow A >= X.Y.(Z+1) && < X.Y.(Z+2), and so on. Both packages must evolve and be released in lockstep.

This is certainly true in principle but I do not believe that it bears on the proposal at hand. In particular, GHC developers have no interest in exposing new instances to exposed types without CLC approval. Most of base makes minimal use of typeclass overloading internally and my interface stability checking work already verifies the stability of instances.

Reading HF TWG meeting notes, there seems to be a rush to ram the proposal asap ("It will be worrying if we don't get this done by next meeting..."), which leaves me puzzled and bewildered, as I do not possess such notion of urgency. Paraphrasing a repeated argument here about re-exports, "this is not a new problem". Unless there is a burning need to tick the box soon, I would strongly suggest to spend as much time as needed to think at least a few steps ahead.

As Simon points out above, there are currently several pieces of work, some of which slated to ship with the up-coming GHC 9.8 release, which are blocked on this issue. At this point I am quite worried that without some sense of urgency we will need to further delay one or more of these items as the fork date is quickly approaching.

[Ericson2314]

Hyrum's Law -style reasoning I think is good, and is indeed why splitting base as opposed to merely documenting modules is the only thing that could possibly work --- users will use the entirety of any library they depend on, but there is a chance of getting them to not use libraries entirely. So I am remain convinced that splitting ghc-internals is a necessary step.

Perhaps you raise some good points that it is not sufficient, but that's OK! Between now and "a couple year's time", we can implement things, and as @bgamari says many of this constituent problems sound like they are amendable to excellent technical solutions:

• Backpack for codifying the interfaces base expects from ghc-internals.
• Warnings / hidden documentation / etc. if one uses the original module directly instead of the reexported one
• Non-exported instances (but without coherence issues people assume this causes) (good for untangling modules, not disagreeing with @bgamari that GHC devs don't want to add internal-only instances)

New tooling / language feature work to allow us to improve the architecture and stability of our ecosystem is extremely high value, both for base's and downstream libraries' sake. If renewed interest in this line of work is caused by a ghc-internals split, I can think of no better outcome.

[Bodigrim]

As Simon points out above, there are currently several pieces of work, some of which slated to ship with the up-coming GHC 9.8 release, which are blocked on this issue. At this point I am quite worried that without some sense of urgency we will need to further delay one or more of these items.

In which sense are they "slated to ship with the up-coming GHC 9.8", if they are not approved? Or if they are fully approved by all relevant parties, what blocks them?

Are you referring to exception backtraces? The PR is still labelled as "needs revision" and was not deemed urgent for three years, during which GHC Steering Committee has been enjoying its freedom to bikeshed it. I don't believe it suddenly became urgent overnight.

[bgamari]

In which sense are they "slated to ship with the up-coming GHC 9.8", if they are not approved? Or if they are fully approved by all relevant parties, what blocks them?

Are you referring to ghc-proposals/ghc-proposals#330?

This is indeed the best example.

The PR is still labelled as "needs revision" and was not deemed urgent for three years, during which GHC Steering Committee has been enjoying its freedom to bikeshed it.

As far as I am aware, that label does not reflect the current state of the proposal. I have expressed that, from my perspective, the proposal is finished to the committee and I have not yet heard any objections. I think that the time we took refine the proposal was well-spent; the proposal is much better than it was two years ago. However, it has now been converged for two weeks and do hope that we can close this chapter soon.

I don't believe it suddenly became urgent overnight.

The issue addressed by this proposal has been a major thorn in the side of industrial users for as long as Haskell has had commercial adoption. Every few months we see yet another call to paper over this issue with HasCallStack at the expense of code-size and runtime performance; the last such call, haskell/core-libraries-committee#115, was closed specifically due to the expectation that exception backtraces were not far off. This, to me, signals that there is indeed some urgency to resolving this matter.

[Bodigrim]

As far as I am aware, that label does not reflect the current state of the proposal.

I see what I see:

• The proposal is not yet approved by GHC Steering Committee (despite three years of discussions).
• The proposal is not yet approved by CLC (because you raised a proposal less than two weeks ago).

Why would you claim it being "slated for GHC 9.8" is beyond my understanding and makes a weak argument to require urgent changes to base architecture. But we digress.

[cartazio]

It really sounds like at the end of the day, as long as there are tools to support internals code from leaking into base non-explcitly, or at least a commitment to making such tools, most of the concerns are resolved right?

I believe the reason it’s a blocker is because basic support for new ghc features is historically exposed in the base library, and this proposal provides a path to not having it be so from the get go?

[tek]

Would it be feasible to prohibit Hackage uploads for packages depending on ghc-internals, pending CLC approval?

[bgamari]

Why would you claim it being "slated for GHC 9.8"

@Bodigrim, with every release, the GHC project has a list of items which we strive to finish and ship. The implementation of the backtrace exception proposal is one of the items which we hope to be a headline feature of GHC 9.8.

[Bodigrim]

Proposed: Work to increase trust. The key problem around which the struggle in this thread has hinged is what I perceive to be a low level of trust from the CLC toward the GHC developers. Furthermore, I think this low level of trust is based on concrete evidence, not on arbitrary prejudice.

I cannot extrapolate on the whole CLC, but this is true about myself. I have little trust in GHC developers to manage changes, and this distrust is not hypothetical or speculative.

Our skirmish with @bgamari "there is a huge change, which GHC SC has not yet approved after three years, but it's urgent, and it has never crossed our minds to ask CLC about it until last two weeks, but it's URGENT, so let's split base" just reinforces my feeling that our approaches to stability, managed evolution and control environment are vastly different. I still hope they are not incompatible.

That said, I'm afraid I've overinvested in this discussion and have no more bandwidth. Happy to continue the discussion at Zurihac and later.

[simonpj]

I agree that we should not make bad decisions through rushing, but I am very keen to use our current momentum to move towards decisions, rather than for us to get worn out and discouraged and end up doing nothing and doing it all again in a year.

We have made a lot of progress. We agree about the broad goals (stability of base, freedom for GHC to innovate in a way that does not threaten that stability). I think we can work together to agree something that meets all our goals. I'll take as my starting point my post last week, plus subsequent clarifications that changes to ghc-internals that affect the base API (incuding types, semantics, and performance) are subject to the CLC purview.

To that, here are three further thoughts.

ghc-experimental

@Bodigrim argues that we should split base into three:

• ghc-experimental, initially empty, depends on base. Functions and data types here are intended to have their ultimate home in base, but while they are settling down they are subject to much weaker stability guarantees. Example: new type families and type constructors for tuples, GHC Proposal #475.

• base: as now, curated by CLC. Depends on ghc-internals, and hence on ghc-bignum and ghc-prim.

• ghc-internals: exposes aspects of GHC's internals that may be of interest to "hard-core" developers interested in maximum performance.
  See Nikita's blog post.

Other things being equal, two is better than three, but @Bodigrim makes good arguments that the extra overheads are worth it. Specifically:

• A major reason for splitting base and ghc-internals is to signal unambiguously what is accidental, changeble, and exposed for hard-core developers (ghc-internals), and what is stable and curated (base). But ghc-experimental isn't accidental, changeable, and exposed for hard-core developers. It is carefully designed, intended ultimately to be stable, and encouraged for ordinary users to give it a go.
• He says (and I agree) that users should be actively disincentivised from using ghc-internals by all possible technical and social means. Why? Because if they start to rely on ghc-internals they will later complain when it changes, regardless of what it says on the tin. By putting experimental features in ghc-internals we risk establishing the idea that using ghc-internals is normal behaviour, whereas actually we want to discourage it. We can't actively disincentivise importing ghc-internals and at the same time say "to use Dependent Haskell style tuples, import them from ghc-internals".
• Suppose we only had ghc-internals. If one of these new experimental features (exported from ghc-internals) depended on an existing base function, that function would have to be in ghc-internals too. @Bodigrim and I differ about the importance of this, but I completely agree that the fewer constraints we have the better.

In short I'm persuaded. Let's have ghc-experimental.

---

Re-exports

My hope is that we can resolve the concerns about re-exports.

Re-exports make import choices difficult both for humans and for tooling. This is already challenging even within the same package. Which module am I supposed to export to access foldl1'? GHC.List, GHC.OldList or Data.List? I know the answer, but newcomers don't and, even worse, tooling such as HLS doesn't. Autocompletion will autoimport any of the modules above.

Spreading re-exports over several packages makes situation exponentially worse, especially if HLS Cabal plugin learns how to add new dependencies automatically. Type when in any project with mtl<2.3 and ask HLS to autocomplete imports: there are 17 (!) suggested modules.

I agree with your diagnosis, but I'd like to suggest that the proposed split above makes these things much better, not worse. As you say, the current situation is problematic. But with the above split, the answer for tooling (e.g. HLS), and for users becomes simple:

• Use the import from base.
• Unless the user explicitly asks for it, don't even offer any imports from ghc-internals (this is part of the active disincentivisation).
  We get a simple, machine-supportable criterion for where to import from. By thereby moving stuff off the "potential import" list, we reduce the choices the suer is faced with, and give much clearer signalling about where to import it from. Everyone wins.

There is a PVP-related issue with re-exports. Namely: you can control re-exported functions and classes, but not instances, which are implicitly re-exported in full.

This is a good point. But to me, the instances are part of the API. If ghc-internals defined a new instance for, say C T, where both C and T are available from base, that would constitute a change in the API of base that we should consult the CLC about. No changes to the base API without consultation!

On the hand, if ghc-internals defined a new class, visible only by import ghc-internals or ghc-experimental, that would make no difference to clients of base. So that's fine.

---

CI support

At the moment we have very little support for checking the stability of the base API. As part of this discussion we
have proposed adding to the CI system:

• Checking for the base API changes.
• Compiling a good chunk of Hackage against base
• Running some of their test suites (a check on semantics)
• Running some of their perf tests (a check on perf)
• A new export deprecation mechanism, which allows the CLC to direct users to the ``right'' place to import a function.

These are substantial improvements over the status quo which will, I hope, improve the stability of the base API
on which we all rely.

[bgamari]

Our skirmish with @bgamari "there is a huge change, which GHC SC has not yet approved after three years, but it's urgent, and it has never crossed our minds to ask CLC about it until last two weeks, but it's URGENT, so let's split base" just reinforces my feeling that our approaches to stability, managed evolution and control environment are vastly different. I still hope they are not incompatible.

@Bodigrim, for better or worse there is a long history of GHC Proposals touching base without a formal CLC proposal. haskell/core-libraries-committee#164 stemmed from my observation that none of the CLC had commented on the proposal as of a few weeks ago, at which point I thought it would be appropriate to extend a formal invitation to do so. It was absolutely not my intent to subvert the CLC's authority and, in hindsight, I should have asked for feedback sooner.

I am not opposed to the CLC weighing in on such matters; far from it. However, whether this change requires a separate CLC proposal is, in light of history, far from obvious. Going forward, it would be useful to explicitly lay out the expected interaction between the GHC and CLC proposal processes to prevent this sort of ambiguity in the future.

[konsumlamm]

Let me re-iterate. Unless people are actively disincentivised from using ghc-internals by all possible technical and social means, they will rely on it massively. Couple of years down the line they will start complaining that there is too much breakage. Five years later the community will insist on installing an Internal Libraries Committee to overview and stabilise ghc-internals.

@Bodigrim Do you have any evidence for this? The only package I know of, that fully consists of internals noone should use, is ghc-prim and I see noone asking for stabilisation of it. Note that until recently, it wasn't even documented that one shouldn't use ghc-prim, you just had to know! Likewise with packages that expose .Internal modules (like containers, vector, ...), I've never seen anyone ask for stabilisation of those (and they don't explicitly say that one shouldn't use them either, just that they're internal and unstable).

Would it be feasible to prohibit Hackage uploads for packages depending on ghc-internals, pending CLC approval?

@tek while I agree that we should discourage users from using GHC internals, I think requiring CLC approval for such packages is wrong. Why should the CLC care what shenanigans you want to use ghc-internals for? CLC members aren't necessarily familiar with GHC internals, so they'd be the wrong persons to judge that anyway imo.

[cartazio]

Would it be feasible to prohibit Hackage uploads for packages depending on ghc-internals, pending CLC approval?

I think that’s a dangerous position to take. If someone is determined enough to write the correct cpp macros/ conditional compilation to have stable code make use of some internals, that’s great. If it’s not written that way, they can go pound sand when their code breaks.

[cartazio]

This discussion is really about making sure base doesn’t silently export unstable stuff. Right? We aren’t talking about dictating how developers choose to use stable vs unstable apis, right? Just about making base robustly stable while still allowing ghc to evolve ?

[hasufell]

Wrt performance

I seem to not have done a god job at explaining this. My point was maybe a bit finicky/fine-lined.

Can you elaborate on what would change from the status quo?

GHC internal functions that get unexposed from base are no longer under CLC purview, but may still affect performance of public base API. CLC will lose an (indeed) accidental advantage, so to speak.

This will change status quo, no matter if this problem already exists elsewhere.

And this is really just a signal of a deeper issue that underlies the CLC charta:

The ownership of base belongs to GHC developers, and they can maintain it freely without CLC involvement as long as changes are invisible to clients. Changes which affect performance or laziness and similar are deemed visible.

Here it says very clearly that GHC developers cannot maintain things freely (even internals) if they affect performance. The only proper way to know if performance is affected is to have performance regression tests.

Exercising common sense is not as good as a test suite, especially if there's already reservations wrt the way changes are executed.

What I'd like to figure out is if we can approach this problem gradually as well.

I also want to note that I think the charta needs adjustment, because here it only talks about base (and not about e.g. ghc-prim). We can't do our job if we have no say over those packages. This is an oversight and needs to be fixed IMO.

Touch points

So, what you're proposing wrt CI seems already quite nice to me.

We need to automate as much of those concerns of the charta as possible to minimize the use of "common sense" or "manual evaluation". This is in both parties interests, because it will reduce the required explicit communication (which is the most costly).

E.g. things of interest may be:

1. is PVP adhered to?
2. have there been any API changes to base?
3. have there been any semantic, laziness or performance changes to base functions?
4. have there been any semantic, laziness or performance changes to GHC internal functions that may affect base?
5. what is the actual total closure of (internal or not) functions that affect exported base API?

Much of this could be statically checked/generated. It could allow both explicit monitoring by CLC, as well as aid GHC developers in deciding when to explicitly consult CLC.

You asked:

Perhaps you mean that members of CLC might pro-actively monitor the GHC repository

Yeah, absolutely not (for me). I have neither the bandwidth nor the expertise. CLC shall be involved when it's relevant, but the interpretation of "relevant" should be very strict and easy to validate.

The remaining question might be how CLC can actively validate that GHC HQ has not accidentally missed something (other than through monitoring every single MR). I think this wouldn't be hard as part of a pre-release effort that lays out the data: executed tests, API diffs, generated code closure information, etc. ... and allows CLC some time to review.

---

If you ask me who's going to do all that work (e.g. wrt performance tests)... I really don't know. We will have to figure out if this can be done gradually as well. But really, we seem to agree that the idea is to reduce explicit communication and reliance on manual evaluation and instead rely on data, static checks, executed tests, etc.

Does this sound grandiose? I don't know... to me this sounds pretty Haskell!

NB: Knowing when to involve someone is really hard... I know this from the tooling side, because things are extremely interconnected and sometimes almost invisibly so.

[tomjaguarpaw]

I cannot extrapolate on the whole CLC, but this is true about myself. I have little trust in GHC developers to manage changes, and this distrust is not hypothetical or speculative.

I have had a lot of interaction with some members of Team GHC over the past 12 months or so and I am confident that the issue of stability has been firmly registered on their radar as something that is critical for the success of Haskell. That said, I can understand why others would remain yet to be convinced. What would convincing look like in these circumstances? Doubtless it would involve "walking the walk" rather than just "talking the talk". But which walk exactly? What initial steps should we take here to build trust?

[simonpj]

Thanks @hausfell, that is really helpful. We seem to agree about a lot.

GHC internal functions that get unexposed from base are no longer under CLC purview, but may still affect performance of public base API. CLC will lose an (indeed) accidental advantage, so to speak.

In the plan I suggest above (point 3), no functions whatsoever will be un-exposed from base. The base API will be absolutely unchanged. To be sure, the GHC team will subsequently make CLC proposals to un-expose functions that should never have really been there, but that's a subsequent step, and one which the CLC can resolve on a case-by-case basis.

Moreover, as things stand, even for exposed functions a CLC proposal is only needed if there is a material change (including performance); and that will remain unchanged.

So I'm not sure what advantage (however accidental) the CLC will lose. I don't want the CLC to lose any advantages! I still think the status quo holds pretty much unchanged.

Here it says very clearly that GHC developers cannot maintain things freely (even internals) if they affect performance. The only proper way to know if performance is affected is to have performance regression tests.

We agree! Even with 100% trust, loads of common sense, and huge expertise, there is no way to be sure that there won't be performance regressions by guesswork. The only thing that will work an agreed set of performance tests. I would welcome that, and (another thing we agree on) they should apply to changes in ghc-internals, in ghc-bignum, in ghc-prim. Actually if a change to GHC itself (notably the optimiser) causes a regression in (say) the perf tests for containers, I would jolly well like to know. (In fact, I think it's far more likely that a change in GHC's optimiser causes a perf regression than a change in ghc-internals or base.)

But can we agree that

• The question of adding CI for performance is an orthogonal issue:

  • It is already a challenge to know what might cause perf regressions.
  • The challenge is not made harder or easier by splitting off ghc-internals.
  • But this conversation has given a welcome and substantial impetus to improving CI-for-base (incl API checking and perf)

• Provided we agree (as we appear to be doing) the criterion for CLC consultation (changes to the base API, including perf, as discussed above), it really doesn't matter a bean whether the code itself lives in base or in ghc-internals, or indeed ghc-prim. It is all equally under CLC purview, insofar as it affects base API, semantics, and perf.

I don't think your plan (call it "CI-for-base") is grandiose. There is indeed Real Work to do here. But some bits of it are under way already; and I think we could invite the Haskell Foundation to consider helping resource some of the rest, so that it doesn't just stall.

Meanwhile, I plead that we don't we stall our base restructuring while this CI-for-base work gets done; it won't make anything worse, and will make some important things better.

[david-christiansen]

I think we could invite the Haskell Foundation to consider helping resource some of the rest,

This is very much the kind of thing that we'd like to support, and also the kind of thing that we actually have this proposal process to hash out. Let's talk with @chreekat at Zurihac and see what bits are missing, and see if we can get a plan together to support it.

[Bodigrim]

This is getting tangential, and I must apologise for broad strokes.

I have had a lot of interaction with some members of Team GHC over the past 12 months or so and I am confident that the issue of stability has been firmly registered on their radar as something that is critical for the success of Haskell.

There is a difference between "having it on a radar" and "sharing a value". Indeed I was under impression that repeated calls for stability were not unheard, but the run-up to GHC 9.6 release demonstrated significant deficiencies of control environment in almost every area. And yet we are not having here a heated discussion (or better: decisive action) to improve the control environment; instead GHC developers are enthusiastic to innovate even faster.

There is no merit to proclaim stability as your value, if in every choice between stability and innovation you prioritise innovation. And also there is absolutely nothing wrong with having different values, but it is harmful to pretend to have the same.

Meanwhile, I plead that we don't we stall our base restructuring while this CI-for-base work gets done; it won't make anything worse, and will make some important things better.

Clearly, working on CI-for-base also "won't make anything worse, and will make some important things better". But Simon demonstrates his system of values, prefering to spend limited resources of our very small community on innovation, not on stability.

What would convincing look like in these circumstances? Doubtless it would involve "walking the walk" rather than just "talking the talk". But which walk exactly? What initial steps should we take here to build trust?

We can start with a talk: a public statement from GHC developers about the place of stability in their system of values would be helpful.

If GHC developers not just have stability on their radar as a nuisance to innovation, but share it as a value, it would be helpful to exercise it more often by prioritising work on stability at the expense of other directions.

---

Back to the topic. @simonpj I'm content with #47 (comment), I'm truly grateful to you for leading the discussion.

[simonpj]

Back to the topic. @simonpj I'm content with #47 (comment), I'm truly grateful to you for leading the discussion.

Thanks @Bodigrim. It sounds as if we are close to having a plan that we can all support -- not just reluctantly but genuinely. Perhaps I'll try to redraft some of the above posts so we have the plan in one place to review. We expect to meet at Zurihac, which will help.

If GHC developers not just have stability on their radar as a nuisance to innovation, but share it as a value, it would be helpful to exercise it more often by prioritising work on stability at the expense of other directions.

I am a GHC developer, and I defintely share the stability of base as a value. I want base to be stable. I want the CLC to curate it assiduously. I want it to follow the PVP. I want ways (embodied in CI) that tell us if we (however accidentally) make it unstable.

I also want GHC to be able to innovate. Not just for selfish reasons (although those too of course) -- I think innovation is a foundational part of Haskell's attractiveness and culture.

Finally I believe that innovation in GHC is entirely compatible with stabilty of base -- provided we give ourselves some new mechanims (specifically ghc-internals and ghc-experimental) that make it possible to meet both aims.

I accept that actions speak louder than words, but we have to start with words. And while I can only speak for myself I believe that other members of the GHC team agree with, and will support in practical ways, the statements I make above.

Resourcing is always an issue. Many of us (GHC devs, CLC members) are volunteers with other day jobs. That places limitations on what we can achieve: none of us can write a blank cheque. But I don't see resourcing as the primary constraint here; we can get a long way just by engaging in constructive dialogue, and deepening trust -- as I believe we have been doing in the last few weeks.