Decide on core principles against which to judge breaking changes #12
Comments
santiweight
changed the title
|
I strongly support this. I will have my own personal opinion about the |
|
Yes exactly - currently we have well-meaning people with thoughtful but differing opinions that cannot be resolved. That is exactly what guidelines aim to prevent! |
The
I'm sorry, but I strongly believe that this is a rabbit hole of getting nothing done. "Well-meaning people will have thoughtful but differing opinions" about proposed guidelines (and their application) as well, asking for higher level principles and ad infinitum. It's untrue that differing opinions about @santiweight @tomjaguarpaw you are most welcome to come up with a set of guidelines and their implications for |
|
I'm sorry but I don't buy this at all... Understand, of course, that I value your work and contribution! What if I want to say, rewrite the But worse yet, I get the impression from the way you're discussing this that there is no plan for What about namespace changes? Can they happen any time? This is crucial information for not just a proposal but for
Yes but they are voted on once. If there are no voted-on guidelines, then we will persistently rehash the same backwards compatibility/extreme breakage discussions and never be able to dismiss arguments as "not within I'll cook up some guidelines to discuss, but I think the most important thing here is to have some values and process, even if they are unilaterally constructed by the CLC. Decision of almost any kind would be better than a lack of principles. Btw, for related work, see the "GHC evolution principles" proposal that Richard Eisenberg posted recently. The specific intention being:
|
Usually each release of GHC is accompanied with a major release of
I'm not sure what you mean by "namespace changes". Potentially they can happen in every major release of
Please refer to https://github.com/haskell/core-libraries-committee/blob/main/PROPOSALS.md for a guidance on how to raise a proposal and what is in scope. |
|
Thank you for the response :) I am not clear that there is any place where any of this information is outlined. I am only aware of the fact that You say that each release of GHC (which I believe to be every ~6 months) is accompanied by a release of I am confused - the proposals readme doesn't say anything about what is in scope according to my reading (I've read it through a few times to be sure). I'm not trying to be facetious, but how do I know whether, for example, exporting
Is your point that any change related to the core libraries is a reasonable proposal? |
Major releases of
Such proposal would be in scope for CLC.
The paragraph you just quoted gives an exact description of what is in scope: pretty much any non-trivial change to |
|
How about a documentation principle, to include counter-examples for type classes: https://blog.functorial.com/posts/2015-12-06-Counterexamples.html |
|
@Icelandjack documentation is generally out of scope for CLC. |
|
(Not a CLC member) I'd like to share my view on the subject as a maintainer of multiple open-source Haskell packages. I don't represent any group, this is just my vision and personal opinion. Maybe some can empathise. I understand the existing discussion as follows (correct me if I'm wrong):
I think Scope is fine. It makes sense to me to have a separate group of experienced people to consider all changes to the standard library as they affect all Haskell developers. However, I have some concerns regarding Outcome. I think this approach is flawed. It means that the result depends not on some objective guiding principles but on the personal preferences of currently elected members. It's not clear what are the preferences of those people (you can deduce some logic from previous decisions or from knowing such people but it's a lot of work and different people can infer different views). In other words, if you open a proposal at a different time with different members, you might get better luck to see your proposal accepted. Another point that concerns me is that the existing approach puts CLC members in a position of uncontrolled power. They (in theory) can accept or reject changes based on their mood, personal preferences, or maybe even some personal qualities of the person who opens a proposal. I'm not at any moment saying that CLC has people with such harmful views. But why allow such a possibly dangerous situation to occur if it's extremely easy to prevent this from happening at all? Isn't "making illegal states unrepresentable" one of the Haskell mottos? Therefore, I believe, it's extremely important to write down explicitly the goals behind the On a different note, I would like to express my personal views on breaking changes and breaking backwards compatibility. I think that with the current state of the Haskell ecosystem breaking changes must not be allowed unless:
or
The main goal of the Haskell Foundation is to "broaden the adoption of Haskell". CLC is affiliated with HF. From the website:
In my personal view, introducing breaking changes decreases the adoption. So I don't understand how much this affiliation means in practice. More importantly to me, almost all people maintain Haskell libraries for free in their free time. By accepting breaking changes you ask volunteers to do even more work if they want to use newer GHC versions on top of their generous contribution to the ecosystem. I would like to emphasise the following: When maintainers constantly need to fix tedious issues for no apparent reason, they can easily burn out, leave the community and stop improving the ecosystem. Again, if |
|
Would it be possible to require that a breaking change includes a migration strategy and patches for the main projects that are affected? |
This is not true. This has been the case for the past few years, yes. But mostly due to happenstance. Everytime I bring up tying base version to GHC in the way you propose in the #ghc channel, I've been informed this is not desirable because base (in principle) should evolve separately from GHC and this has happened in the past. |
But |
|
There is at least one guiding policy for breaking changes, which is the three release policy: https://gitlab.haskell.org/haskell/prime/-/wikis/libraries/3-release-policy "Changes to basic libraries are planned and implemented so that, at any time, it is possible to write code that works with the latest three releases of GHC and base, without resorting to CPP, and without causing warnings even when compiled with -Wall. Such code may not necessarily be idiomatic, though." This is effectively what's been requested in that breaking changes include a migration strategy. The three release policy has been a key part of all CLC decisions since its formation (just about), and it may be worth placing it directly in this repo, or highlighting it more explicitly. |
|
Context setting: I am not a member of the CLC. Though I am a member of the Haskell Foundation board, I write purely in my personal (and professional) capacity as an interested Haskeller. I continue to support the proposal suggesting that broad guidelines be articulated for the evolution of base. I think such guidelines serve many goals:
However, I do not think that such guidelines will remove the (in my opinion) essential human element: the CLC is composed of individuals with strengths and weaknesses, and these individuals will vote according to inscrutable internal processes (as we all do in all of our decision making). This means that @chshersh's very valid concerns that the results of a proposal may depend on, say, the timing of the proposal will not be negated by the introduction of guidelines. The guidelines will help, to be sure -- and I support writing them! -- but they will not "fix" this problem. Instead, I think the best we can do is to make the selection process of these individuals as transparent as possible. The repo currently has no guidelines about how the CLC membership is selected and how it is refreshed over time. I think this is an oversight that should be corrected. About Haskell Foundation affiliation: I see that the CLC is affiliated (see https://haskell.foundation/affiliates/), but I do not see that the CLC has met all the requirements for affiliation, as listed at https://haskell.foundation/affiliates/about/. In particular, I see no code of conduct or information about refreshing CLC membership. (Maybe there are other gaps, too -- I have not checked closely.) It looks like there is some work to do here. Timing: The CLC reboot is still very fresh. The housekeeping details I'm advocating for here (guidelines for evolution, rules for membership, code of conduct, etc.) are important, but perhaps not as exciting as actually improving Thanks, all, for a great conversation here -- it's wonderful to have a place to discuss these important issues, and I am thus very grateful for the work in rebooting the CLC. |
As long as the committee's membership is not too volatile, I see no problem in that. A CLC decision once taken can (and should be able to) be overturned later. There should be no ban on reconsidering a proposal, say, 3 years later, as the basis -- the state of base -- has changed by then. There might be merit to the proposal then, even if at its first proposition it was found not to be convincing. Of course, reopening a proposal, say, 6 months after it was rejected should likely be not considered, as it's unlikely that the fundamental basis has changed enough to merit a different outcome. |
|
@chshersh As a fellow maintainer I do empathise with your perspective. I suspect that, as a maintainer of an alternative prelude, you are particularly exposed to breaking changes in While I do feel that breaking changes need to be done carefully, I'm already quite happy that proposals involving them require an impact analysis:
I remember one proposal a few years ago that was approved without an impact analysis, which suggested to remove the Apart from that, my experience with breaking changes in I'm very optimistic that the new CLC and the new proposal process will lead to these necessary improvements, and that the CLC and the Haskell ecosystem can manage the breaking changes involved. |
The thing is however that CLC is responsible for minority of breakages. Almost every release of GHC intricately breaks type checking. Almost every version of GHC changes I'm happy to embrace a principle saying that if there are no breaking changes in GHC, |
|
As the risk of being a broken record, the essential step is to decouple base from GHC. The total amount of breakage is not an issue, it's that it comes lockstep that makes it's more annoying than it need be, and prevents us from making |
|
What about |
@Ericson2314 Are there any concrete proposals on how to decouple base from GHC releases ?
This ^ seems to be the only practical proposal so far. Beyond that and documenting thoroughly all breaking changes, I think finding overarching principles for the evolution of base is mostly a distraction. The research half of Haskell will always push for research-oriented changes to GHC and base will follow in a piecemeal fashion. |
|
(Sorry for piecemeal answers, I'm doing too many things at once.)
That's not quite so, there are escape hatches in place. As outlined in |
|
@ocramz Yes. Uou can do it today with enough stomach for CPP --- c.f. how glibc supports many different syscall ABIs but hopefully not that bad. I think it could well be worth it to start with that for anything that doesn't cleanly separate into a ghc-specific package for GHC to depend on. See what @Kleidukos wrote in https://gitlab.haskell.org/ghc/ghc/-/issues/20647 for what such a split might look like. Longer term, we would want to use backpack, and perhaps my idea in https://gitlab.haskell.org/ghc/ghc/-/wikis/Rehabilitating-Orphans-with-Order-theory, to make this less annoying. |
|
I propose: Reasons for breaking changes to
|
|
@hasufell: Do I understand your criteria right, that AMP,FTP,SMP which I guess all are Trying out alternative API approaches, and therefore won't be possible to do? EDIT: I think that some voices are still not settled with these changes and are still frustrated. I'd like to see changes of AMP scale to still be possible in the future (not easy, just possible). |
|
AMP would probably fall under 2. correctness improvements, where I explicitly include math/CT correctness topics. |
|
I can only give half-baked thoughts currently (job search) - though the big guns of the Haskell are now here and my opinion will carry less (a good thing)! I think something that is sorely missed in this discussion is an acknowledgment that the current severe uproar is really an indication that Haskell looks like a nightmare in production. I agree with many comments that "Eq of No Neq is not a big change". However, the overarching issue is the one that @chshersh raised: a prospective business faces the very serious and severe problem that they cannot predict whether their code will be easily-updatable to modern GHC versions. The issue with Eq of no Neq is not that the change itself causes churn or no churn - it's that no one can say how much churn maintainers and businesspeople will face in say the next year. It is seriously completely unpredictable, and subject to the whims of 6 people, even if those people are awesome! While there might be an answer, it is not in writing in a public location, and that simply won't cut it when millions of dollars are on the line. I for one want to use Haskell at my day job, and while it is frustrating, the bay and many other cultures are safety-first, because product stability and predictable productivity is the name of the game. I have personally faced the "use Haskell at work" conversation, with an experienced Haskeller and self-exiled contributor who was pretty much terrified of using Haskell in production. The reality of the matter, whether you think it's reasonable or not, is that many Haskellers and non-Haskellers alike have no confidence that the decision making process is global or reasonable. While that is a statement of fear, economic recessions have occurred over similar amounts of uncertainty... I personally would advocate for:
|
|
The barrier for changes needs to improve a lot more also. I believe a migration "strategy" won't cut it if we want people to have confidence in Haskell. The issue is not whether or not there is a strategy, but whether "I will have to spend my weekend updating my 50 libraries" which is quite an awful user experience. If you advocate for a change to In other words - there is a very interesting technical PL problem here of how to avoid having downstream users pay the cost of trivial (non-bugfixing) migrations, such as A good way, imo to think about this is thinking of
I love Haskell, and will stay because I love it, but if I were on the fence or outside of Haskell - I would delete the Haskell app extremely quickly if I felt that way! |
|
For the purposes of looking at Simon's comment with "scientific control" I think it would also be beneficial to gather evidence of the cost of upgrading in other ecosystems. How many person years are required to upgrade Java, Go, C++, Python, for example? |
|
Upgrading the C++ toolchain probably takes a lot more effort overall, but there's a lot more C++ code of course. I don't have any actual data I can share unfortunately. I realised I should explain a little more why release notes and warnings don't end up being as useful as you might think. The larger the codebase, the more you want to centralise the job of maintaining and upgrading the toolchain and libraries. So it'll be one person or a small number of people doing most of the work. Not only are there a lot of release notes and announcements to read during an upgrade, but since the person doing the upgrade isn't the person who wrote the application code, they are likely to miss important things or not realise the implications of changes mentioned in the release notes anyway. Therefore instead of trying to proactively fix things, we rely heavily on automation: turn on The changes that worry me the most are things like this: https://www.haskell.org/ghc/blog/20210607-the-keepAlive-story.html The compiler, warnings, and HLint are not going to catch that. If we're lucky the benchmarks will catch the performance regression before it gets into production, but even if the benchmarks catch it, tracking down the cause of the regression will be an adventure for somebody. The only way around that is for someone to be reading the release notes carefully and proactively acting on it - fortunately I know about this one so when we do the upgrade to 9.0+ I'm going to have to go around and change all the |
|
Yes it's the semantic changes that scare me - and I do read the release
notes. These are the ones that need careful consideration and planning.
They're also often the ones - like the noted one - we can't really avoid.
We should focus our planning and solutions more on those.
…On Fri, Nov 19, 2021 at 12:14 PM Simon Marlow ***@***.***> wrote:
Upgrading the C++ toolchain probably takes a lot more effort overall, but
there's a lot more C++ code of course. I don't have any actual data I can
share unfortunately.
I realised I should explain a little more why release notes and warnings
don't end up being as useful as you might think. The larger the codebase,
the more you want to centralise the job of maintaining and upgrading the
toolchain and libraries. So it'll be one person or a small number of people
doing most of the work.
Not only are there a lot of release notes and announcements to read during
an upgrade, but since the person doing the upgrade isn't the person who
wrote the application code, they are likely to miss important things or not
realise the implications of changes mentioned in the release notes anyway.
Therefore instead of trying to proactively fix things, we rely heavily on
automation: turn on -Wall -Werror and use HLint extensively, fix compile
errors and hope that batteries of tests and benchmarks will catch anything
else that gets through. Project owners would normally get a chance to check
that an upgrade is OK, but typically they'll just look at the
test/benchmark results. If your tests and benchmarks aren't catching
problems, then they're not good enough!
The changes that worry me the most are things like this:
https://www.haskell.org/ghc/blog/20210607-the-keepAlive-story.html
The compiler, warnings, and HLint are not going to catch that. If we're
lucky the benchmarks will catch the performance regression before it gets
into production, but even if the benchmarks catch it, tracking down the
cause of the regression will be an adventure for somebody. The only way
around that is for someone to be reading the release notes carefully and
proactively acting on it - fortunately I know about this one so when we do
the upgrade to 9.0+ I'm going to have to go around and change all the
withForeignPtrs to unsafeWithForeignPtrs. Here's hoping I remember :)
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
<#12 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AABZLSYUAWLBFLYFT23YY53UM2AYFANCNFSM5HCTHEQQ>
.
Triage notifications on the go with GitHub Mobile for iOS
<https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675>
or Android
<https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>.
|
|
I would recommend @simonmar's team and others take advantage of things like At Obsidian, we were in a bit a special position in that GHCJS is usually the limiting factor preventing upgrades, blocking us after the ecosystem has by-and-large resolved most regular platform-agnostic issues. But we try to take an analogous separating our Nixpkgs and GHC upgrades so we are never upgrading both at the same time. We sometimes get stuck on weirder errors than intended breakage, and this "tick tock" alternation of what we are upgrading also helps troubleshooting, and the "upgrade experiment" is better controlled. In particular, the worst part about upgrades is not to the total time investment, but that the work is so nonparallel: when fixing failures, once typically only gets a few failures at a time. With Similarly, we should:
Here's the most important thing to remember: the closer large organizations are to the latest release, the better incentivized they are to help out with upstream FOSS development, as any contributions they make will trickle back to them and benefit them sooner. We want to get that upgrade latency to an absolute minimum, independent of the throughput of breaking changes. |
|
I've worked at three Haskell startups. I more or less agree with Simon's overall sentiment, and the observation that warnings don't save much time (the exception being when they can contain more information than an error would); certainly we won't notice warnings in dependencies. But the particulars sound very different from my experience at much smaller companies, even though e.g. we've been running a fork of ghc with patches. At hasura we have 400-500 modules primarily in a single mono repo. We're in the process of upgrading to 9.2 from 8.10; upgrading our own code is probably a day's work, while fixing dependencies and (in particular) getting those changes upstreamed, temporarily vendoring and tracking PRs, etc... is a soul-crushing and thankless (like, being actually scolded) slog. When we're done we'll need to have a discussion about whether our upgrade will be blocked on hls, and then maybe try to help there (9.0 is still not quite fully supported iirc) I think in general the breaking changes to ghc and core libs are really thoughtful, well-considered, and reasonable. But I think the principle I'd propose is: if you break it, fix hackage. If that's too difficult to do by hand, then write a script (if that's too difficult write a tool). If maintainers don't want to merge a change to support an unreleased ghc version, figure out how to automate staging PRs from head.hackage I don't think that's too much to aspire to (at least for "small" breaking changes, and for the popular/active subset of hackage) EDIT: actually one benefit of changes that come with a warning the version prior is that it does allow you to open a PR ahead of time without maintainers yelling at you. But again we're only going to start fixing other people's libraries when we go to upgrade and find they break |
That's ... a bit troubling. If you don't read release notes until you notice something "doesn't work", that means you could miss important but subtle changes in behavior. Ouch! |
Personally I like to address future issues on an ongoing basis, as they come up, rather than big-bang. So warnings are helpful. And build and maintain compatibility with future releases to which I'm not necessarily immediately ready to switch. Approaches vary... |
|
@vdukhovni, for reference and context, what kind of projects and code bases do you work on? |
|
It usually takes us 6 months to upgrade to a newer GHC version in Standard Chartered. We start building our codebase from the first released version of each new GHC (e.g. We are using GHC 8.10.7 at the moment. We've started the process of upgrading to GHC 9.0 in Feb 2021, and now we're migrating to 9.2.1 instead. However, we're waiting for the dependencies to catch up before finishing the migration, so it'll probably take about a year in total to move from 8.10 to 9.2 for us. This migration work is done by a single person mostly and they work not only on this 🙂 It's usually "make some progress -> hit the problem -> work on other stuff while waiting for the solution -> repeat". A few details about our project (in a team that uses GHC/Haskell and not Mu/Haskell):
So, usually, the main problems that delay our GHC upgrades are:
I'm all hands for contributing patches upstream ⬆️ However, the bank doesn't allow us to contribute to OSS projects during work time and patching almost 500 dependencies by ourselves each time is not a sustainable option. Hence the waiting. But if GHC and other libraries had fewer breaking changes, this could significantly speed up our upgrades and the process of testing GHC versions earlier. |
I understand that is a business/management decision, also likely at a much higher level than your unit, but tbf., it sounds like a particularly uneconomic one, given the lack of upstreaming makes you accumulate more and more patches to maintain yourself down the line. |
|
@szabi Yeah, I agree that it's a suboptimal position to be in. There were several attempts to change the situation. But they went nowhere and this is not something I can change, unfortunately 😞 |
|
I'm hearing here that several companies are doing real work to patch dependencies in the process of upgrading. This work sounds duplicated across the companies. Would it make sense to somehow work together to avoid this duplication? One model for this is to have, say, a pool of money that gets spilled out at every GHC release to get some pre-defined portion of our ecosystem up-to-date. Would that be of general interest? Or maybe the overhead of coordinating that drowns out the improvement to efficiency -- not sure. |
|
In the OCaml community, we had a problem of "wait for dependencies to upgrade", and we have improved the issue a lot thanks to work that happens during the "new compiler release process" (from branching to release):
On most releases we are in a good state where most of the ecosystem is compatible with the new release essentially on the day that the release is officially shipped -- instead of individual packages having to wait for weeks or months as we did in the past. |
[niche OSS project with limited resources] The latter, after the original code author / Haskell guru moved on. We use GHC 5+ years old for a Xen toolstack with ongoing minor enhancements. No GHC patches. Source: https://github.com/OpenXT/manager/tree/master/xenmgr As the cost of GHC uprev remained stable at infinity for a small team without a Haskell expert, alternatives like moving to Rust have slowly become more practical. |
|
GHC haskell has it too:
Kind of. See https://gitlab.haskell.org/ghc/head.hackage/-/graphs/master However as the How that work is funded: I don't know. |
|
We ask for mergeable patches that are actually submitted to the various upstream projects. I believe that most managers are happy to merge the patches proposed by @kit-ty-kate. (I forgot to mention it, but this work (in particular enabling people to test proposed patches/changes or debug issues) requires a specific opam-repository overlay, just like head.stackage.) |
|
I would add that for library authors and maintainers, compat warnings are a very good thing. In particular, if there's a full release cycle between warning and breakage, there are high odds that either you or another library consumer will notice the warnings and attend to them early, rather than waiting until a confirmed breakage. |
@goldfirere I don't think duplication is a big issue; once a ghc is released we can open PRs. And the impression I get from this thread and my own experience is everyone waits for someone else to fix dependencies :) The duplication is probably between head.hackage and the patches that actually make it to upstream. I'm very keen to learn more about the system OCaml has, that @gasche describes. It seems to address the two pain points:
And I like the idea of breaking changes being decided in coordination with a team that can be consulted and say "this would break things in a way that would require X, Y and Z" or "it would be easy for us to migrate the ecosystem for this change". It's a mechanism of accountability. And at the end of the day, if libraries (and as a consequence, applications) are lagging way behind ghc or core libs then presumably ghc itself suffers, unable to judge whether changes actually benefit users (wrt performance, compile time, that kind of thing). |
I bet a lot of people just bump the stackage version and run CI without reading all the release notes for all the packages that have changed in the upgrade. That's all we're doing here. I'll also say that if your CI isn't detecting changes in behavior then you already have problems waiting to happen. One thing I've learned doing this over the years is to rely on humans as little as possible! |
For a start (and probably the most important), OCaml has been very conservative in avoiding breaking changes, unless deemed concerning an internal detail. One unfortunate part of the ecosystem that suffers from breakage are the PPX preprocessors, since they rely on internal representations of the compiler. For this, they have chosen an “upgrade the world” approach (see https://github.com/ocaml-ppx/ppxlib/wiki/The-State-of-the-PPX-Transition, https://discuss.ocaml.org/t/ppxlib-0-22-an-update-on-the-state-of-ppx/7296/9). |
|
The last message here is six months old. The topic of stability has been picked up by Haskell Stability Work Group. The raging storm around I'd like to close this issue soon, it accumulated too many different threads to be actionable. If there are any outstanding proposals left, which you'd like to pursue, feel free to open new discussions. |
|
I support closing this issue also. Thanks @Bodigrim |
and others
Let's decide on some principles that guide the design of
baseand other CLC-maintained libraries, and let's write them down in a central location.As it stands, well-thought-out and considered proposals such as Joachim Breitner's no
/=inEqproposal get accidentally hijacked by high-level discussion behind what is sufficient breakage to make a proposal untenable.For example, within that thread alone, the proposal maker and myself both disagree on the cost of such a change. Joachim argues that the change will cause minimal issues, whereas I argue that while the breakage is relatively small, the change will cause confusion and frustration, especially for those less experienced Haskellers and time-limited library maintainers.
Similar discussion occurred around what should a typeclass' methods contain in #3. In particular I see discussion around whether a typeclass should contain methods. For example, many of the typeclasses contain
defaultfunctions that are included for the sake of runtime efficiency. However, that principle seems nebulous and not clearly defined. For example,elemandmaximum, which require anEqandOrdconstraint respectively, are user-definable in theFoldableclass. However,^^and^, which I would imagine can have their own efficient implementations, are not defined on any of their required classes, and therefore cannot be more-speedily defined...Just to reiterate, this is not specific to any singular function or class (perhaps
^^and^are actually consistent!). The point of this proposal is that we don't have a document or central discussion, as far as I know, that outlines the parameters that make a proposal worth or not worth accepting. If we don't have a central discussion of such simple principles, work is very hard to get done because everyone has a slightly different perspective on what the right balance of breakage-for-improvement vs backwards-compatibility is.The text was updated successfully, but these errors were encountered: