Long-term compatibility plans for Scala 3 #1387
Conversation
|
Regarding the roadmap picture: What does the minor release in the black track between 3.3 and 3.4 mean? If it represents 3.3.1 etc. then it seems to be a duplicate of the first minor release on the green track of 3.3... So it seems to that there should be no black minor releases in the picture as all LTS releases should be on the green LTS track. Or am I missing something here? @Kordyjan |
|
@bjornregnell Good catch! The 3.3.1 patch release (3.3 & 3.4 are minors) should only be present on the LTS track. The picture is fixed now. |
|
|
||
| ### Owners of commercial projects | ||
|
|
||
| You should stick to the LTS version. It will give you the best tooling support and a stable compiler. If you are migrating a bigger project from Scala 2, you should migrate it straight to the newest LTS. |
There was a problem hiding this comment.
Could you explain more what the risks for bigger or commercial projects would be if they choose not to stick with an LTS release?
There was a problem hiding this comment.
IMO this is worded a bit too strongly. I think there are two valid choices for commercial projects:
- Stick to the LTS version -> update less often (from LTS to LTS), with potentially more changes to deal with at once, or
- Be up-to-date to the latest minor version -> update more often, but with smaller deltas
In both cases, you will get the best tooling support and a stable compiler.
The catch is that you take the option of the latest minor version, you will have to update more often in order to keep having the best tooling support, performance improvements and bug fixes (including potential security fixes).
If you are prepared to keep up in that sense, it is perfectly fine to use the latest version in a commercial project.
There was a problem hiding this comment.
The catch is that you take the option of the latest minor version, you will have to update more often in order to keep having the best tooling support, performance improvements and bug fixes (including potential security fixes).
I suppose this was the only option before LTS releases were introduced?
There was a problem hiding this comment.
If you are prepared to keep up in that sense, it is perfectly fine to use the latest version in a commercial project.
Yes that is indeed important information so that commercial projects, are not "scared away" from being up-to-date if they care about frequent updates. It is good to know that following the frequent release train also provides a stable compiler. (As I understand it, it's the RC:s that are less stable, and nightly even less stable, in terms of how much real-world testing has been made so far.)
There was a problem hiding this comment.
I suppose this was the only option before LTS releases were introduced?
Yes, indeed.
As I understand it, it's the RC:s that are less stable, and nightly even less stable, in terms of how much real-world testing has been made so far.
Also yes, indeed. ;)
There was a problem hiding this comment.
I don't think we are ready to commit to the guarantee that LTS and the latest Next would have the same quality of support from the tooling. The features that are present in the LTS branch will of course work, but anything added after the last LTS might have only partial support; similar to the way how experimental features are handled right now.
Let's take language.experimental.fewerBraces as an example. It is not supported in any way in scalameta or any derivate tooling. I don't think this will change soon.
Also, we need to be aware that there is tooling for Scala that is not directly connected to Scala Center or VirtusLab. The Scala plugin for IntelliJ is the most important example here. We have no control over it and we cannot guarantee that it will always be up to date with the latest Scala Next. On the other hand, new LTS versions will be rare (max one every two years) and their features will be present in the Next versions, way before the release. It will make work for plugin maintainers much easier and should result in an overall better quality of the tooling.
There was a problem hiding this comment.
- Be up-to-date to the latest minor version -> update more often, but with smaller deltas
This is impossible for the large majority of the codebases in the industry. Nevertheless, it is true. I will add a sentence about it being possible if the code maintainers are willing to commit to the constant updates.
There was a problem hiding this comment.
You may also choose to be up to date with the latest minor version. That will allow you to use new features as soon as they are added to the language. However, you will need to be ready for frequent compiler updates. Otherwise, your project may end up with an unsupported version of the language.was added.
There was a problem hiding this comment.
I guess you want to update this blog as well, and remove some stuff:
https://www.scala-lang.org/blog/2022/04/12/scala-3.1.2-released.html
if scalaOutputVersion is cancelled. And align sbt with docs etc...
Otherwise there is a risk that people get hold of outdated info when googling and miss the updated policy on LTS etc.
There was a problem hiding this comment.
-scala-output-version was supported only in one pre-release version of the sbt, and the support is being removed right now. Regarding the previous blog post, it may indeed be a good idea to add an update there.
| @@ -9,6 +9,8 @@ We are happy to announce the release of Scala 3.1.2. This new release brings a f | |||
|
|
|||
| ## Configurable Scala output version | |||
|
|
|||
| **June 2022 Update:** configurable Scala output version turned out to be really hard to maintain and is not solving some of problems it was intended to solve. We have decided to drop it in 3.2 and propose long term support versions instead. You can read more in [the dedicated post]({{ site.baseurl }}/blog/2022/05/30/long-term-compatibility-plans.html). | |||
There was a problem hiding this comment.
"some of the problems", missing the
or perhaps even better: "and it did not really solve the problems it was intended to solve"
|
Here comes the usual feedback that I always give on everybody's blog posts :-) I would suggest beginning the entire blog post with a short "Summary" section that says as concisely as possible what the takeaways are from the entire thing. Then go into the full version, including all the material about how the conclusions were arrived at. Doing it this way serves multiple purposes. It helps the reader decide whether they need or want to read the rest. And even for people who do end up reading the whole thing, it's useful to have the summary to check their understanding against, and to consult again when returning to the blog post later. (This is one that library maintainers are likely to return to multiple times.) Sorry I can't be in Krakow this weekend like I'd planned :-/ |
|
|
||
| You may also choose to be up to date with the latest minor version. That will allow you to use new features as soon as they are added to the language. However, you will need to be ready for frequent compiler updates. Otherwise, your project may end up with an unsupported version of the language. | ||
|
|
||
| ### Library maintainers |
There was a problem hiding this comment.
I like the strategy and these "future" best practices, but as a library maintainer, we would benefit from best practices now. The most aggressive dependency decides for all of us, and it has been chaotic downstream.
Until the first LTS (3.3?), is the recommendation that we stick to 3.1 (where most of us are now) or jump on 3.2? Can we say so here?
There was a problem hiding this comment.
Seconded. I think it would be nice to clearly state that 3.0 isn't worth targeting anymore as the ecosystem has already moved on. (IMO, enough of it has already moved that "the ecosystem has already moved on" is a true simplification.)
There was a problem hiding this comment.
Right, if the implication of this post is that when 3.2 lands, 3.1 will stop getting (security) patches and thus is EOL, seems to me that libraries may as well move on. OTOH, if 3.1 or something was blessed an "STS" (short-term-support?) to hold us over until the real LTS lands, that would at least get security patches or something, then it could make sense to pin.
There was a problem hiding this comment.
The general recommendation is similar to the one we had during the 3.0-3.1 transition. So if you are an author of some library - bump the compiler version in the next minor release. It would be a truly unique scenario when someone would be able to bump the minor version of some dependency but unable to bump the compiler version.
Use the latest compiler version if you are working on the leaf project.
There was a problem hiding this comment.
I wonder if and how I should put that recommendation in this post.
There was a problem hiding this comment.
I wonder if and how I should put that recommendation in this post.
You definitely should :) it should go in the summary section: "if you are a library author, here's what you should do today".
| While this sounds good, it can be problematic for an intertwined ecosystem. Even though we do what we can to make the transition between minor versions of the compiler as smooth as possible, there will always be projects stuck on older versions for various reasons. That means that they may not be able to update some of their dependencies if said libraries updated themselves to the newer compiler. That, in order, encourages libraries to stay on the oldest possible version of the compiler, which bars them from potentially beneficial improvements in the newer compilers. | ||
|
|
||
| Let’s examine the simple example. We have a project that for some reason, is stuck on Scala 3.1. We depend on version 1.3.5 of some library that was compiled with Scala 3.1. The library publishes version 1.4.0 with the compiler updated to 3.2. We cannot update our dependency because of output incompatibility. That also means that all patches for security issues won’t be available unless the library author decides to backport them and release them as version 1.3.6. Knowing all this, the library author would never migrate from Scala 3.1 to 3.2. | ||
|
|
There was a problem hiding this comment.
"would never" is a bold statement and "never" is a very long time...
Perhaps phrase it as "is not likely to want to" or something.
| title: Long-term compatibility plans for Scala 3 | ||
| --- | ||
|
|
||
| Scala 3 has stronger compatibility guarantees than Scala 2 ever had. That allows us to improve the language at a much faster pace than it was possible before 3.0. However, with the current scheme, when you bump the dependencies of your project, you may also have to bump the version of Scala used by the project. After an extended evaluation, we have decided to solve them by creating a new kind of Scala release - Scala LTS, that will be supported and receive updates for at least three years after the initial release. |
There was a problem hiding this comment.
"we have decided to solve them" - sorry, maybe I'm just blind, but what does "them" refer to?
There was a problem hiding this comment.
😅 Refactoring leftover
|
If people have feedback/concerns not about the clarity of the blog post (which is what the comments on this PR are for), but about the versioning/compatibility plan itself, where do they go to lodge that feedback? I think the blog post should be explicit on that point. |
I intended that the first paragraph serves this purpose. When I get back to it, I think I should add the information about drooping the Do you have something other in mind when speaking about "Summary"? |
Imagine I am a library author reading this post. What do you actually want me to do after reading? This might be in two parts: what do you want me to do right now, and what do you currently expect or propose that I will need to do once the new scheme has been fully rolled out? What is the upshot, the take-home? Do you want me (the library author) to carefully read the entire blog post and give feedback? If so, where? Do you want me to drop something from my current CI configuration (perhaps More specifically: what is the expected status of 3.0? What is the expected status of 3.1? What is the expected status of 3.2? The more concrete you can be about this, the better. Perhaps my library has already moved to requiring 3.1. Perhaps my library still supports 3.0 because I wasn't clear on whether it was okay to drop. What do I do now, in either case? Imagine I haven't been following all this discussion and debate and I just want to know what you, the maintainers of Scala 3, want from me, a library author. Tell me what to do. And perhaps tell me very briefly why you want me to do it. (And then the rest of the blog post can go into absolutely as much detail as you want about all this and provide as much history and as much justification as you feel is appropriate.) |
| title: Long-term compatibility plans for Scala 3 | ||
| --- | ||
|
|
||
| Scala 3 has stronger compatibility guarantees than Scala 2 ever had. That allows us to improve the language at a much faster pace than it was possible before 3.0. However, with the current scheme, when you bump the dependencies of your project, you may also have to bump the version of Scala used by the project. After an extended evaluation, we have decided to solve them by creating a new kind of Scala release - Scala LTS, that will be supported and receive updates for at least three years after the initial release. |
There was a problem hiding this comment.
Scala 3 has stronger compatibility guarantees than Scala 2 ever had. That allows us to improve the language at a much faster pace than it was possible before 3.0.
This raises questions and doesn't give any answers. Is that intended to make people read the article? IMO it would be better to summarize and give the information upfront.
|
|
||
| More importantly, during work on 3.2, we realized that maintaining this flag may be more challenging than anticipated. With time, it will be increasingly harder to be sure that our handling of it is correct. **Thus we have decided to remove the possibility of configuring the output version altogether in Scala 3.2.** | ||
|
|
||
| ## Our solution: Scala LTS & Scala Next |
There was a problem hiding this comment.
I feel like this section only tells me what the solution is without saying how it solves the problem.
IIUC, the goal is to encourage library maintainers to stick to LTS versions as much as they can. In return, we promise to maintain these versions so that they don't need to use some old, non-supported Scala release. Does that sound about right?
There was a problem hiding this comment.
I've added We think this will allow projects to retain a more conservative approach to updating the compiler versions and still receive the bug fixes and some improvements. Do you think it needs to be elaborated further?
|
|
||
| ### Owners of commercial projects | ||
|
|
||
| We suggest you stick to the LTS version. It will give you the best tooling support and a stable compiler. If you are migrating a bigger project from Scala 2, you should migrate it straight to the newest LTS. |
There was a problem hiding this comment.
My humble 2 cents:
The OpenJDK authors recommend something along the lines of
- using the newest Java version when the project is in the state of active development
- once the project moves to the maintenance phase of its life, update up to the at-that-time newest LTS version and stay there
https://news.ycombinator.com/item?id=28821316
Maybe we could recommend something similar here about Scala version?
|
I hope I didn't stall this PR with my well-meaning advice :-) |
as per scala/scala-lang#1387, there's no longer any need to keep supporting Scala 3.0.x. it's fine to expect people to upgrade to 3.1 or higher
as per scala/scala-lang#1387, there's no longer any need to keep supporting Scala 3.0.x. it's fine to expect people to upgrade to 3.1 or higher
|
So, unfortunately, I wasn't able to find time to work on the announcement (or many other things, sadly) lately. But I'm back at it, and I believe we should be able to have it in a satisfying state soon. I've added a TL;DR-ish section at the top. I don't know if this is the right approach, so I'm open to suggestions. Also, this post was initially meant to be only an announcement about dropping |
Co-authored-by: Jamie Thompson <[email protected]>
|
|
||
| ## Key takeaways | ||
|
|
||
| - Scala 3 compiler that you are using to compile your code should be treated the same way as any library dependency in your project. You shouldn't be afraid of updating it. Bumping the patch version (e.g. 3.1.2 -> 3.1.3) is a no-brainer. Bumping the minor version (e.g. 3.1.3 -> 3.2.0) requires the same considerations as changing the minor version of any other dependency but is generally preferable. |
There was a problem hiding this comment.
| - Scala 3 compiler that you are using to compile your code should be treated the same way as any library dependency in your project. You shouldn't be afraid of updating it. Bumping the patch version (e.g. 3.1.2 -> 3.1.3) is a no-brainer. Bumping the minor version (e.g. 3.1.3 -> 3.2.0) requires the same considerations as changing the minor version of any other dependency but is generally preferable. | |
| - The Scala 3 compiler that you are using to compile your code should be treated the same way as any library dependency in your project. You shouldn't be afraid of updating it. Bumping the patch version (e.g. 3.1.2 -> 3.1.3) is a no-brainer. Bumping the minor version (e.g. 3.1.3 -> 3.2.0) requires the same considerations as changing the minor version of any other dependency but is generally preferable. |
There was a problem hiding this comment.
preferable to what? (preferable is a comparative that requires something to be compared to)
Co-authored-by: Sébastien Doeraene <[email protected]>
The post with communication about our decision to drop
-scala-output-versionand introduce the new Scala LTS / Scala Next scheme.