Language Server 2.0

[wdanilo]

Language Server 1.0

The current version of the language server (LS) does not have an optimal performance. Back in the days it was built with the different requirements in mind, and can be implemented in a more optimal way.

Language Server 2.0

A new language server can be created with different requirements based on the experience of building the first version.

• Focus on performance. Performance should be the main requirement when designing a new LS.
• Switch to binary protocol. The text protocol is a legacy, and nothing prevents us to switching to more efficient binary format.
• Drop multi-user support. LS 1.0 can support multiple users but we've never used it this way. Removing this requirement should drastically simplify the logic. Experiments with y.js sharing has been made
• Switch to in-memory protocol between the LS and the Engine. In LS 1.0 we have to serialize-deserialize message every time it crosses the LS/Engine boundary. In-memory protocol will eliminate the overhead of serialization.

Use Virtual Threads

JDK21 is the first LTS version to provide support for virtual threads. Virtual threads are light to create and light to block. One can write sequential code and invoke it as effectively as reactive code. We should use one of the frameworks that provides support for virtual threads and simplify the current actor based system. Consider:

• Helidon Níma: https://www.youtube.com/watch?v=aYY04D4_OQA
• Micronaut: Virtual thread support micronaut-projects/micronaut-core#8180
• Quarkus: https://quarkus.io/guides/virtual-threads

Micronaut & Quarkus rely a lot on dependency injection. It may be impossible to use them without all the DI magic. Níma provides imperative API to build and handle the routes.

Distribute Language Server with the IDE

There is an effort to run language server as hosted Java. It'd be great if these two efforts emerged together.

Write Language Server in JavaScript or Enso

An alternative to above mentioned Java frameworks is to write language server in JavaScript and execute it by GraalVM. We cannot easily support all node.js modules, but we can support latest, greatest JavaScript specification and emulate essential node.js APIs (for example for handling websockets via WebSocketServer). Enso Engine is already running JavaScript (to process JSON), so running JavaScript for language server isn't going to add much overhead.

Alternatively the language server could be written in Enso itself. Using the same emulation of websockets we could handle the incoming and outgoing messages in plan Enso code. Using JavaScript or Enso comes with familiarity of the languages and also support the Distribute Language Server with the IDE goal - having it in JS or Enso will require less changes in IDE build system than writing the server in Java.

[hubertp]

Another thing to consider is that we cannot upgrade to latest Akka nor should we base any further development on it because of its restrictive license change - https://www.lightbend.com/akka/license.

Lightbend

Akka BSL 1.1 License | @lightbend

[wdanilo]

Another thing to consider is that we cannot upgrade to latest Akka nor should we base any further development on it because of its restrictive license change - https://www.lightbend.com/akka/license.

Lightbend**Akka BSL 1.1 License | @lightbend**

Oh wow. I did not dig deep enough to understand the new license, but are you sure that such projects as our would not be able to use it? Looks like a killing license for Akka.

Lightbend

Akka BSL 1.1 License | @lightbend

[hubertp]

@wdanilo Yeah, fairly sure. Here is more on the topic https://www.infoq.com/news/2022/09/akka-no-longer-open-source/. There is a whole discussion on forums about it when it happened but the general sentiment is toward moving away from it.

We are OK for now, but in the longer term it won't make sense to base more infrastructure on it.
@JaroslavTulach was suggesting Micronaut which would be inline with reducing Scala presence in the codebase.

InfoQ

Lightbend Changes Akka License and Is No Longer Open Source

Akka, a toolkit for writing concurrent distributed applications based on the actor model, was created thirteen years ago by Jonas Bonér, founder and CEO at Lightbend. The company has recently announced a new Akka license model that has changed from the open-source Apache 2.0 to the source-available Business Source License (BSL) 1.1.

[JaroslavTulach]

@wdanilo wrote:
Oh wow. I did not dig deep enough to understand the new license, but are you sure that such projects as our would not be able to use it?

A quote from https://mariadb.com/bsl-faq-adopting/

Q: Can I use BSL software with an Open Source project?

A: No.

@wdanilo wrote:
Looks like a killing license for Akka.

Right. Nobody will ever write any new open source library against Akka, if it is licensed under BSL. Using Akka in an open source project will also become quite painful. What a path into oblivion!

MariaDB

Adopting and Developing BSL Software | MariaDB

[wdanilo]

This is crazy!

[JaroslavTulach]

Idea: Use y.js AST for communition between the engine and IDEs.