Metals (scala language server) integration with jupyterlab-lsp

[skakker]

Description

The work done for jupyterlab-lsp is great! I want to support IntelliSense for Jupyter Labs and found this to be a good fit. Integrating with Python wasn't an issue, but I am unable to integrate Metals with jupyterlab-lsp for the Spark kernel.

Found this similar issue: https://github.com/krassowski/jupyterlab-lsp/issues/17. But in this issue, the language server is set byjsonrpc-ws-proxy/dist/server.js, but it seems to be deprecated now. I couldn't find any such file.

Going by the metals documentation, I have installed the metals package by:
curl -Lo coursier https://git.io/coursier-cli && chmod +x coursier
./coursier bootstrap org.scalameta:metals_2.12:0.7.0 --force-fetch -o metals -f

Help appreciated in going forward from here.

Reproduce

I am using the spark kernel. Installation instructions for Spark kernel:

pip install sparkmagic
jupyter-kernelspec install sparkmagic/kernels/sparkkernel

After installation, running the Jupyterlab in watch mode.

It doesn't identify the language of the server and shows no available servers.

For Almond Scala kernel though, it was able to identify the language as "Scala", and said that the server is missing:

Expected behavior

Metals integrated with the Spark kernel from Spark magic.

Context

The env was created with the following specs: conda create -c conda-forge -n lsp 'python >=3.7,<3.8' 'jupyterlab=2.2.0' 'nodejs>10'

• Operating System: MacOS
• Browser: chrome
• JupyterLab version: 2.2.0
• @krassowski/jupyterlab-lsp version: On top of v2.0.7, latest commit: 80459bf
• Language Server: metals
• Language Server installed with: coursier

[krassowski]

Have a look here:

• https://jupyterlab-lsp.readthedocs.io/en/latest/Configuring.html
• https://github.com/krassowski/jupyterlab-lsp/issues/384
• https://github.com/krassowski/jupyterlab-lsp/issues/313#issuecomment-672816651

Sorry the docs are not in the great state - contributions welcome!

[skakker]

Thanks for sharing the links. I understand that we have to mention the command to start the server in the python specs file.

After installing the metals server, it starts by invoking the binary. (/Users/skakker/projects/jupyterlab-lsp/metals)

I have created this spec under the path: /py_src/jupyter_lsp/specs/scala_languageserver.py :

from .utils import ShellSpec
class ScalaLanguageServer(ShellSpec):
key = "scala-languageserver"
cmd = "/Users/skakker/projects/jupyterlab-lsp/metals"
languages = ["scala", "spark"]

In the file __init__.py, imported: from .scala_langaugeserver import ScalaLanguageServer and added these lines:

scala = ScalaLanguageServer()
spark = ScalaLanguageServer()

Still when I re-installed the jupyterlab-lsp and restarted jupyter lab, the metals server doesn't start (no additional logs under ~/Library/Caches/org.scalameta.metals/global.log) and the screenshot still says "Missing: Scala".

Am I missing something?

[bollwyvl]

Sorry you're having to muck about outside of your desired language(s)! The language-agnostic way is, like the first example, to put something into jupyter_notebook_config.json, as in the very first example in the docs. So yours might be:

{
  "LanguageServerManager": {
    "language_servers": {
      "metals": {
        "version": 2,
        "argv": ["/Users/skakker/projects/jupyterlab-lsp/metals"],
        "languages": ["scala", "spark"],
        "mime_types": ["application/whatever", "text/this-is"]
      }
    }
  }
}

So, doing the python way is kind of "the hard way," and mainly there as a convenience for servers that someone cared enough to add with first-class CI support on this repo. We inject a lot of extra metadata, and sometimes need to do funny script things to get the language server environment right (r, latex).

Note also, if you want notebooks (or much of anything) to work, they need their metadata to line up quite specifically with what the kernel reports, and what CodeMirror reports, with mime_types. You can figure out what these values should be by looking in the notebook metadata created with the kernel in question.

Anyhow, if you were to want to PR metals support, and help fix it in the future:

• to get an entry_point based thing to work, you'd need to add it to setup.cfg as well, and then re-install the package (python -m pip install -e)
• add a robust, multi-platform way to install it
  • conda is preferred, for CI purposes, despite its warts
• add unit tests
• add acceptance tests of features you think should keep working

It's not an easy road, particularly, and we'd love to make it easier, but there's just a lot going on.

[skakker]

Hi @bollwyvl
Thanks for your suggestion! I was able to run metals with the Scala kernel.

However, I am interested in running with the Spark kernel from Spark magic. On that kernel, it's not able to identify the language as scala.

How can this information be passed?

Thanks! :)

[bollwyvl]

Make an Untitled.ipynb with the kernel in question and paste it here... We have to match the languages, mimetypes, etc.

[skakker]

This is the metadata for the notebook created:

""metadata": {

"kernelspec": {
"display_name": "Spark",
"language": "",
"name": "sparkkernel"
},
"language_info": {
"codemirror_mode": "text/x-scala",
"mimetype": "text/x-scala",
"name": "scala",
"pygments_lexer": "scala"
}
},"

Language seems to be empty and mimetype is "text/x-scala".

This is my jupyter_notebook_config.json

"{
"LanguageServerManager": {
"language_servers": {
"metals": {
"version": 2,
"argv": ["/Users/skakker/projects/jupyterlab-lsp/metals"],
"languages": ["scala", "spark", ""],
"mime_types": ["text/x-scala"]
}
}
}
}"

Still no luck!

[skakker]

I also tried editing the notebook metadata to add the language as scala.

But the language server doesn't start (probably it's not dynamic loading). On refresh, the metadata edit goes away.

[skakker]

Update: Added file_extension: .sc to the language_info and language: scala and it worked! Able to start the server.

Thank you so much for all the help. Really appreciate it :)

[bollwyvl]

Great! Glad that worked out. Even if we don't put it under test, it would still be fabulous to have an example in the docs. If you could boil your steps down to a gist/binder we can work it into the docs.

[skakker]

Sure! Would love to contribute.. will formalize the steps and get back :)

[skakker]

Created a pull request including the example: https://github.com/krassowski/jupyterlab-lsp/pull/415