Be less strict with wrong LSP types? #326
Comments
|
Thanks for the detailed write up. I'm not quite clear on the architecture. It seems like your YaLafi-LS is something like a proxy for messages from LaTeX Workshop? Or put another way: what generates the JSON message you pasted, and why is it sent to Pygls? |
|
YaLafi-LS is an LSP wrapper around YaLafi which is a spell checker for LaTeX documents. And LaTeX Workshop is the editor plugin providing LaTeX code completion et cetera in VS Code, it directly uses the VS Code API (no language server). YaLafi LS publishes diagnostics with spelling mistakes and supports code action requests to fix these spelling mistakes (all that works quite well). However, if VS Code sends a code action request, it sends all available diagnostics in the affected range (where the cursor currently is) including those of other extensions. In this particular case, it also sends faulty ones from LaTeX Workshop. This of course should not happen and hopefully gets fixed within VS Code. My intention here is to ask whether it is feasible for pygls to also support LSP messages with wrong types as long as they are within an optional parameter of the message and can safely be discarded. The idea behind that is assuming that similar bugs might occur elsewhere. A valid answer would be: “No, we require the client to send correct types.” |
|
I think this is an area of LSP I'm not so familiar with. So the YaLafi-LS VSCode JSON extension (and LSP client) is sending Code Action Requests (on behalf of VSCode) to your Pygls implementation of YaLafi-LS? That makes sense. So yes, it's an interesting area for discussion. Although it is not Pygls' responsibility, it seems like a reasonable feature for it to support, namely allowing optional sections of JSON payloads to be malformed. My first thought is that I don't know enough about all the possible consequences of that. I could imagine that it might have unforeseen and unexpected consequences in other contexts. We're not just talking about converting an error to a warning, but deleting entire sections of JSON payloads. It's definitely a reasonable idea to reflect on. But my intuition is that Pygls core shouldn't take responsibility for it. I certainly think Pygls should offer a way to intercept incoming payloads. I believe that has been discussed before although I can find those issues right now. I did find these issues talking about null values in optional fields: #145, #124 And what about intercepting those payloads in YaLafi-LS's VSCode JS extension? |
Yes, I agree up to ‘JSON’.
Indeed, I also don’t know whether that could have bad consequences. At least the server should be able to process and answer the request, because it would also do so without the optional sections. I would not try to process sections, when the whole JSON payload is malformed, only if the types are out of spec.
Correct, that comment was more saying, that if something like that would be implemented, one should note somewhere that pygls received malformed JSON payloads, maybe as a warning in the log.
These are exactly the opposite, right, where pygls sent malformed JSON and VS Code did not process the message (but also didn’t throw an error)?
For my specific issue, I will just wait, until VS Code handles them correctly and does not send faulty messages. The YaLafi-LS VS Code extension is more or less the same as the But also in general, if accepting some kind of malformed JSON messages is feasible, it should be done by pygls in my opinion. Isn’t the point of pygls, that one can write language servers in python without dealing with JSON messages directly? |
TLDR: Yes it is possible, but it is currently up to you to define what "less strict" means. A quick bit of context As of v1, >>> from lsprotocol.converters import get_converter
>>> from lsprotocol.types import Position
>>> converter = converters.get_converter()
>>> p = converter.structure({"line": 1, "character": 2}, Position)
>>> p.line
1
>>> p.character
2Each Relaxing the rules However, it is possible to override how the Here is an example hook where any malformed diagnostics in the # example.py
import json
import pathlib
from lsprotocol.types import CodeActionContext
from lsprotocol.types import Diagnostic
from lsprotocol.types import TextDocumentCodeActionRequest
from pygls.protocol import default_converter
def my_converter_factory():
converter = default_converter()
def relaxed_code_action_context_hook(obj, type_):
diagnostics = []
for diag in obj['diagnostics']:
try:
diagnostics.append(converter.structure(diag, Diagnostic))
except Exception:
pass
return CodeActionContext(diagnostics=diagnostics)
converter.register_structure_hook(CodeActionContext, relaxed_code_action_context_hook)
return converter
converter = my_converter_factory()
example_request = json.loads(pathlib.Path('example.json').read_text())
parsed_request = converter.structure(example_request, TextDocumentCodeActionRequest)
print(json.dumps(converter.unstructure(parsed_request), indent=2))example.json (Your example request, modified to have one valid, one invalid diagnostic){
"jsonrpc": "2.0",
"id": 10,
"method": "textDocument/codeAction",
"params":{
"textDocument":{ "uri": "file:///.../test.tex" },
"range":{
"start":{ "line": 7, "character": 14 },
"end":{ "line": 7, "character": 14 }
},
"context":{
"diagnostics":[
{
"range":{
"start":{
"line": null,
"character": 0
},
"end":{
"line": null,
"character": 65535
}
},
"message": "Using \\overbracket and \\underbracket from\n(unicode-math)\t`mathtools' package.\n(unicode-math)\t\n(unicode-math)\tUse \\Uoverbracket and \\Uunderbracket for\n(unicode-math)\toriginal `unicode-math' definition.\n",
"severity": 2,
"source": "LaTeX"
},
{
"range":{
"start":{
"line": 1,
"character": 0
},
"end":{
"line": 1,
"character": 65535
}
},
"message": "I'm going to overwrite the following commands\n(unicode-math)\tfrom the `mathtools' package: \n(unicode-math)\t\n(unicode-math)\t\\dblcolon, \\coloneqq, \\Coloneqq, \\eqqcolon.\n(unicode-math)\t\n(unicode-math)\t\n(unicode-math)\tNote that since I won't overwrite the other\n(unicode-math)\tcolon-like commands, using them will lead to\n(unicode-math)\tinconsistencies.\n",
"severity": 2,
"source": "LaTeX"
}
]
}
}
}demoNote how this request now only contains the single, valid, diagnostic. This behavior may or may not work for you. Say for example you wanted to handle invalid diagnostics the same regardless of the context in which they appeared. Another option would be to provide a hook for the pygls integration To use your custom converter with YaLafi-LS you can then pass server = LanguageServer(
name="my-language-server", version="v1.0", converter_factory=my_converter_factory
) |
|
Thank you for this very detailed answer @alcarney. It works perfectly. It might be possible to generalize that using a Would you like to include something like that as an option in pygls, or would you want a little documentation about what you wrote above. I am happy to invest some time here. |
Thanks very much! 😄 Personally, I'd start with a bit of documentation to show that it's possilble. If a lot of servers start implementing their own non-strict mode it might highlight some common patterns that |
|
One potential hook we could add to a... 🤔 Context: vscode-restructuredtext/vscode-restructuredtext#429 & vscode-restructuredtext/vscode-restructuredtext#433 |
Description of a VS Code bug I experienced
While writing a Language Server using pygls, I experienced a VS Code issue where the LaTeX Workshop extension (which doesn’t use LSP, but is a ‘usual’ typescript VS Code extension) publishes a diagnostic entry at line
NaN. To my understanding, this is in spec with the corresponding VS Code types used in typescript, but still a bit weird. [corresponding LaTeX Workshop issue] However, when sending Code Action Requests, VS Code sends"line": nullwhich, to my understanding, is not in spec with the LSP. [corresponding VS Code issue] To make it even worse, these diagnostic entries are sent with all Code Action Requests, although they are only displayed in the first line in VS Code. As a result pygls shows an error (see below for details) whenever one moves in the document, because VS Code will send Code Action Requests.Suggestion for pygls
The question I want to bring up here is, whether it would be feasible to handle wrong types less strict?
And so far I would suggest the following: Assume pygls gets a valid JSON message, but with wrong types. Whenever these wrong types live inside an optional entry (i.e. an item in a list or an entry marked as optional in LSP), one could just remove the optional parts, issue a warning, but proceed. In my case, the faulty diagnostics should just be removed from the Code Action Request. If there were additional valid diagnostics from my own language server, they would survive.
The error I experienced
For how to reproduce the faulty diagnostics published by the LaTeX Workshop extension, see the corresponding LaTeX Workshop and VS Code issues. Here I want to just give the JSON message sent to pygls and the corresponding log. Note that there were no good diagnostics from my own language server in the test document, the diagnostics come from the LaTeX Workshop extension.
JSON message
{ "jsonrpc": "2.0", "id": 10, "method": "textDocument/codeAction", "params":{ "textDocument":{ "uri": "file:///.../test.tex" }, "range":{ "start":{ "line": 7, "character": 14 }, "end":{ "line": 7, "character": 14 } }, "context":{ "diagnostics":[ { "range":{ "start":{ "line": null, "character": 0 }, "end":{ "line": null, "character": 65535 } }, "message": "Using \\overbracket and \\underbracket from\n(unicode-math)\t`mathtools' package.\n(unicode-math)\t\n(unicode-math)\tUse \\Uoverbracket and \\Uunderbracket for\n(unicode-math)\toriginal `unicode-math' definition.\n", "severity": 2, "source": "LaTeX" }, { "range":{ "start":{ "line": null, "character": 0 }, "end":{ "line": null, "character": 65535 } }, "message": "I'm going to overwrite the following commands\n(unicode-math)\tfrom the `mathtools' package: \n(unicode-math)\t\n(unicode-math)\t\\dblcolon, \\coloneqq, \\Coloneqq, \\eqqcolon.\n(unicode-math)\t\n(unicode-math)\t\n(unicode-math)\tNote that since I won't overwrite the other\n(unicode-math)\tcolon-like commands, using them will lead to\n(unicode-math)\tinconsistencies.\n", "severity": 2, "source": "LaTeX" } ] } } }pygls log
The text was updated successfully, but these errors were encountered: