[LSP] Can't find definition of external dependencies

[ericdallo]

Using only LSP (no repl connected) It seems calva can't find the definition of external dependencies/jars even if clojure-lsp returns the correct response.

• This makes calva doesn't go to the jar file
• This makes calva not how properly the hover information of external jars after trying to open the definition

It's seems to me related with custom jar parsing code of calva

[Cyrik]

I'm looking into it.
I think it might have something to do with the change I made to the util getJarContents function. It previously didn't resolve the promise on not found files and now returns an empty string.

[Cyrik]

Ok so sadly it's not my change. As far as my testing shows it broke on 2.0.229 release.
I'll have a look into the changes, but I'm not sure how much more time I have this weekend.

[Cyrik]

So it seems to be caused by #1431 ironically.
If I comment out { scheme: 'jar', language: 'clojure' } in my calva dev build I can go-to def into a jar file. I just can't use LSP stuff inside the jar anymore...

[ericdallo]

Good finding @Cyrik, we could as last resource revert that, but it'd be perfect if we would understand why this happen with that change

[Cyrik]

So I'm giving up for now. I've found out that the sendRequest to the LSP server for 'textDocument/definition' actually returns the correct result for the first call, but returns nil on every call afterward. If I throw out the { scheme: 'jar', language: 'clojure' } it keeps working on successive calls.

@ericdallo So it seems the calls for 'codeLense' and so on that happen inside the jar produce a state inside LSP that makes it return null on 'textDocument/definition'. So I'm not sure anymore that the bug is in Calva.

[ericdallo]

actually returns the correct result for the first call, but returns nil on every call afterward

Last time I investigate I noticed the same! very odd indeed

@Cyrik Hum, do you have client<->server logs showing that clojure-lsp is returning null?

[Cyrik]

@ericdallo No, I see it when stopping in the debugger on calva itself. I still haven't found the client<->server logs for LSP, only the LSP log itself 😅

[ericdallo]

@Cyrik Check Viewing the Logs Between the Client and Server

[bpringe]

@ericdallo Is this issue related to this one: #1458?

[ericdallo]

Oh, yes, sorry, I didn't find easily that issue before opening, feel free to close this one if you think it makes sense

[bpringe]

I closed the other, since this one has troubleshooting info already.

[bpringe]

Ok, so here are some messages between client and server when it works and when it doesn't work, which basically just confirms what @Cyrik said. I ran Go to Definition on lower-case from clojure.string, it worked, I closed out the clojure.string namespace file, then ran Go to Definition again on lower-case, and it did not work.

--- Worked ---

[Trace - 12:52:29 PM] Sending request 'textDocument/definition - (10)'.
Params: {
    "textDocument": {
        "uri": "file:///Users/brandon/development/clojure-test/src/core.clj"
    },
    "position": {
        "line": 13,
        "character": 12
    }
}


[Trace - 12:52:29 PM] Received response 'textDocument/definition - (10)' in 21ms.
Result: {
    "uri": "jar:file:///Users/brandon/.m2/repository/org/clojure/clojure/1.10.3/clojure-1.10.3.jar!/clojure/string.clj",
    "range": {
        "start": {
            "line": 212,
            "character": 14
        },
        "end": {
            "line": 212,
            "character": 24
        }
    }
}

--- Did not work ---

[Trace - 12:53:03 PM] Sending request 'textDocument/definition - (23)'.
Params: {
    "textDocument": {
        "uri": "file:///Users/brandon/development/clojure-test/src/core.clj"
    },
    "position": {
        "line": 13,
        "character": 12
    }
}


[Trace - 12:53:03 PM] Received response 'textDocument/definition - (23)' in 11ms.
No result returned.

[bpringe]

@ericdallo This does seem to be an issue in clojure-lsp. What Calva is sending to it seems fine, and is the same both when it works and doesn't work. 🤔 I don't know what could be going wrong in clojure-lsp, though.

[ericdallo]

I really doubt it's a clojure-lsp issue as it works for other editors, but if we prove it is I'd be glad to fix it.
Do we have any logs showing clonire-lsp returning nothing for that?

Maybe calva is not using the dependencyContents request to retrieve the jar content like emacs and vim does?
https://github.com/emacs-lsp/lsp-mode/blob/master/clients/lsp-clojure.el#L427

[bpringe]

Do we have any logs showing clonire-lsp returning nothing for that?

Yes, see my comment above. I'm not sure what Calva can do about this if clojure-lsp sends nothing back, unless Calva is doing something before the request that could cause clojure-lsp to send nothing back, but I don't know what that could be.

[Cyrik]

It feels to me like we are breaking LSP with the jar contents. Right after the jar opens we send:

[Trace - 11:16:10 PM] Sending notification 'textDocument/didOpen'.
Params: {
    "textDocument": {
        "uri": "jar:file%3A///Users/lukas/.m2/repository/org/clojure/clojure/1.10.3/clojure-1.10.3.jar%21/clojure/core.clj",
        "languageId": "clojure",
        "version": 1,
        "text": ";    ... the whole file contents ...

There are a few messages in between that don't look too bad but at some point I get:

Received response 'textDocument/semanticTokens/full - (16)' in 25588ms.
Result: {
    "data": [
        8,
...

with a very, very long array of numbers.

This would also be consistent with the observation that it's ok for the first call and breaks right after, always returning No results found. The jar contents breaking LSP would also explain why it keeps working if we don't allow LSP to work inside jars.

[ericdallo]

Yes, see my comment above. I'm not sure what Calva can do about this if clojure-lsp sends nothing back, unless Calva is doing something before the request that could cause clojure-lsp to send nothing back, but I don't know what that could be.

Sorry, I missed your comment.
Yes, quite odd the log say clojure-lsp returned null, I'll debug clojure-lsp server side to confirm that

[ericdallo]

Good and bad news:

The good news is that I found, the bad (for me) is that it's a clojure-lsp bug 😅
Sorry for not taking a closer look before, the issue is that calva sends the jar URI encoded, example:

 jar:file%3A///home/greg/.m2/repository/clj-commons/iapetos/0.1.12/iapetos-0.1.12.jar%21/iapetos/collector/ring.clj

This makes clojure-lsp think that the file doesn't exists when receives a didClose event and remove all analysis of that file thinking it was deleted here

I'll try to make a fix on clojure-lsp side, but maybe we should understand why calva send the URI encoded like that, but not on other places @bpringe @Cyrik

Edit:
It seems the file is properly parsed by clojure-lsp, but clojure-lsp think that file doesn't exists, so probably fixing that should fix the whole issue

[ericdallo]

Fixed, it should be available on next clojure-lsp version, sorry for not noticing it before. LMK if any issues

[Cyrik]

It's fine, I didn't like my sanity anyway 😇
I think it's sending that one when we say to use LSP inside JARs, otherwise, it skips sending jar files to LSP.

[ericdallo]

I think the tricky part of that is that vscode send didClose which encoded uris which other editors not encode the URI, not sure it's a bug on client though

[Cyrik]

@ericdallo didClose sounds confusing, not sure why it's needed. But it does say to send the uri unless I'm misunderstanding your statement and that it's only meant as "hey, we aren't displaying this currently" message.

[ericdallo]

Yes, you are correct, didClose means We are not displaying this document anymore, it was closed from the editor, but it doesn't means the file was removed though which server should check.

[bpringe]

I think it's the vscode-languageclient library, or VS Code itself, sending the URIs encoded like that. I don't know the reasoning.

[bpringe]

I'll close this since the fix will be released in clojure-lsp soon, so we don't have to manually close this later.