Capturing quick fix telemetry

[rcjsuen]

If you're working on a language server with a lot of stuff computed, you may be including the WorkspaceEdit when you send CodeAction items back to the client. This unfortunately means that the server can send actions back to the client until the cows come home without ever realizing the user has accepted any of them. How are other language server authors capturing this kind of usage telemetry? I see a few options...and none of them seem very attractive.

1. Server parses textDocument/didChange and determines which code action was applied. I presume people are not using this option.
2. Client uses codeAction.command and then self-calls a local telemetry.callback.command and sends telemetry back home. This means every LSP client will have to add this code when integrating with said language server which is not ideal.
3. Server declares telemetry.callback.command in its server capabilities ({"executeCommandProvider": ["telemetry.callback.command"}) so that workspace/executeCommand can be used. There's a lot of back and forth to achieve this but it does at least keep the telemetry code in the server and the client does not have to do anything extra (assuming the client supports workspace/executeCommand in the first place).

It feels to me like 3 is the best choice even if it means there are a lot of round trips but I wanted to open this up for discussion to see if I just missed something obvious...? 🤔

Option 1:

sequenceDiagram
user->>client: opens a document
client->>server: textDocument/didOpen (notification)
client->>server: textDocument/codeAction (request 1)
server->>client: textDocument/codeAction (response 1)
user->>client: accepts a quick fix code action
client->>document: applies the codeAction.WorkspaceEdit
client->>server: textDocument/didChange (notification)
server->>telemetryServer: parses content change and then sends telemetry

Loading

Option 2:

sequenceDiagram
user->>client: starts the client
client->>server: initialize (request 1)
server->>client: initialize (response 1) with telemetry.callback.command as ExecuteCommandOptions
user->>client: opens a document
client->>server: textDocument/didOpen (notification)
client->>server: textDocument/codeAction (request 2)
server->>client: textDocument/codeAction (response 2)
user->>client: accepts a quick fix code action
client->>document: applies the codeAction.WorkspaceEdit
client->>telemetryServer: looks up codeAction.command and uses telemetry.callback.command locally and sends telemetry back

Loading

Option 3:

sequenceDiagram
user->>client: starts the client
client->>server: initialize (request 1)
server->>client: initialize (response 1) with telemetry.callback.command as ExecuteCommandOptions
user->>client: opens a document
client->>server: textDocument/didOpen (notification)
client->>server: textDocument/codeAction (request 2)
server->>client: textDocument/codeAction (response 2)
user->>client: accepts a quick fix code action
client->>document: applies the codeAction.WorkspaceEdit
client->>server: textDocument/didChange (notification)
client->>server: look up codeAction.comand, request workspace/executeCommand with telemetry.callback.command (request 3)
server->>telemetryServer: sends telemetry event
server->>client: workspace/executeCommand (response 3)

Loading

[rcjsuen]

I guess if you need to ask the server to run telemetry.callback.command with workspace/executeCommand...then you might as well just not use codeAction.edit and use codeAction.command instead and pairing it with a workspace/applyEdit call back to the client. 🤔

[dbaeumer]

Actually no good idea either. There is a sendTelemetry route from the server to the client to help reporting telemetry (https://microsoft.github.io/language-server-protocol/specifications/lsp/3.17/specification/#telemetry_event).

You could also stick the telemetry into the data field of the code action and then do the logging from the client