LSP import completions prefix search

[lucacasonato]

Some registries are too large to return a list of all modules in a JSON endpoint (for example Skypack). For registries like these we should provide a way to provide partial results. Partial results would be fully opt-in. I think we could do this by adding a new "partial" property to variable definitions in the registry schema. This "partial" property could either be undefined/false or true. If the property is true the "url" property of a variable would be allowed to include a replacement variable for it's own key. The LSP would then query this url with the partial variable after every user keystroke. A small example:

{
  "version": 1,
  "registries": [
    {
      "schema": "/:package([a-z0-9_]*)@:version?/:path*",
      "variables": [
        {
          "key": "package",
          "url": "https://api.example.com/packages?filter=${{package}}",
          "partial": true
        },
        {
          "key": "version",
          "url": "https://api.example.com/packages/${package}/versions"
        },
        {
          "key": "path",
          "url": "https://api.example.com/packages/${package}/${{version}}/paths?filter=${{path}}",
          "partial": true
        }
      ]
    }
  ]
}

cc @FredKSchott

[lucacasonato]

If we can land this before Tuesday it won't be a breaking change to the schema 🙃

[lucacasonato]

Actually we might not even need the partial option. We can imply it supports partial responses by checking if the completion URL contains the completed variable itself. I will update the proposal.

[kitsonk]

I don't want to race and ship it before Tuesday. We need to think more about this, like under what conditions would we require, and how do we cache the responses. If we query on every keystroke, that could easily overwhelm the client or the server. It would be good to ship the feature and get some wider feedback as well.

I'm not sure how we could imply that it is a partial response from the server, if queried for the variable initially with "" and it didn't return anything, is that because it is actually empty or partial? What if a "package" actually didn't have any files?

I would like to wrap everything up into a "v2" and be a bit more planned about it. I think we could find a way to combine the repeats and partials.

[kitsonk]

Actually with the registry v2 protocol, we now provide the ability to do this ("isIncomplete": true in the response) which mirrors the flag on completion items sent via the lsp to the client. This provides a mechanism, per request, to indicate that further typing would change the result set. The implementation on deno.land/x does this, as we don't return 5500 modules to the client when providing completions! 😉