Library Related API Design for the Language Server

[radeusgd]

Summary

We need to design and document what API changes are required for the Language Server so that the IDE can use the library-related features.

Value

• We have a clear definition of the new API and so both sides can use it as a reference for the implementation.

Specification

• Design the required endpoints
  • List available editions
  • Get/modify edition settings of a project.
    • Set if local libraries should override published ones.
    • Add/remove local library override.
    • Get the edition and the above overrides.
  • List libraries added to a project.
  • Create a new library project (will be used in extractions).
    • This endpoint may be done in the Project Manager, but that depends on how cloud should be handling that.
    • This in particular needs to be discussed how that will be integrated with the cloud.
  • Publishing a library.
    • And any related endpoints for authorization that may be required to publish.
  • (Pre)Installing a library.
    • Including progress notifications.
  • Remember that the progress notifications can also show-up for example when the imports are manually modified or on startup.
  • Adding/removing libraries is handled by adding/removing imports, so the imports themselves will be added within the IDE as is the case with the searcher.
• Make sure if there are other endpoints than the above that should be added.

Acceptance Criteria & Test Cases

• The design is documented and discussed.

[radeusgd]

As shortly discussed with @iamrecursion this cannot be done directly in the project-manager, because the Cloud uses a separate mechanism for managing projects and we would need to duplicate that work.

So most likely this will need to be done in the Language Server.

What we need then is a way for the LS to:

• localize the package manager executable (related issue (TODO)),
• modify the project's package.yaml to check/change the edition.

When the edition is modified, the context may likely need to be reinitialized (because library versions may have changed in an incompatible way).
@4e6 Do you think this context reset and package.yaml can be done from the Language Server in a reasonable way or should we consider other designs (like doing it in the project manager anyway)?

[radeusgd]

Since the package-manager will be a library, my first guess would be to have it be used directly by the launcher and project-manager. But this poses the issue that the cloud distribution would need to roll their own logic for using that (as they have a separate project-manager) and we do not want to duplicate this work.

Thus it seems mostly natural to 'put' the package-manager into the language server. It would actually be a nice solution if the package-manager would be version-bound with the engine rather than with the launcher/project-manager, as any updates to the package-manager logic can then be downloaded with newer engine versions.

For the CLI frontend we can provide a frontend in the runner.jar (which is responsible for starting the actual language server, REPL etc.) and then just wrap it within the launcher, so that the package manager from the engine that belongs to a given project is always used.

[radeusgd]

There are two issues which may be complicated to get working on the cloud, so let's discuss them with @lolczak:

1. Creating new libraries.
   We want to be able in the IDE to extract a node into an existing or new library, so we need a way to create a new library and also modify it.
   This seems like it should be done in the project-manager since it is the one responsible for creating new projects, and in the cloud this logic may need to differ, so let's discuss the differences.
   This is actually also non-trivial even locally - we need to modify files of another project - can the language server do that? How do we know that other instances do not modify this? (cc: @MichaelMauderer)
2. Downloading libraries (Downloading and Installing Libraries in the Package Manager #1772) - when we download libraries we need to put them somewhere. In the desktop version that will be somewhere next to the engine distributions. How will this work on the cloud? Will each instance have a separate storage for that? In the local version we will likely re-use our locking mechanism for ensuring consistency, will something similar be needed in the cloud? If yes it likely will need to be a separate mechanism altogether.