vsx-registry: implement extension compatibility handling

[vince-fugnitto]

What it does

Fixes: #7464

The following pull-request includes extension compatibility handling when searching, and installing extensions from the open-vsx view. The changes include using the new open-vsx api to determine a compatible version of an extension by parsing the engines.vscode property against our supported API version.

The pull-request includes:

• checking for extension compatibility based on the engines.vscode property
• obtaining the supported API version through the cli (defaults to our default api version)
• handling of installing specific extension versions
• update search with new open-vsx api (downloadUrl): question: should the search response include the downloadUrl as part of 'allVersions' eclipse/openvsx#149
• performance improvements

How to test

1. start the application (plugins should work correctly as before)
2. the extensions view should search for plugins which are compatible with the current api version
3. installing plugins should work correctly as before
4. the new command should display the current api version correctly
5. using yarn start --vscode-api-version={version}, the api version should properly be updated, searching for extensions should still respect the cli argument

Review checklist

• as an author, I have thoroughly tested my changes and carefully followed the review guidelines

Reminder for reviewers

• as a reviewer, I agree to behave in accordance with the review guidelines

Signed-off-by: vince-fugnitto [email protected]

[vince-fugnitto]

We currently perform multiple api.getExtension(id) calls on master which makes the optimization we did on open-vsx (to return the engines during the search so we can determine the compatibility) not as useful as it should be.

We should probably adjust the code to properly store an extension's information (obtained through the search) which would then be used to perform different operations such as installation. At the moment, we are performing multiple external requests and always solely by id which is insufficient to obtain the correct version of an extension.

[vince-fugnitto]

I'll need to address the recent changes to open-vsx to make the pull-request working again.

[vince-fugnitto]

@spoenemann are there any plans for the openvsx api to support fetching only compatible extensions from the marketplace, based on a client's expectations? For example, being able to pass the currently supported engines.vscode version as query parameter so that clients (ex: theia-based apps) can only install extensions which are more likely to be supported, and not always "latest".

Currently, the api does not allow clients to easily determine and fetch compatible extensions without multiple queries which raises performance concerns. In addition, each client would have to perform this same extension compatibility fix instead of the server supporting it for all.

• vsx-registry: implement extension compatibility handling #8191 (comment)

[spoenemann]

I added the engines to search results in eclipse/openvsx#144. Since eclipse/openvsx#181 you need to set includeAllVersions=true for this information to be included. Isn't that enough?

[vince-fugnitto]

I added the engines to search results in eclipse/openvsx#144. Since eclipse/openvsx#181 you need to set includeAllVersions=true for this information to be included. Isn't that enough?

@spoenemann I am already using includeAllVersions=true in the changeset when using the query api, but this only gives me all versions of an extension and their url. I still need to include additional handling to iterate over the result list, and fetch individual extensions to determine if they are compatible (which is the blocking issue at the moment for performance, and not querying excessively the registry).

For example: (theme-dracula-at-night):

{
  latest: 'https://open-vsx.org/api/bceskavich/theme-dracula-at-night/latest',
  '2.6.1': 'https://open-vsx.org/api/bceskavich/theme-dracula-at-night/2.6.1',
  '2.6.0': 'https://open-vsx.org/api/bceskavich/theme-dracula-at-night/2.6.0'
}

Perhaps you see something incorrect in my approach, or notice some room for improvement in the api itself.

[spoenemann]

The includeAllVersions=true option is only for the search endpoint, not for query. What is the situation where you would use query and need full compatibility data?

[vince-fugnitto]

The includeAllVersions=true option is only for the search endpoint, not for query. What is the situation where you would use query and need full compatibility data?

Correct, we currently use the query endpoint to fetch compatible extensions when performing a refresh and resolve of an extension, which is used to update the installed version on startup when newer versions of a given plugin are available.

@akosyakov due to this limitation, and the fact it is used for resolve and refresh, do you still believe that there is a performance degradation of the application (ex: startup) which needs to be addressed before the changes can be accepted?

Extensions are installed during the first start of workspaces in Gitpod it can significantly affect first time experience for each new workspace if it is slow.

For this comment, how would you propose the api or functionality be updated in order to ensure only compatible extensions are installed, without the negative performance impact.

[spoenemann]

We could add an engine option to the query endpoint so it returns data of the latest version that supports that engine? The value could be something like

    "engine": "[email protected]"

[vince-fugnitto]

We could add an engine option to the query endpoint so it returns data of the latest version that supports that engine? The value could be something like

    "engine": "[email protected]"

@spoenemann that's what I was hoping for, to add a new query parameter to make it easier for clients to determine compatibility. Having the engines option would definitely be helpful 👍

[spoenemann]

I totally forgot that there's an includeAllVersions option in the query endpoint. I though you mean the option with the same name at the search endpoint.

By posting a query with includeAllVersions: true, you get an array of the full metadata of all versions of the extension, including the engines of every version. By using this it should not be necessary to send multiple requests per extension.

[vince-fugnitto]

@spoenemann thank you for your response on the way forward, I updated the pull-request to do as you suggested.

[vince-fugnitto]

@marechal-p thank you for the feedback, I applied your suggestions 👍

[spoenemann]

Thanks, the code looks good to me now.

[marcdumais-work]

@vince-fugnitto I think you wanted to wait after this week's release before merging, so we have next month to fix any issues, before the PR is part of a release. I think it makes sense.

[vince-fugnitto]

@vince-fugnitto I think you wanted to wait after this week's release before merging, so we have next month to fix any issues, before the PR is part of a release. I think it makes sense.

Right, I'll wait for after this week's release, fix conflicts and merge if there are no objections 👍
Thank you for the review Miro!