Allow to publish an extension pack without need to publish all extension to registry at first

[vymarkov]

First of all, I want to say a huge thanks for all community that work on this registry to provide for all of us use such awesome products like GitPod :)

I'm working on gitpod to adopt this tool as my IDE on daily basis for a few open source projects. at the same time I've been using VS Code (build by microsoft) since vscode was released, then I've created an extension pack to install all extensions on every new machine and share with my colleagues without hassle.

A few months ago I started to use Gitpod then I faced with issue that didn't allow to me to publish my extension pack cause some of extensions were not published.

So there is my question.

Could we allow to publish extension packs even if some extensions from pack are not presented yet on Open VSX?

It will allow to install most of extensions, for instance Node.js + DevOps Extension Pack
include 29 extensions but only 2 extensions are not present on Open VSX registry, however it could be more useful to publish extension pack to avoid installing of all extension manually

Possible issues:

• extension pack can include extensions that are not present even on Microsoft Marketplace

I would like to hear your thoughts.

[spoenemann]

Do you know how the clients (VS Code, Gitpod etc.) handle this? Would they gracefully install only parts of an extension pack if they cannot resolve all extensions?

[vymarkov]

Do you know how the clients (VS Code, Gitpod etc.) handle this? Would they gracefully install only parts of an extension pack if they cannot resolve all extensions?

In order to ensure that Gitpod, VS Code, VS Code and Code Server does not fail i perform the next steps:

1. Clone repo https://github.com/vymarkov/nodejs-devops-extension-pack.

$ git clone https://github.com/vymarkov/nodejs-devops-extension-pack
$ npx vsce package
$ open command pallate and use command  'Extensions: Install from VSIX' to install extension from local workstation

I've tested it on GitPod with enabled Code editor, it works without any issues.

Version: 1.49.0
Commit: 20ded0d97c41cb686098d14b9e1f0533177300d1
Date: 2020-12-17T09:27:27.064Z
Browser: Mozilla/5.0 (Macintosh; Intel Mac OS X 11_0_0) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/87.0.4280.88 Safari/537.36 OPR/73.0.3856.284

Also I've tested it on VSCodium that use Open VSX registry as well, it works pretty well.

Version: 1.52.1
Commit: ea3859d4ba2f3e577a159bc91e3074c5d85c0523
Date: 2020-12-17T00:29:09.284Z
Electron: 9.3.5
Chrome: 83.0.4103.122
Node.js: 12.14.1
V8: 8.3.110.13-electron.0
OS: Darwin x64 20.2.0

Also I've tested it on Code Server with enabled Open VSX registry, it works pretty well.

code-server: v3.8.0
 VS Code: v1.51.1
Commit: c4610f7829701aadb045d450013b84491c30580d
Date: 2020-12-18T20:25:37Z
Browser: Mozilla/5.0 (Macintosh; Intel Mac OS X 11_0_0) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/87.0.4280.88 Safari/537.36 OPR/73.0.3856.284

[vymarkov]

Unfortunately I can't test it on GitPod Theia editor cause this editor don't allow to install extension from vsix, docs don't contain any info about that.

[spoenemann]

Currently we store direct references to the contained extensions in the database. We would need to change this to a list of "namespace.extension" strings instead, which requires a DB migration.

[vymarkov]

btw I would like to describe another use case when we probably should allow to publish extensions pack with missing extensions.

Some extensions are not open sourced and can't be published by building extension from source and still not published by owner for any reason into open vsx registry.

For instance:

• wallabyjs.quokka-vscode

UPD:

Extensions that are not open source could not be published to Open VSX registry due to licence restrictions.

See more info at wallabyjs/public#2436 (comment)

I've just wanna add that it becomes very important to allow publish extension pack even if some extension is not present in OpenVSX registry to migrate hundreds of packs that are very useful to establish the desirable developer experience by using them.

[spoenemann]

Open VSX Registry is not restricted to open source licenses. In the issue you pointed at, they've been looking at the wrong agreement. The relevant one is this: https://www.eclipse.org/legal/documents/eclipse-openvsx-publisher-agreement.pdf

[vymarkov]

Currently we store direct references to the contained extensions in the database. We would need to change this to a list of "namespace.extension" strings instead, which requires a DB migration.

@spoenemann hi, are there any plans to implement that? it's a huge blocker to move to GitPod finally :)

[GitMensch]

@vymarkov I've read this discussion but I still fail to see how it benefits the user if he tries to install an extension pack where possibly the most important extension to get the desired result is not available. Please elaborate on that.
The db migration issue is a separate issue for me and does sound reasonable in any case. @spoenemann would this effect performance?

[vymarkov]

@GitMensch it's better to have 9 extensions from 10 extensions installed by using an extension pack rather nothing at all.

It just allows to don't modify an extension pack that exists on VS Marketplace, the pack is the same for both registries.

There is another question, what will happen to extension pack if author of included extension decides to remove from the registry? It looks like the extension pack should be also unpublished.

[spoenemann]

The proposed change would decouple extension packs from the extensions they contain. We would need to indicate in the UI / API whether a contained extension is available or not.

I don't have time to implement this change at the moment, but I'm ok with it. So if anyone is interested to contribute, please go ahead!

We use Flyway for DB migration, which is quite easy. It should be possible to implement this with plain SQL statements. The table is defined here:

openvsx/server/src/main/resources/db/migration/V1__Base_version.sql

Lines 115 to 118 in a9154dc

	CREATE TABLE public.extension_version_bundled_extensions (
	extension_version_id bigint NOT NULL,
	bundled_extensions_id bigint NOT NULL
	);

bundled_extensions_id should be changed to a string of the form {namespace}.{extension}.