[extension] already published, but currently isn't active and therefore not visible

[filiptronicek]

For many extensions, something goes wrong during the publishing making the new versions not "active" on the registry.

This happens for many extensions and is really inconvenient, because there is no apparent action to be taken on the side of the extension author. The occurrence of this issue seems random and I don't see a pattern with what goes and what doesn't.

From the logs:

❌ Extension ms-toolsai.jupyter 2023.8.100 is already published, but currently isn't active and therefore not visible.

Possibly related to #1444

[filiptronicek]

Notably, this is happening for the following extensions:

• ms-python.python (1.7M downloads)
• ms-toolsai.jupyter (1.1M downloads)
• ms-azuretools.vscode-docker (516K downloads)
• golang.Go(307K downloads)
• vscjava.vscode-java-test (305K downloads)
• bradlc.vscode-tailwindcss (115K downloads)
• ms-vscode.cpptools-themes (20K downloads)

@kineticsquid I think we should mark this as a high-priority issue, because it's disrupting the use of the registry 🙏

[filiptronicek]

I just tried publishing a smaller extension and the same issue occured. It looks pretty widespread.

[kineticsquid]

@filiptronicek Agree with the priority and agree that something funny is going on. Here's another data point: #2050. In this case, it looks like the extension got into a bad state with a publication that was not activated. Then, no matter what was published, it was not activated. I tried and failed. I was able to publish to the same code to my namespace with no problem. When I removed the deactivated versions, I was able to publish.

[filiptronicek]

Because of this issue, our biggest extension on the public instance of the registry, ms-python.python with close to 2 million downloads is outdated by two release versions.

I am not sure how exactly we can proceed here considering the scale. @kineticsquid could you for these cases try to unpublish the extensions listed above so that I can try re-publishing them? It's no long-term solution, but I'm curious to see how exactly it will come back

[kineticsquid]

@filiptronicek I tried a few experiments last night. The extensions golang.Go and bradlc.vscode-tailwindcss provided .vsix files for the releases, the other do not. I downloaded the latest version we show as active and all the releases after those. I then attempted to publish these extension versions to our staging environment. The publishing was successful including the versions that are showing as deactivated in production.

I then added myself as a contributor to these two name spaces in production and attempted to publish the same .vsix files. That worked. You can see the more recent versions for Go and vscode-tailwindcss.

I removed ms-toolsai.jupyter 2023.8.100 and ms-python.python 2023.14.0, these were the releases that were not activated. Can you try to re-publish these now. One thing I noted, the repo link for the Jupyter extension only had releases up to 2022.

If these works, I'd like to go back and remove the other versions I published and have you re-publish those.

[filiptronicek]

Thanks a bunch for your effort here, @kineticsquid 🙏. I'll take a look at Python and Jupyter now - Jupyter indeed has stoppped publishing GitHub Releases, so they only release to the Microsoft Marketplace now.

I'll get back to you and we'll see about republishing the extensions you fixed.

[filiptronicek]

@kineticsquid Python and Jupyter published ✅

[kineticsquid]

Woo Hoo!

[zFernand0]

Hey all,
I'm currently experiencing this issue and I'd like to know if the root cause is known?
In my case, I'm trying to publish the Zowe.cics-extension-for-zowe, and even though the CLI works fine, I can't seem to get metadata information or republish the same version since it's published as deactivated.

Any help will be greatly appreciated! 😋

---

UPDATE:
The extension now shows as Active and I can ovsx get it 😋

Thank you!

[DanTup]

I published v3.72.0 of the Dart extension earlier today, and then 3.72.1 with a fix around 30 mins ago. The initial release showed up, but I'm seeing this issue after publishing the fix. I'm not sure whether it just takes a while to go through or if it's broken.

The fix is quite important so if it doesn't show up in the next 20-30 mins I'm just going to bump the version number and try again, but I thought it was worth recording here in case you can gain any insight from logs/whatever.

Edit: 3.72.1 just showed up on the website (although not yet osvx --metadata.. edit2: it did right after I posted). I did push two pre-release versions to see if they made any difference but they didn't. Unless someone manually fixed this, seems like there might be a 50-60 min delay?

[kineticsquid]

@amvanbaren Might be able to add more here. The file processing is done asynchronously. As a consequence of this, an extension tile will show deactivated until the file processing is complete (and there are no errors).

@zFernand0 Any idea how long after publishing it took your extension to show up as active?

@DanTup Similar question, how long after your extension showed as active it take for ovsx --metadata to return the info?

[DanTup]

@kineticsquid it showed up in metadata very shortly after it showed up on the website (right after I posted above). So it was around 1hr for the extension to show up both on the website + metadata, there was just a very slight delay between them (which I suspect was just some caching).

[shivaypiece]

Any estimated timeline for fixing this? @kineticsquid

[filiptronicek]

@kineticsquid could you please look into removing the latest of https://open-vsx.org/extension/vscjava/vscode-java-test? It should be 0.40.0 in the database and server, but probably not enabled again, since we cannot publish to the version, although this time with no error message – just a successful message from the CLI, although publishing the same version multiple times.

The latest downloadable version, 0.39.1 has shown to be broken, therefore the need to update to 0.40.0.

[kineticsquid]

unpublish both 0.39.1 and 0.40.0?

[kineticsquid]

Removed 0.40.0.

[kineticsquid]

@filiptronicek @amvanbaren Do we need to keep this one open?

[filiptronicek]

@kineticsquid let's close and re-open if needed 🙏

[hoangphamEclipse]

I investigated this issue on my end and I have some interesting findings that I would like to share. While debugging locally, I found out that this is a concurrency issue. When a new extension is published, the web ui will send a request to the server to publish the extension [1], then immediately send another request to re-fetch the list of extensions published by the user [2]. The server will run these 2 requests in parallel.

The problem occurs because we are trying to write to the database while performing a GET request. The diagram below shows how [1] and [2] affect the active status of an extension as they are executed when the error happens.

The problem is caused exactly by this line: https://github.com/eclipse/openvsx/blob/84d01402f007233c8a1e5b28587e4f4784c7d3fd/server/src/main/java/org/eclipse/openvsx/util/VersionService.java#L38

entityManager.merge(extension) will try to persist the extension parameter to the database. If extension is stale - that is if it is retrieved before its status is updated by [2] - then the entityManager will overwrite the changes done by [2] with the stale version, ultimately making the extension inactive. I think the intention of calling merge(extension) is to obtain a JPA managed extension thus enabling the code to fetch its extension versions lazily; however, the side effect is this bug.

[DanTup]

This still seems to be happening.. I noticed the last two Dart-Code releases (one around a month ago, and one earlier today) are not showing. Both of them give this same error if I try to re-publish them:

PS C:\Dev\Dart-Code\Dart-Code> npx ovsx publish dart-code-3.80.0.vsix
❌  Extension Dart-Code.dart-code 3.80.0 is already published, but currently isn't active and therefore not visible.
See the documentation for more information:
https://github.com/eclipse/openvsx/wiki/Publishing-Extensions

PS C:\Dev\Dart-Code\Dart-Code> npx ovsx publish dart-code-3.82.0.vsix
❌  Extension Dart-Code.dart-code 3.82.0 is already published, but currently isn't active and therefore not visible.
See the documentation for more information:
https://github.com/eclipse/openvsx/wiki/Publishing-Extensions

https://open-vsx.org/extension/Dart-Code/dart-code

It's possible the latest one just hadn't gotten through the caches yet, but it's been a month since 3.80.0 was published.

Is it possible someone can either make them active or delete them so I can re-publish them?

[kineticsquid]

@DanTup I just looked and I see 3.82.0, but not 3.80.0. Re-opening this one. Separate from fixing the root cause, do you want me to unpublish 3.80.0 so you can re-publish?

[DanTup]

@kineticsquid that would be great, thanks!

[kineticsquid]

@DanTup 3.80.0 is toast.

[DanTup]

@kineticsquid thanks! I re-published that one and it's showing up now :)

[sjsepan3]

@kineticsquid
Just had this happen.
eclipse/openvsx#783 (comment)

UPDATE: I had several to update this last weekend, all of which were affected:

user, ext, new, active
sjsepan, sjsepan-e-inkish, 0.1.0, 0.0.4
sjsepan, sjsepan-newspaperish, 0.3.0, 0.2.7
sjsepan, sjsepan-sketchish, 0.1.0, 0.0.2
sjsepan, sjsepan-matrixish, 0.2.0, 0.1.3
sjsepan, sjsepan-humanelike, 0.4.2, 0.3.1
sjsepan, sjsepan-blueish, 0.3.1, 0.2.4
sjsepan, sjsepan-reddish, 0.3.0, 0.2.3
sjsepan, sjsepan-purpleish, 0.3.0, 0.2.3
sjsepan, sjsepan-greenish, 0.3.1, 0.2.4
sjsepan, sjsepan-limeish, 0.3.0, 0.2.3
sjsepan, sjsepan-redalertish, 0.2.0, 0.1.2
sjsepan, sjsepan-sneakersish, 0.2.0, 0.1.2
sjsepan, sjsepan-ambercrtish, 0.2.1, 0.1.2
sjsepan, sjsepan-oceanish, 0.1.0, 0.0.4
sjsepan, sjsepan-royalish, 0.1.0, 0.0.4
sjsepan, sjsepan-ghostish, 0.2.1, 0.1.3
sjsepan, sjsepan-humanedark, 0.2.0, 0.1.2
sjsepan, sjsepan-zeddish, 0.1.0, 0.0.1
sjsepan, sjsepan-forestish, 0.2.0, 0.1.1
sjsepan, sjsepan-pippish, 0.1.0, 0.0.2
sjsepan, sjsepan-tropicalish, 0.1.0, 0.0.1
sjsepan, sjsepan-summerish, 0.1.0, 0.0.3
sjsepan, sjsepan-solarish, 0.1.0, 0.0.3
sjsepan, sjsepan-nightingaleish, 0.1.0, 0.0.5
sjsepan, sjsepan-coffeeish, 0.1.0, 0.0.4
sjsepan, sjsepan-bergyish, 0.1.0, 0.0.4
sjsepan, sjsepan-nevadaish, 0.0.3, 0.0.2

[kineticsquid]

@sjsepan3 I can see you have other extensions that seem to be publishing successfully. Sometimes, though not always, extensions fail to activate because of an error in the package contents (admittedly we don't do a good job of reporting on this). I'd investigate but the repos above appear to be private.

[sjsepan3]

@sjsepan3 I can see you have other extensions that seem to be publishing successfully. Sometimes, though not always, extensions fail to activate because of an error in the package contents (admittedly we don't do a good job of reporting on this). I'd investigate but the repos above appear to be private.

The GitLab links were a long-standing error that was among the things I was updating with this latest round of publishing.
An example of a corrected one is:

https://gitlab.com/sjsepan/sjsepan-nevadaish

Steve

[kineticsquid]

@sjsepan3 I cloned your repo and attempted to build the extension. I get this error message:

 ERROR  The specified icon 'extension/./images/icon.png' wasn't found in the extension.

I then changed this line in your package.json file:

	"icon": "./images/icon.png",

to:

	"icon": "images/icon.png",

Packaged it again and published and it activated.

[sjsepan3]

@sjsepan3 I cloned your repo and attempted to build the extension. I get this error message:

 ERROR  The specified icon 'extension/./images/icon.png' wasn't found in the extension.

I then changed this line in your package.json file:

	"icon": "./images/icon.png",

to:

	"icon": "images/icon.png",

Packaged it again and published and it activated.
@kineticsquid
Thank You for tracking that down.
Now that brings up some questions/observations...

-the path ought to evaluate to the same thing, but I can simplify it (Done, w/ Nevadaish)
-however, I wonder if I used that format because the simpler form was not working (Nope, No issues)
-now that i think about it, although I am used to working on Linux, I wonder if forward-slash would work on Windows. I have not had trouble installing the extensions in WIndows (Re-checked, OK)
-with that change, I'll have to re-upload to the MS Mkt and see if it squawks (Nope, OK)
-since that one has uploaded successfully in the past, what caused it to fail this time, as that is not something I changed this time, and these had been out there for months. (I can see that my non-theme extensions do not have './' in that setting, but probably all of the themes were out there at one time with that value.)
-finally, how does one determine these kind of errors during the publish process? (OK, I think I know that one -- I remember early on playing with extensions that there was some sort of build/pkg tool, which I think I gave up on figuring out, and did my own layout and bash script to zip it.)

[sjsepan3]

@kineticsquid
I just made that change to a local copy v0.0.4 (not pushed to GitLab yet), and it publishes, but is not active; I see v0.0.2 active.

Update: I cleared the page cache and refreshed the page, but still the same
Update: v0.0.4 of Nevadaish is on GitLab

[kineticsquid]

@sjsepan3 Using my admin super powers I removed v 0.0.4. I then added myself to your namespace as a contributor and published the 0.0.4 I packaged. It activated. I then removed myself from your namespace. I'm thinking this must be something with your packaging process. I just used vsce package from the root directory. It was in packaging this way that I say the error I listed above.

Are you saying that some of these extension versions that were active because deactivated? Or that you published new versions, without changing anything and the newer versions didn't activate?

[sjsepan3]

@sjsepan3 Using my admin super powers I removed v 0.0.4. I then added myself to your namespace as a contributor and published the 0.0.4 I packaged. It activated. I then removed myself from your namespace. I'm thinking this must be something with your packaging process. I just used vsce package from the root directory. It was in packaging this way that I say the error I listed above.

I will have to re-investigate 'vsce package', although the "root" where I run it will likely be the pkg folder, because that is the content that goes into a .vsix.

Are you saying that some of these extension versions that were active because deactivated? Or that you published new versions, without changing anything and the newer versions didn't activate?

The latter, except I did change the .json file in several and the .vsixmanifest file in all of them.

[sjsepan3]

@kineticsquid
I was able to get vsce ls --tree and vsce package to run, only when inside the extension folder (the level below the pkg folder where the manifest resides) and the .vsix generated both publishes and activates.

You can consider this closed. I will go through my other projects later...

I have opened up the generated .vsix (it is jsut a .zip with a folder structure and content like my extension folder) and I'm looking for differences -- because I want to know why my bash-generated packages worked for so long and suddenly stopped working here. So far the only differences (according to Meld) are in what goes into the content types and the manifest. The former omitted the xml mime (<Default Extension=".xml" ContentType="text/xml"/>), and the latter encoded ', omitted my PreviewImage key, and added some additional keys (Microsoft.VisualStudio.Code.EnabledApiProposals,
Microsoft.VisualStudio.Services.Links.Getstarted,
Microsoft.VisualStudio.Services.Links.Repository,
Microsoft.VisualStudio.Services.Links.Learn).

[sjsepan3]

@kineticsquid
Some additional tests...and I should note that running vsce package does not show any errors with the content:

• I made changes to content types and manifest based on what vsce generated, and since that content worked with vsce packaging, I packaged that content with bash script zip, but although it publishes, it does not activate
• I took a zip file generated by vsce, scooped out the contents, put in the same content that I had zipped myself, but although it publishes, it does not activate. (However, this may not be a good test, as the zip mechanism is different than that done by vsce.)
• I took the content out of a vsce-generated pkg and zipped that myself, but although it publishes, it does not activate

So far it is looking like it is not the content, but the archive (zip -r).
Why am I doing this? Let's say an equal mix of stubbornness and wanting to know why it stopped working (here). ;-) (However, I'm moving on for now so I can get the others corrected and published.)

[amvanbaren]

Hi @sjsepan3,
Yes, it's the archive. It's being flagged as potentially malicious.
Here's an excerpt from the logs:

2024-09-19T12:40:55.070Z  WARN [openvsx-server,,] 1 --- [openvsx-server] [task-35] o.e.o.p.PublishExtensionVersionHandler   : Extension version is potentially malicious: sjsepan.sjsepan-pippish 0.1.7
2024-09-19T12:32:44.111Z  WARN [openvsx-server,,] 1 --- [openvsx-server] [task-34] o.e.o.p.PublishExtensionVersionHandler   : Extension version is potentially malicious: sjsepan.sjsepan-pippish 0.1.6
2024-09-19T12:13:04.235Z  WARN [openvsx-server,,] 1 --- [openvsx-server] [task-72] o.e.o.p.PublishExtensionVersionHandler   : Extension version is potentially malicious: sjsepan.sjsepan-pippish 0.1.5
2024-09-19T12:03:19.040Z  WARN [openvsx-server,,] 1 --- [openvsx-server] [task-33] o.e.o.p.PublishExtensionVersionHandler   : Extension version is potentially malicious: sjsepan.sjsepan-pippish 0.1.4
2024-09-19T01:49:01.131Z  WARN [openvsx-server,,] 1 --- [openvsx-server] [task-2]  o.e.o.p.PublishExtensionVersionHandler   : Extension version is potentially malicious: sjsepan.sjsepan-pippish 0.1.2

Please use ovsx or vsce to package your extensions.

[sjsepan3]

Hi @sjsepan3, Yes, it's the archive. It's being flagged as potentially malicious. Here's an excerpt from the logs:

2024-09-19T12:40:55.070Z  WARN [openvsx-server,,] 1 --- [openvsx-server] [task-35] o.e.o.p.PublishExtensionVersionHandler   : Extension version is potentially malicious: sjsepan.sjsepan-pippish 0.1.7
2024-09-19T12:32:44.111Z  WARN [openvsx-server,,] 1 --- [openvsx-server] [task-34] o.e.o.p.PublishExtensionVersionHandler   : Extension version is potentially malicious: sjsepan.sjsepan-pippish 0.1.6
2024-09-19T12:13:04.235Z  WARN [openvsx-server,,] 1 --- [openvsx-server] [task-72] o.e.o.p.PublishExtensionVersionHandler   : Extension version is potentially malicious: sjsepan.sjsepan-pippish 0.1.5
2024-09-19T12:03:19.040Z  WARN [openvsx-server,,] 1 --- [openvsx-server] [task-33] o.e.o.p.PublishExtensionVersionHandler   : Extension version is potentially malicious: sjsepan.sjsepan-pippish 0.1.4
2024-09-19T01:49:01.131Z  WARN [openvsx-server,,] 1 --- [openvsx-server] [task-2]  o.e.o.p.PublishExtensionVersionHandler   : Extension version is potentially malicious: sjsepan.sjsepan-pippish 0.1.2

Please use ovsx or vsce to package your extensions.

Well that's a detail that we didn't see before. I don't like magic -- its just a zip file. The layout I used inside is the same, and the content is the same. Is there some sort of hidden signing going on?
I don't mean to sound contentious, but this used to work, without complaints, and still does on the MS Mkt.

[amvanbaren]

You can do some interesting things with a ZIP archive's extra field: https://gitlab.eclipse.org/security/vulnerability-reports/-/issues/213

[sjsepan3]

You can do some interesting things with a ZIP archive's extra field: https://gitlab.eclipse.org/security/vulnerability-reports/-/issues/213

I see, Thank You. Good enough reason to alter my workflow slightly. VSCE it is.

Please use ovsx or vsce to package your extensions.

BTW, I don't see a package option in the OVSX tool help. I've started using vsce, but how do I do that in the ovsx tool?

[amvanbaren]

I've started using vsce, but how do I do that in the ovsx tool?

ovsx uses vsce for packaging.
If you only publish to open-vsx.org you can run ovsx publish --packagePath <path>. It packages the extension and then publishes it.

If you also publish to the MS marketplace, then just package (and publish) the extension using vsce and use ovsx publish <packagePath> to publish to open-vsx.org.

[sjsepan3]

I've started using vsce, but how do I do that in the ovsx tool?

ovsx uses vsce for packaging. If you only publish to open-vsx.org you can run ovsx publish --packagePath <path>. It packages the extension and then publishes it.

If you also publish to the MS marketplace, then just package (and publish) the extension using vsce and use ovsx publish <packagePath> to publish to open-vsx.org.

OK, nothing for me to do differently with that, then. Thank You @amvanbaren !
And Thank You @kineticsquid for your assistance too.

[kineticsquid]

@sjsepan3 No worries, glad we got to the bottom of this.