Search for plugins to install on demand

[goanpeca]

Fixes #83
Fixes #76

As discussed in #37 as more and more plugins are added to the ecosystem loading a big list of plugins might become unsustainable, specially if we keep using complex custom and styles widgets within the Qlistwidgetitems.

Another approach to this, that could still preserve the rich styling of widgets is to only display the items after a search, that way we only populate the list on demand if a user is looking for something. This is what other tools like VSCode handle it. Added a small preview, of course this would require some styling, but this work fits nicely into what has already been provided by the latest PRs

[psobolewskiPhD]

I like this @goanpeca
I demo'd napari at SciPy and people liked scrolling the plugins, but this interface is snappy and nice.
One comment: the filter at the top doesn't filter installed plugins:

So maybe just move the line edit below?
It's not a filter anymore, just have Search for plugins as a label?

[goanpeca]

I demo'd napari at SciPy and people liked scrolling the plugins, but this interface is snappy and nice.

Glad you like it, indeed is snappy :)

One comment: the filter at the top doesn't filter installed plugins:

Yes!, this was a quick POC to see how it could work hence all the UI issues remain like this one you correctly point out.

If this is something we would like to pursue, I would need to merge the other remaining PRs and then continue here, although I would probably just do a full rebase from main and add the few changes (those are actually small now that the other PRs ara almost in)

Also @Czaki suggested we could have on the bottom a "suggested" plugins or, most downloaded or something, so that it is not empty empty, but showcases the most installed ones.

Anyway food for thought :)

Thanks for the reviews!

[codecov]

Codecov Report

Attention: Patch coverage is 96.39640% with 4 lines in your changes missing coverage. Please review.

Project coverage is 89.54%. Comparing base (dbcd362) to head (0eb2eee).
Report is 3 commits behind head on main.

Files with missing lines	Patch %	Lines
napari_plugin_manager/qt_plugin_dialog.py	92.85%	4 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main      #51      +/-   ##
==========================================
- Coverage   93.54%   89.54%   -4.00%     
==========================================
  Files          10       10              
  Lines        1813     1856      +43     
==========================================
- Hits         1696     1662      -34     
- Misses        117      194      +77     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[psobolewskiPhD]

Meant to post this here:
We discussed this in the Pacific meeting See notes here:
https://hackmd.io/VSaKh4hMQ12Bn7uWD_ij6A

The gist was that the current approach is likely untenable in the long term, but the merged-to-main fixes are a significant improvement, so lets look at a release?

Then, approach here can be a followup, with some extra UI polish prior to being released, such as additional filter categories--assuming they are available anyways.

[goanpeca]

@psobolewskiPhD haha yes I understood :)

All good making a release today 🚀

[goanpeca]

Ok, back to work here :)

@dalthviz @psobolewskiPhD do you think you could give it a try to see how it feels, works?

Thanks!

[dalthviz]

Did a quick check and seems like things are working as displayed over the OP video 👍 However, a couple of things I noticed (maybe the last one not completly related with the work here):

• When having something typed over the search lineedit if you click Refresh the list with plugins available to be installed gets cleared. After the refresh is finished, however, the search text is not reapplied to repopulate the list. Maybe to prevent that kind of inconsistency (having some search text but not showing results even when there are results that could be displayed) when you press Refresh the search text should be cleared?
• Seems like I got a couple of warning messages related with network connection issues when pressing Refresh?: WARNING: Plugin manager: There seems to be an issue with network connectivity. Remote plugins cannot be installed, only local ones. (and the number of plugins available being shown is set to 0). However, seems like pressing again Refresh a couple of times loads things normally (the number of available plugins goes again to be 457). Maybe that is related with my internet connection or maybe the server from where the info related to plugins gets retrieved 🤔. Anyhow, just sharing that here in case creating an issue for that is worthy.

A preview of the behaviors described above:

[dalthviz]

Hi there, with the little change pushed above. now things here work something like:

As seen over the preview, I'm still experiencing the issue with the connectivity. Still not sure if that's something on my end or over the web service that provides the plugins info to be honest 🤔

Anyhow, let me know if the change pushed above makes sense @jaimergp @goanpeca

[jaimergp]

Still not sure if that's something on my end or over the web service that provides the plugins info to be honest

I doubt Vercel is having issues, and I don't see anything in https://www.vercel-status.com/

[jaimergp]

@psobolewskiPhD @Czaki any further comments?

[psobolewskiPhD]

If i type something into the filter and click Refresh, I get the connectivity error that I can't really read or expand. Re-open/close the manager doesn't solve it, and it persists when I relaunch napari, but after a few min and another relaunch it works.

Edit:
I'm thinking about the UX "shock therapy" here, going from the old way where the bottom box would start populating whether anything was typed or not.
It's not so obvious you should type something vs. previously waiting? So it may feel like it's not working?
Maybe we need a label at the top, above the search box, something like:
Type below to begin searching for plugins on the napari hub:

[dalthviz]

Just in case, regarding the connectivity issue, I was able to reproduce it using the main branch so I think is not completly related with this (although maybe is more easily triggered with this PR for some reason?).

Also, checking what is being catch to show the connectivity issue message, the exception says:

HTTP Error 500: Internal Server Error

Further checking with my browser on an incognito window (Firefox), the first time I try to get something from the web service (for example https://npe2api.vercel.app/api/extended_summary) I get a HTTP 500 error:

🤔

[dalthviz]

I'm thinking about the UX "shock therapy" here, going from the old way where the bottom box would start populating whether anything was typed or not.
It's not so obvious you should type something vs. previously waiting? So it may feel like it's not working?
Maybe we need a label at the top, above the search box, something like:
Type below to begin searching for plugins on the napari hub:

Maybe changing the search lineedit placeholder text could help with that too? So something like:

[dalthviz]

Last commit makes the search widget look something like the following (with a new placeholder text and be able to show a tooltip with extra info):

[dalthviz]

Seems like the latest CI fail was caused due to a HTTP 500 error triggered while testing the np2api endpoints (particulary related with the https://npe2api.vercel.app/api/conda endpoint): https://github.com/napari/napari-plugin-manager/actions/runs/11840208794/job/32993483316?pr=51#step:7:2371 Launched a job re-run and the test now passes: https://github.com/napari/napari-plugin-manager/actions/runs/11840208794/job/32995746148?pr=51 🤔

[jaimergp]

If you like it, I like it. LGTM! I'll let someone from @napari/core-devs do the honors with the merge, though.

[jaimergp]

Btw @dalthviz, does this PR also address #76 by any chance?

[dalthviz]

Btw @dalthviz, does this PR also address #76 by any chance?

I think so 👍

Just in case, this what a got when doing a search while the available plugins for installation are being queried (in this case by triggering a refresh):

[jni]

@psobolewskiPhD do you want to do the honours here? (I have little experience with the plugin manager, to put it mildly.)

[goanpeca]

Hi @dalthviz thanbks for updating the PR. Indeed I also experienced the 500 errors, it seems the endpoint was crashing more and more, and this PR for some reason I do not fully comprehend, was making the error more apparent. The refresh button can solve this but in some cases the plugin list does not populate because of this. I am not sure if the endpoint is doing some blocking in which case we could/should evaluate a different provider? or see if vercel needs to have some rules added with respect to "known requesters".

That is why I added the flaky test checker :)

PR looks good!

[jaimergp]

I'll start a 48h timer. If we don't hear back in that time we'll do the merge by Thursday EOD. Thanks!

[psobolewskiPhD]

Sorry for the delay, I like it!
Thanks y'all!