hosting nwbwidgets

[edublancas]

hi @bendichter,

as discussed. I deployed nwbwidgets in Ploomber Cloud, here's the live app: https://ploomberapp.io/dry-field-0371 (provisional URL)

A few pending items:

resources

when running some examples, I noticed that some of the nwb files are quite large. Looks like nwbwidgets is streaming them and that alleviates how much storage is required.

do you have a ballpark estimate of how much storage is ok? also, are files removed automatically? (otherwise, we'll have to come up with a mechanism to clean up the disk)

links to specific files

we also discussed having URLs to specific files. This is currently implemented in the DANDI Archive, where we can click on the three dots and have a direct link to visualize the dataset:

We can add a similar thing. Example: https://ploomberapp.io/dry-field-0371?dataset=path/to/dataset/in/s3

I found out that voila already supports this (example) but I couldn't find a way to pass this to nwbwidgets.

Is there a way to do something like this:

from nwbwidgets.panel import Panel

Panel(path_to_s3='path/to/s3')

This way, once the voila app begins, it'd automatically display the selected dataset.

---

anything else I missed or you want for this deployment?

[bendichter]

Eduardo,

resources

If the cache is enabled, it could get quite large. It might be better to
simply develop kwargs of the Panel to disable this option. It would also
make sense to disable the local file option, since you won't have access to
any NWB files locally in this app. @luiztauffer, can you help with this?

links to specific files

One option would be to launch the Panel widget and
then manually set values of the dropbox selectors. The advantage of this approach is that a user could easily navigate the data on DANDI on their own once the start the panel.

Another option is, as you suggest, to run NWB Widgets on a specific s3
path. You can do this with

import fsspec
import h5py
import pynwb
from nwbwidgets import nwb2widget


fs = fsspec.filesystem("http")

# open the file
f = fs.open(s3_url, "rb")
file = h5py.File(f)
io = pynwb.NWBHDF5IO(file=file, load_namespaces=True)
nwbfile = io.read()

nwb2widget(nwbfile)

[edublancas]

@bendichter, thanks for the feedback. it sounds like someone on your team can work on making the changes to Panel, right?

On our end, I noticed that the app I initially deployed crashed because it ran out of disk space, so I'll check our configuration and ensure that there's reasonable disk space so this doesn't happen. Would a couple of GB be enough?

I think the combination of the changes you're proposing to Panel + having some minimum disk size should be enough to get this up and running!

Of course, we can make changes down the road as we see fit, but sounds like this would be enough for the first version.

[luiztauffer]

hi @edublancas
If you're going to run the widgets only for a pre-defined NWB file, I suggest you do as Ben showed, no need for Panel.
Panel is useful if you want to give your users the possibility to choose form any dandiset directly from the widgets UI, or to load from another S3 path.

Regarding the caching problem, once NeurodataWithoutBorders/nwbwidgets#316 gets merged, you'll be able to disable it, so storage errors should stop happening. For that, you'd like to run Panel like this:

from nwbwidgets.panel import Panel

Panel(
    enable_local_source=False,
    enable_cache=False
)

[edublancas]

alright, I updated the deployment: https://autumn-dew-6622.ploomberapp.io/

I followed @bendichter's snippet, you can now pass a url argument:

https://autumn-dew-6622.ploomberapp.io/?url=https://dandiarchive.s3.amazonaws.com/blobs/5b6/477/5b647726-95f6-4aef-88fc-3a54d33301c3

source code is here: https://github.com/ploomber/doc/blob/main/examples/deployments/nwbwidgets/app.ipynb

I guess if we use nwb2widget, we won't have the caching problem?

is the plan to include this as an option in the dandiarchive menu?

[luiztauffer]

you can now pass a url argument:

that's a neat solution!

[edublancas]

@bendichter any feedback or next steps?

[edublancas]

closing this due to inactivity - I'll delete the deployed app but feel free to create a free account on our platform and deploy it again (source code)

[edublancas]

and if you have any questions, feel free to reach out!