Connectome Tab

[daniel-wer]

This PR implements the internal work package D.1 Connectome Viewing in webKnossos (see internal communication). Refer to the work package for the full feature set and a mockup.

This PR adds a new "Connectome Tab" and I tried to encapsulate the new functionality fairly well so that most of it is contained in a couple of files. Still, I had to adapt the code in a couple of spots to support some new use cases which I'll describe in the following:

• Allow to set the enableRenderOnDemand prop for layout tabs. Setting the prop to false will disable the lazy-loading of the tab and instead always render it. For the connectome tab, this is important to allow hotlinking to agglomerates/synapses without the tab being visible when loading the annotation. Otherwise, the code encapsulation would not work as well. I don't think this is a problem, though, since the tab doesn't do much if it's not actively being used.
• Allow to add multiple Skeletons (skeleton.js) to the scene. This is used to separate the connectome skeleton data from the "normal" skeleton data so the two don't interfere with each other and the connectome skeleton data is effectively read-only. Also allow to remove a Skeleton from the scene. This is done during cleanup or when the segmentation layer is switched.
• Introduce the lastVisibleSegmentationLayerName to allow to continue to use the connectome tab even if the segmentation was temporarily disabled. This might be useful in other spots as well.

ToDo:

• Separate connectome_view.js into separate logical units
• Adapt docs/sharing.md#sharing-link-format to new link format
• Check all check boxes that represent the same segment id (not only the clicked one). No idea what would be a good way to do that yet
• Show the segment color next to the entry in the tree (if the agglomerate skeleton is loaded) Wait for feedback, maybe followup
• The connectome tab should work in view mode
• Do we need a layout migration or is it sufficient to increase the layout version number?
• Hovering over a Segment entry in the tab could highlight the segment in the data viewport. The same feature exists for the segment tab, so it might be easy to copy.
• The context menu in the data viewport could have an entry for loading a segment into the connectome tab
• If an agglomerate could not be loaded, because the node limit is exceeded, a nicer error message could be rendered. Currently, one has to expand the details of the error message to find out that this is not a bug.

URL of deployed dev instance (used for testing):

• https://connectomeviewing.webknossos.xyz

Steps to test:

• Open a dataset without a connectome. The connectome tab should state there is no connectome.
• Use the l4dense_motta_et_al_dev dataset that has a linked connectome file on the deployed dev instance
• The connectome tab should work in view mode as well as in annotations
• Right-Click on a segment in the viewport and select "Import Agglomerate and Synapses". The segment should be shown in the connectome tab, its agglomerate skeleton loaded and all synapses shown as single node trees as well.
  • The same should happen if you enter the segment ID by hand
• Entering two segment IDs (comma-separated) should show both skeletons and only the synapses between the two agglomerates
• It should be possible to toggle the segmentation layer while a segment is shown in the connectome tab without the info in the connectome tab to disappear
• For datasets with multiple segmentation layers, if the segmentation layer is switched the tab should be reset
• The filters by synapse direction or synapse type should work and hide the filtered synapses/agglomerates from the tree list as well as the viewports
• Right-click on a 2nd-level segment in the tree list and select "Show All Synapses of This Segment" to see the synapses of that segment, instead
• The "Reset" buttons to reset the filters as well as the connectome tab should work
• It should be possible to link to a specific agglomerate and its synapses (tested in the screenshot tests), for example, this link

Issues:

• fixes Connectome Viewing in webKnossos #5828

---

• Updated (unreleased) changelog
• Updated documentation if applicable
• Adapted wk-connect if datastore API changes
• Needs datastore update after deployment
• Ready for review

[daniel-wer]

What does the frontend do if a wkconnect dataset is opened, where the new routes do not exist? Should we add the list connectomes route there (always returning empty array)?

Yes, good call, that's the only route that should be needed 👍

[fm3]

I created the PR scalableminds/webknossos-connect#127 to restore compatibility. That one should be probably merged first.
During testing I noticed that this branch is quite out of date with the master, you may want to reserve some time to check if everything still works after master merge :)

[daniel-wer]

During testing I noticed that this branch is quite out of date with the master, you may want to reserve some time to check if everything still works after master merge :)

Do you mean this connectome-viewing branch? If so, I merged the current master, yesterday, so it shouldn't be that out of date 🤔 Still, we'll definitely test the new functionality before merging 👍

[fm3]

You are of course right, I just did not realize I did not pull before testing.

[philippotto]

• Currently the connectome tab is always visible. Since it is kind of a niche functionality, should it maybe only be visible for specific organizations (that want to test this functionality and/or we know to be in the connectomics field)? And if so, does that have any implications for the viewport configurations (if the tab is sometimes there and sometimes not)?

Yeah, I think, it would currently be best if the tab could be enabled/disabled via a feature flag so that we can control it per organization. Otherwise, the tab might cause a lot of irritation, as no one will know how to use it. I think, it should "just work" ™️ to adapt default_layout_configs.js so that it uses the features. Hopefully, this will work. The layout version needs to be bumped then, too, I assume..

[philippotto]

Code looks great :) Didn't test, yet, though, since there are some empty checkboxes still.

[daniel-wer]

Yeah, I think, it would currently be best if the tab could be enabled/disabled via a feature flag so that we can control it per organization. Otherwise, the tab might cause a lot of irritation, as no one will know how to use it. I think, it should "just work" tm to adapt default_layout_configs.js so that it uses the features. Hopefully, this will work. The layout version needs to be bumped then, too, I assume.

Hmm, yeah, I think this method would work, but I really dislike that each time another organization is added, (one needs to remember to increase the layout version and) all layout configs become invalidated :/
What I could try to write would be a kind of evolution that runs after loading the persisted layout (or loading the default one) and adds or removes the Connectome Tab, depending on the feature flag 🤔 This would also allow us to add new tabs in the future without having to reset all layout configs for all users.

[philippotto]

Yeah, I think, it would currently be best if the tab could be enabled/disabled via a feature flag so that we can control it per organization. Otherwise, the tab might cause a lot of irritation, as no one will know how to use it. I think, it should "just work" tm to adapt default_layout_configs.js so that it uses the features. Hopefully, this will work. The layout version needs to be bumped then, too, I assume.

Hmm, yeah, I think this method would work, but I really dislike that each time another organization is added, (one needs to remember to increase the layout version and) all layout configs become invalidated :/ What I could try to write would be a kind of evolution that runs after loading the persisted layout (or loading the default one) and adds or removes the Connectome Tab, depending on the feature flag thinking This would also allow us to add new tabs in the future without having to reset all layout configs for all users.

You are totally right. Dynamically adding/removing the tab would be best probably.

[daniel-wer]

Right-Click on a segment in the viewport and select "Import Agglomerate and Synapses".

I confirmed that this works now, even if a mapping is activated on-the-fly, with my latest commit :)