Add additional coordinates to segment index

[frcroth]

Steps to test:

• Open nd dataset and create some segments
• Open segment context menu in viewport and see BB and volume info
• In the segments tab, click on a group and see the segment statistics. Also test CSV export.
• Make sure everything is working for 3D datasets, such as export
• Make sure ad-hoc mesh rendering and its download is still working

TODOs:

• Frontend
  • Supply additional coordinates on segment statistics routes
  • Use the new POST /volume/:tracingId/segmentIndex/:segmentId route (instead of GET), which also supports additional coordinates
  • take care of and remove TODO comments in frontend/javascripts/oxalis/view/context_menu.tsx, ll. 1166 and 1308
  • fix l. 1098 in https://github.com/scalableminds/webknossos/pull/7394/files/e971675c54c92a55e8c37de12835c1efa4435b76#r1404463304
  • remove condition in segment_mesh_saga > loadAdHocMesh: usePositionsFromSegmentStats
  • test mesh download (currently blocked by local CSP)

Issues:

• fixes Segment Index + Stats for ND volume annotations #7276

---

(Please delete unneeded items, merge only when none are left open)

• Updated changelog
• Removed dev-only changes like prints and application.conf edits
• Considered common edge cases
• Needs datastore update after deployment

[fm3]

@frcroth do you know what happens if I created an ND volume annotation on master, brush some, and then upgrade to this change? Will everything explode or will this be handled somehow? It might be too much to ask to add another migration for this case (there aren’t many ND annotations in production yet), but if things get into inconsistent states, maybe we should just disable the segment index features for such annotations?

[frcroth]

@frcroth do you know what happens if I created an ND volume annotation on master, brush some, and then upgrade to this change? Will everything explode or will this be handled somehow? It might be too much to ask to add another migration for this case (there aren’t many ND annotations in production yet), but if things get into inconsistent states, maybe we should just disable the segment index features for such annotations?

I validated that the annotations can still be viewed, but the segment index won't work anymore. Also I noticed that the front end does not provide additional coordinates for segment volume calculations leading to unhandled exceptions whenever one right clicks segments in the canvas.

[knollengewaechs]

I seem to be blocked, see here: https://scm.slack.com/archives/C5AKLAV0B/p1703082065289029?thread_ts=1702477845.669909&cid=C5AKLAV0B

[philippotto]

Getting there :)

[fm3]

Good stuff! I added a comment about the json format.

Also, do you know what happens with nd volume annotations created before this PR? Do they have a broken segment index? Or don’t they have one? Do we need a migration for them, similar to the original #7376 ?

[philippotto]

front-end looks good to me :) testing also went well.

[frcroth]

Also, do you know what happens with nd volume annotations created before this PR? Do they have a broken segment index? Or don’t they have one? Do we need a migration for them, similar to the original #7376 ?

Well the segment index previously was invalid since all additional coordinates were mapped to the same bucket key. With this PR the previous segment index can't be used since the keys have changed. So I think if there are nd annotations with segment indices, it would be best to recalculate the segment index.

Maybe we need another route for that, since addSegmentIndex doesn't do anything if the segmentIndex already exists.

[fm3]

As discussed in person, we found that no productively used volume annotations on nd datasets exist yet. So let’s add an sql migration archiving (state='Finished') any (experimental) existing annotations on nd datasets.

[fm3]

🚀