Render patterns in volume segments to increase distinguishability

[philippotto]

• adds patterns to volume segments with varying angles, pattern shapes (stripes vs. grid), frequencies
• changes the color generation code to use a more predictable approach
• uses the jet color scheme for the segments

URL of deployed dev instance (used for testing):

• https://segmentationpatterns.webknossos.xyz

Steps to test:

• open a volume tracing or dataset with segmentations and inspect the patterns
• play with the "pattern opacity" setting in the segmentation layer settings
• zoom in and out and ensure that the frequency of the stripes changes to pleasant values when the mag is switched
• check that the color in the "segmentation" tab matches the rendered color

Issues:

• fixes Improve distinguishability for volume segments #4240

---

• Updated (unreleased) changelog
• Updated (unreleased) migration guide if applicable
• Updated documentation if applicable
• Adapted wk-connect if datastore API changes
• Needs datastore update after deployment
• Ready for review

[philippotto]

@daniel-wer I think, you can delay the review until we decide that the feature itself is really ready for merging. The feedback until now wasn't that enthusiastic.

[philippotto]

@daniel-wer Ok, I think this is ready-for-review-ish. I still need to adapt the table in the "segmentation tab" so that the new color generation is used there, too. But I think the rest is ready to review and test.

Thanks to @valentin-pinkau's suggestion, I also revised the pseudo-random generation of pattern features to use primitive roots which gives us an elegant approach without collisions within these sequences. I added a detailed explanation to the source code and hope this makes sense :)

[daniel-wer]

Absolutely great work! I can only imagine how hard it was to write and debug all of this shader code, but it works very well!

I reviewed most of the code and added some questions and suggestions. I didn't review the part involving the scaling of the world coordinates in detail (lines 107+ in segmentation.glsl) as my brain shut off trying. I'm not sure whether it's needed, though as it works very well 👍

I would propose to add a general description of what kinds of patterns, angles and colors we're using now - a little high-level description of what's going on.

I'm not sure whether it's only me, but I find it very hard to distinguish most of the patterns with the default pattern opacity of 40 (in datasets with a full segmentation layer). A value like 70 works a lot better for me. Not a big deal, as it's configurable, but I wanted to throw this suggestion in here :)

[daniel-wer]

One more thing: The isosurface color seems to still be the old color as well. I don't think the isosurfaces need the patterns, but the color should be equal :)

[philippotto]

I reviewed most of the code and added some questions and suggestions. I didn't review the part involving the scaling of the world coordinates in detail (lines 107+ in segmentation.glsl) as my brain shut off trying. I'm not sure whether it's needed, though as it works very well +1

I added some explaining comments there and hope it's easier to grasp now :)

I would propose to add a general description of what kinds of patterns, angles and colors we're using now - a little high-level description of what's going on.

Good idea! Added that now.

I'm not sure whether it's only me, but I find it very hard to distinguish most of the patterns with the default pattern opacity of 40 (in datasets with a full segmentation layer). A value like 70 works a lot better for me. Not a big deal, as it's configurable, but I wanted to throw this suggestion in here :)

Yeah, I noticed that the ideal segmentation pattern opacity value highly depends on the actual segmentation opacity value (so, 40 works well for me when the volume opacity is higher). This is not really ideal, but I couldn't come up with a quick solution which improves this. In the end, most users should fine tune this, anyway, I think.

One more thing: The isosurface color seems to still be the old color as well. I don't think the isosurfaces need the patterns, but the color should be equal :)

Yeah, that was the last missing piece (also affected the segmentation tab table). It was a bit tricky to get right, since the color calculation suffers from imprecisions. In the end, I ensured that the JS implementation suffers from the same imprecisions 🙈 Since it's consistent, it seems fine to me. I also changed the color count so that the used primitive root is 2 instead of 3, since this yields better precision.

@MichaelBuessemeyer Since @daniel-wer is on vacation, could you give this PR a final pass (test it and also look over the code)? Thank you!

[MichaelBuessemeyer]

Thanks for this awesome PR. It really improves the distinguishability of the segments 🤓

I let some comments about a few parts I do not understand and I would place a few methods in another file.

The merge this when the questions and comments are resolved 😄

[philippotto]

@MichaelBuessemeyer Thanks for your feedback! I just incorporated it. Could you have another look?

[MichaelBuessemeyer]

I got one last thing: I mentioned to maybe move the none-shader implementation of getElementOfPermutation , colormapJet and rgb2hsv to another js file. What do you think about this?

After this is resolved, let's merge it 🚢

[MichaelBuessemeyer]

Alright, the last thing that might need to be done is updating the screenshots 🖼️. After that let's merge this awesomeness 🕺