Isosurface generation (now inluding frontend) #3495

jfrohnhofen · 2018-11-24T20:31:45Z

~~This PR only contains the backend code (new route) for generating iso-surfaces. The corresponding frontend changes can be found in #3493.~~ #3493 was now merged into this, so front-end and backend changes happen in this PR. Check #3493 for front-end discussion.

URL of deployed dev instance (used for testing):

https://isosurfacetesting.webknossos.xyz

Steps to test:

open dataset with segmentation
activate rendering of iso-surfaces in dataset settings
move around
be amazed by iso-surfaces in the 3D viewport

Issues:

fixes Live isosurface (backend) #3313
fixes Isosurface Frontend v1.0 #3440

Updated changelog (should be done as part of Isosurface frontend #3493)
Needs datastore update after deployment
Ready for review

* origin/master: Add debugging methods to visualize wireframe of buckets (#3402) fixed bug that selects a just removed layout (#3379) Added button to revoke admin rights in frontend (#3378) Revert "Add REST API versioning support (#3385)" (#3404) Add REST API versioning support (#3385) limit number of tasks to be created in one api request (#3386) Use correct volume download route in TracingStoreRpcClient (#3403) clearer error message when uploading nml for inaccessible dataset (#3390) Refactored dropdown item events (#3383) Show dataset extent in right menu tab (#3371)

* master: remove -v (verbose) from backend-tests in CI (#3425) Add recommended configuration to TaskTypes (backend only) (#3415) Even if initialData is disabled, insert local stores (#3421) Add REST API docs (#3376) clearer message when annotation couldnt be found while logged out (#3413) prepare release 18.11.0 (#3414) Handle Cross-Datastore Duplicates, introducing scratch datastores (#3394) restrict user visibility (#3389) Add REST API versioning support (second try) (#3408) Enable hybrid tracings by default (#3399)

* master: Fix rgb support (#3455) Fix docker uid/gid + binaryData permissions. Persist postgres db (#3428) Script to merge volume tracing into on-disk segmentation (#3431) Hotfix for editing TaskTypes (#3451) fix keyboardjs module (#3450) Fix guessed dataset boundingbox for non-zero-aligned datasets (#3437) voxeliterator now checks if the passed map has elements (#3405) integrate .importjs (#3436) Re-write logic for selecting zoom level and support non-uniform buckets per dimension (#3398) fixing selecting bug and improving style of layout dropdown (#3443) refresh screenshots (#3445) Reduce the free space between viewports in tracing (#3333) Scala linter and formatter (#3357) ignore reported datasets of non-existent organization (#3438) Only provide shortcut for tree search and not for comment search (#3407) Update Datastore+Tracingstore Standalone Deployment Templates (#3424) In yarn refresh-schema, also invalidate Tables.scala (#3430) Remove BaseDirService that watched binaryData symlinks (#3416) Ensure that resolutions array is dense (#3406) Fix bucket-collection related rendering bug (#3409)

* origin/master: Optimize performance for the list request /api/datasets (#3441) add annotation dataset foreign key (#3482) thumbnails: correctly use zoom value if specified (#3487) Store Meshes in Postgres (#3367) fix alpha return (#3483) Added script to apply all new evolutions (#3427) Simple fix to speed up dataset gallery (#3480) better errors for screenshot tests, fix imports, refresh screenshots (#3479) (Backend only) Add project priority to progress report json (#3476) Handle missing write access on datastore (#3411) Re-introduce "Flightmode improvements"" (#3473) Circleci-notify: linkify PR number (#3469) Revert "Flightmode improvements" (#3472) also flow-ignore binaryData when using symlinks (#3471) Flightmode improvements (#3392) Circleci custom notification (#3465) enable /api/switch cross-organization (#3464)

fm3 · 2018-12-03T15:19:50Z

...-datastore/app/com/scalableminds/webknossos/datastore/controllers/BinaryDataController.scala

+          } yield {
+            val responseBuffer = ByteBuffer.allocate(vertices.length * 4).order(ByteOrder.LITTLE_ENDIAN)
+            responseBuffer.asFloatBuffer().put(vertices)
+            Ok(responseBuffer.array())


please add a comment about the float array semantics (3 floats being a point etc)

fm3 · 2018-12-03T15:20:32Z

...-datastore/app/com/scalableminds/webknossos/datastore/controllers/BinaryDataController.scala

+            isosurfaceRequest = IsosurfaceRequest(dataSource, segmentationLayer, request.body.cuboid(dataLayer), request.body.segmentId, request.body.voxelDimensions, request.body.mapping)
+            vertices <- isosurfaceService.requestIsosurfaceViaActor(isosurfaceRequest)
+          } yield {
+            val responseBuffer = ByteBuffer.allocate(vertices.length * 4).order(ByteOrder.LITTLE_ENDIAN)


something like sizeof(float)?

did not find anything useful. but unlike C, floats are guaranteed to be 4 bytes. added a comment, though.

fine by me :)

fm3 · 2018-12-03T15:23:45Z

...atastore/app/com/scalableminds/webknossos/datastore/models/datasource/DataLayerMapping.scala

+      Whenever this data class is changed, the parser needs to be modified accordingly.
+*/
+
+trait AbstractDataLayerMapping


please add a comment about this empty trait being there to support putting these into a shared cache

fm3 · 2018-12-03T15:25:05Z

webknossos-datastore/app/com/scalableminds/webknossos/datastore/models/requests/Cuboid.scala

@@ -12,6 +12,12 @@ case class Cuboid(topLeft: VoxelPosition, width: Int, height: Int, depth: Int) {

  val volume: Int = width * height * depth

+  val hasValidDimensions: Boolean = width > 0 && width <= 512 && height > 0 && height <= 512 && depth > 0 && depth <= 512


also, pls add a comment about the intention (or name the magic number constant)

jfrohnhofen · 2018-12-03T15:27:20Z

...nossos-datastore/app/com/scalableminds/webknossos/datastore/services/BinaryDataService.scala

@@ -144,6 +141,8 @@ class BinaryDataService(dataBaseDir: Path, loadTimeout: FiniteDuration, maxCache
    compressed
  }

+  // private def applyMapping()


jfrohnhofen · 2018-12-03T15:38:50Z

...ossos-datastore/app/com/scalableminds/webknossos/datastore/services/DataServicesHolder.scala

@@ -8,14 +8,18 @@ import javax.inject.Inject
 /*
 * The BinaryDataService needs to be instantiated as singleton to provide a shared DataCubeCache.
 * The TracingStore one (for VolumeTracings) already is, since the surrounding VolumeTracingService is a singleton.
- * The DataStore one is singleton-ized via this holder.
+ * The DataStore is singleton-ized via this holder.


leave as is

jfrohnhofen · 2018-12-03T15:44:16Z

...nossos-datastore/app/com/scalableminds/webknossos/datastore/services/IsosurfaceService.scala

+  val binaryDataService: BinaryDataService = dataServicesHolder.binaryDataService
+  val mappingService: MappingService = dataServicesHolder.mappingService
+
+  implicit val timeout: Timeout = Timeout(30 seconds)


jfrohnhofen · 2018-12-03T15:44:25Z

...nossos-datastore/app/com/scalableminds/webknossos/datastore/services/IsosurfaceService.scala

+
+  implicit val timeout: Timeout = Timeout(30 seconds)
+
+  val actor = actorSystem.actorOf(RoundRobinPool(1).props(Props(new IsosurfaceActor(this, timeout.duration))))


jfrohnhofen · 2018-12-03T15:51:42Z

...nossos-datastore/app/com/scalableminds/webknossos/datastore/services/IsosurfaceService.scala

+      math.ceil(cuboid.height / voxelDimensions.y).toInt,
+      math.ceil(cuboid.depth / voxelDimensions.z).toInt)
+
+    val offset = Vector3D(cuboid.topLeft.globalX,cuboid.topLeft.globalY,cuboid.topLeft.globalZ) / Vector3D(cuboid.topLeft.resolution)


fm3 · 2018-12-03T16:15:08Z

util/src/main/scala/com/scalableminds/util/geometry/Vector3D.scala

@@ -40,6 +40,10 @@ case class Vector3D(x: Double = 0, y: Double = 0, z: Double = 0) {

  def *:(o: Double) = this.*(o)

+  def *(o: Vector3D) = Vector3D(x * o.x, y * o.y, z * o.z)


please add a comment warning that this is not scalar product nor vector product

fm3 · 2018-12-03T16:15:57Z

...ossos-datastore/app/com/scalableminds/webknossos/datastore/services/DataServicesHolder.scala

+  val mappingService = new MappingService(
+    Paths.get(config.Braingames.Binary.baseFolder),
+    config.Braingames.Binary.mappingCacheMaxSize)
+


I feel like the mapping service does not have to be in the holder, I’ll undo that and test if everything still works

* button in frontend * startup parameters * add mapping to isosurface request, #3313 * loading and parsing mappings, #3313 * build faster * use raw float array as isosurface protocol, #3313 * expose voxel dimensions as parameter, proper scale, #3313 * smooth shading, #3313 * improve lighting for isosurface * improve lighting and clean up code * fix merge conflicts * refactor isosurface front-end code and enable lookups according to bucket structure * adapt code to cubeSize vector * clean up * clean up * use mapped voxels if mapping is active; hide setting in non-view-mode

jfrohnhofen added 27 commits October 24, 2018 21:47

Isosurface: route and actor setup

7f4f200

isosurface wip

8f16f9e

removed debug print (#3313)

32f2dc8

add size parameter

c367096

Restructure isosurface actor interface (#3313)

ac40e8f

isosurfaces roughly working

cfa1469

support for v oxel dimensions, #3313

27ca052

loading and parsing mappings, #3313

9f71bef

parsing mappings, generics, #3313

87878ce

use raw float arrays as protocol for isosurfaces, #3313

079c08e

expose voxel dimensions as parameter, proper scale, #3313

ef7974c

wip, #3313

0793fd8

working on mappings, #3313

1ca3847

request via actor, #3313

4a5be05

mapping cache WIP, #3313

b317fc5

mapping cache working, #3313

c3f00f4

cubeSize is now vector, not scalar, #3313

043949d

Refactoring, #3313

63aee6a

support voxelDimensions in data requests, #3313

6840787

Fix data requests with nnon-trivial voxelDimension, #3313

f342087

fixing isosurfaces with nnew voxelDimensions data requests, #3313

5aaec42

isosurface refactoring, #3313

10527cf

skip locations where the segment id is not present, #3313

f688751

jfrohnhofen requested a review from fm3 November 24, 2018 20:39

jfrohnhofen added 2 commits November 24, 2018 21:39

Merge branch 'master' into isosurface

7f38389

Merge branch 'master' into isosurface

cf54cbe

fm3 reviewed Dec 3, 2018

View reviewed changes

jfrohnhofen commented Dec 3, 2018

View reviewed changes

fm3 reviewed Dec 3, 2018

View reviewed changes

jfrohnhofen and others added 4 commits December 3, 2018 18:16

PR feedback, #3313

f4a6860

Merge branch 'master' into isosurface

b67c2e5

move mappingservice out of holder

0c5b991

Merge branch 'master' into isosurface

19c2c80

fm3 approved these changes Dec 12, 2018

View reviewed changes

jfrohnhofen and others added 2 commits December 12, 2018 13:50

Merge branch 'master' into isosurface

9da457c

fm3 changed the title ~~Isosurface generation (backend)~~ Isosurface generation (now inluding frontend) Dec 12, 2018

fm3 added backend frontend labels Dec 12, 2018

fm3 assigned fm3, jfrohnhofen and philippotto Dec 12, 2018

update changelog

cdcdf9f

philippotto merged commit 0638d53 into master Dec 12, 2018

fm3 mentioned this pull request Dec 12, 2018

[DO NOT MERGE] code for isosurface testing #3439

Closed

normanrz deleted the isosurface branch February 20, 2019 16:37

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Isosurface generation (now inluding frontend) #3495

Isosurface generation (now inluding frontend) #3495

jfrohnhofen commented Nov 24, 2018 •

edited by fm3

Loading

fm3 Dec 3, 2018

fm3 Dec 3, 2018

jfrohnhofen Dec 3, 2018

fm3 Dec 3, 2018

fm3 Dec 3, 2018

fm3 Dec 3, 2018

jfrohnhofen Dec 3, 2018

jfrohnhofen Dec 3, 2018

jfrohnhofen Dec 3, 2018

jfrohnhofen Dec 3, 2018

jfrohnhofen Dec 3, 2018

fm3 Dec 3, 2018

fm3 Dec 3, 2018

		@@ -12,6 +12,12 @@ case class Cuboid(topLeft: VoxelPosition, width: Int, height: Int, depth: Int) {

		val volume: Int = width * height * depth

		val hasValidDimensions: Boolean = width > 0 && width <= 512 && height > 0 && height <= 512 && depth > 0 && depth <= 512


		implicit val timeout: Timeout = Timeout(30 seconds)

		val actor = actorSystem.actorOf(RoundRobinPool(1).props(Props(new IsosurfaceActor(this, timeout.duration))))

		@@ -40,6 +40,10 @@ case class Vector3D(x: Double = 0, y: Double = 0, z: Double = 0) {

		def :(o: Double) = this.(o)

		def (o: Vector3D) = Vector3D(x o.x, y * o.y, z * o.z)

Isosurface generation (now inluding frontend) #3495

Isosurface generation (now inluding frontend) #3495

Conversation

jfrohnhofen commented Nov 24, 2018 • edited by fm3 Loading

URL of deployed dev instance (used for testing):

Steps to test:

Issues:

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jfrohnhofen commented Nov 24, 2018 •

edited by fm3

Loading