Tensorflow segmentation (#3461)

* integrate first version of tensorflow-js segmentation model

* add flood fill and whole viewport prediction

* implement high-perf way of accessing arbitrary cuboid data in front-end

* fix some things, wand is working again

* fix everything

* copy tf models via webpack

* upgrade webpack

* increase node memory

* fix lint, tests

* fix model asset path

* update tensorflowjs to 1.0

* move viewport when pressing alt and moving mouse

* Tensorflow segmentation statistics (#3946)

* [WIP] provide sampled mean and stdev for color layer data

* convert to unsigned, clean up

* best resolution is actually min, not max

* pretty-backend

* adapt tensorflow segmentation to rectangular viewports

* fix lint

* added mean and stdDev fetching route and using it for the magic wand tool

* applied  new formular again

* fixed lint and flow

* predict for next 5 slices as well

* allow tf inference in webworker

* made magic wand tool optional

* fix shard in webpack-dev-server mode

* enabled magic wand tool as default

* fix tests

* use new segem model, restrict viewport to 100x100 voxel for perf

* small perf/cleanup fixes

* load model from right path

* only use webworker if offscreencanvas is supported

* use 3d-floodfill for magic wand

* add and adapt floodfill lib, do 3d floodfill, do tile-based predict and floodfill

* adapt floodfill to work slice for slice, optimize voxel labeling, remove unneccessary floodfill parts

* add flow to floodfill, fix flow

* add magic wand unlimited mode, will predict next x slices until aborted via toast

* use global tiles instead of viewport local, makes everything easier

* update the whole position, so that segments that move out of the viewport later will be correctly predicted

* fix magic wand for xz and yz planes

* fix mean,stddev memoization, refactor common code

* small changes to toast and code

* add new tensorflow model, apply PR feedback #1

* apply PR feedback #2

* update redux-saga and typedefs to 1.0, avoid spawning magic wand twice

* clean up prediction data for old z slices, rename magic wand to automatic brush

* refactor automatic brushing code into own module

* disable magic wand by default

* react to zoom changes, amend automatic brush label

* apply PR feedback #3, use newest model with slightly different normalization, make floodfill_threshold configurable

* disable automatic brush by default

conf/messages

@@ -96,6 +96,8 @@ dataSet.dataStore.missing=dataStore missing in the supplied json
 dataSet.dataSet.missing=dataSet missing in the supplied json
 dataSet.downloadAlreadyRunning=Sample dataset download is already running.
 dataSet.alreadyPresent=Sample dataset is already present.
+dataSet.noResolutions=Data layer does not contain resolutions
+dataSet.sampledOnlyBlack=Sampled data positions contained only black data
 
 dataSource.notFound=Datasource not found on datastore server
 

docker-compose.yml

@@ -99,6 +99,7 @@ services:
       - HOME=/root
       - CIRCLE_TAG=${CIRCLE_TAG}
       - CIRCLE_BUILD_NUM=${CIRCLE_BUILD_NUM}
+      - NODE_OPTIONS=--max_old_space_size=2048
     working_dir: /home/${USER_NAME:-sbt-user}/webknossos
     volumes:
       - ".:/home/${USER_NAME:-sbt-user}/webknossos"

docs/keyboard_shortcuts.md

@@ -38,6 +38,7 @@ Find all available keyboard shortcuts for webKnossos listed below.
 | Key Binding                   | Operation                                   |
 | ----------------------------- | ------------------------------------------- |
 | Left Mouse Drag or Arrow Keys | Move                                        |
+| Alt + Mouse Move              | Move                                        |
 | SPACE                         | Move Forward by a Single Slice              |
 | Scroll Mousewheel (3D View)   | Zoom In And Out                             |
 | Right Click Drag (3D View)    | Rotate 3D View                              |
@@ -78,6 +79,7 @@ Find all available keyboard shortcuts for webKnossos listed below.
 | CTRL + Left Mouse Drag            | Add Empty Voxels To Current Cell (in Trace / Brush Mode)    |
 | Right Mouse Drag                  | Remove Voxels From Current Cell                             |
 | CTRL + Right Mouse Drag           | Remove Voxels From Any Cell                                 |
+| Alt + Mouse Move                  | Move                                                        |
 | C                                 | Create New Cell                                             |
 | W, 1                              | Toggle Modes (Move / Trace / Brush)                         |
 | SHIFT + Mousewheel or SHIFT + I, O | Change Brush Size (Brush Mode)                              |