Skip to content

Commit

Permalink
Upgrade ThreeJS and use WebGL 2 (#6350)
Browse files Browse the repository at this point in the history
* don't add default recommendation configuration to task type if there is already a recommended configuration

* update changelog

* tmp: first draft of cuckoo hashing

* add todo comment

* clean up a bit

* rewrite insert implementation

* fix some bugs and implement rehash mechanism

* temporary debugging code

* dont use entry format for interface; avoid duplicate keys in test sets

* improve handling of self-loops by picking next seed function randomly

* clean up

* extract cuckoo table to own module

* tune performance a bit

* allow 0,0,0,0-keys by using -1,-1,-1,-1 as empty key

* adapt cuckoo table to have vector2 as value

* rename set/get methods

* update three to 114 and prepare webgl2 upgrade

* update threejs to 120

* update threejs to 124

* update threejs to 125 (cross sections might be broken now)

* switch to webgl2 (float texture is still broken)

* update threejs to 137 (allocation of UpdatableTexture might be inefficient now since size cannot be altered later anymore)

* clean up

* fix RG usage of look up texture in shader

* fix uint24 layers by padding RGB to RGBA

* fix deprecated slerp method for quaternions

* type UpdatableTexture and clean up

* remove cuckoo related diff from branch

* readd deleted prepare invocation

* fix linting

* remove commented-out computeFaceNormals()

* fix texture bucket manager spec

* replace glsl parser in test; bump node requirement to >= 16

* bump node to 16 in Dockerfile.dev and readme

* remove unnecessary import

* fix texture bucket manager spec

* remove shader rendering spec due to incompatibility and minor impact

* update import-js to fix gyp building errors with sqlite3

* remove import-js since TS LSP already offers the same feature

* temporarily run nightly on ci push

* use new puppeteer docker image

* use new master tag for puppeteer image

* revert to default CI workflow

* properly check webgl2 support before trying to render anything

* add rel=noreferrer

* update changelog

* fix STL export

* fix type error in isosurface saga

* fix the cross sections

* fix react warning for HoverIconButton

* fix highlighting of active node by migrating shader to es300 and not expecting GL_OES_standard_derivatives

* update migration guide

* remove unused import

* mitigate too-bright brush color

* fix that last 8 slices of each RGB bucket were black

* use a different position for the rgb screenshot test

* temporarily run nightly on ci push

* remove accidental only modifier for screenshot tests

* enforce no only modifiers in ava via linting

* update screenshots

* update screenshots

* Revert "temporarily run nightly on ci push"

This reverts commit c7089db.

* remove accidental only() modifiers in tests

* fix flycam_accessors spec

* fix volume tracing saga spec

* update backend snapshots

* try to reduce ram overhead for dummy canvas

* temporarily disable most CI checks

* only create 1x1 canvas and use bigger image data

* tweak login form for small screens

* remove superfluous geometry update code (already done by updatePositionForCrossSections)

* explicitly release canvas for iOS

* null image after first update in UpdatableTexture

* add debugging alert

* Revert "temporarily disable most CI checks"

This reverts commit 4053cf8.

* remove debugging alert

* don't do createElement(canvas) at top level of module
  • Loading branch information
philippotto authored Aug 4, 2022
1 parent 217ce17 commit 5d85b9f
Show file tree
Hide file tree
Showing 49 changed files with 796 additions and 1,081 deletions.
3 changes: 2 additions & 1 deletion .eslintrc.json
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@
"plugin:react/recommended",
"prettier"
],
"plugins": ["@typescript-eslint", "react", "react-hooks", "import"],
"plugins": ["@typescript-eslint", "react", "react-hooks", "import", "ava"],
"parser": "@typescript-eslint/parser",
"parserOptions": {
"ecmaVersion": 8,
Expand Down Expand Up @@ -102,6 +102,7 @@
]
}
],
"ava/no-only-test": "error",

// reconsider rules after finishing Typescript conversion
"react/static-property-placement": "off",
Expand Down
1 change: 1 addition & 0 deletions CHANGELOG.unreleased.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@ For upgrade instructions, please check the [migration guide](MIGRATIONS.released
- The NML file in volume annotation download now includes segment metadata like names and anchor positions. [#6347](https://github.com/scalableminds/webknossos/pull/6347)

### Changed
- webKnossos uses WebGL 2 instead of WebGL 1 now. In case your browser/hardware does not support this, webKnossos will alert you and you need to upgrade your system. [#6350](https://github.com/scalableminds/webknossos/pull/6350)
- The sharing modal now automatically saves changes of the sharing options. [#6314](https://github.com/scalableminds/webknossos/pull/6314)
- The Layers tab now displays an Add Skeleton Annotation Layer button with which volume-only annotations can be converted to hybrid annotations. [#6330](https://github.com/scalableminds/webknossos/pull/6330)
- The Zarr directory listings no longer include the current directory “.”. [6359](https://github.com/scalableminds/webknossos/pull/6359)
Expand Down
2 changes: 1 addition & 1 deletion Dockerfile.dev
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
FROM scalableminds/sbt:master__113
ARG VERSION_NODE="12.x"
ARG VERSION_NODE="16.x"

ENV DEBIAN_FRONTEND noninteractive

Expand Down
2 changes: 2 additions & 0 deletions MIGRATIONS.unreleased.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,5 +8,7 @@ User-facing changes are documented in the [changelog](CHANGELOG.released.md).
## Unreleased
[Commits](https://github.com/scalableminds/webknossos/compare/22.08.0...HEAD)

- webKnossos requires node 16 now. [#6350](https://github.com/scalableminds/webknossos/pull/6350)

### Postgres Evolutions:
- [084-annotation-contributors.sql](conf/evolutions/084-annotation-contributors.sql)
4 changes: 2 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -86,7 +86,7 @@ For non-localhost deployments, check out the [installation guide in the document
* [PostgreSQL 10+](https://www.postgresql.org/)
* [Redis 5+](https://redis.io/)
* [Blosc](https://github.com/Blosc/c-blosc)
* [node.js 12+](http://nodejs.org/download/)
* [node.js 16+](http://nodejs.org/download/)
* [yarn package manager](https://yarnpkg.com/)
* [git](http://git-scm.com/downloads)

Expand Down Expand Up @@ -175,7 +175,7 @@ See: http://www.scala-sbt.org/release/docs/Getting-Started/Setup.html

##### node.js & yarn
* Install node from http://nodejs.org/download/
* node version **12+ is required**
* node version **16+ is required**
* Install yarn package manager: `npm install -g yarn`

### Run locally
Expand Down
2 changes: 1 addition & 1 deletion frontend/javascripts/admin/auth/login_view.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,7 @@ function LoginView({ history, redirect }: Props) {
}}
align="middle"
>
<Col span={8}>
<Col xs={24} sm={16} md={8}>
<h3>Login</h3>
<LoginForm layout="horizontal" onLoggedIn={onLoggedIn} />
</Col>
Expand Down
6 changes: 3 additions & 3 deletions frontend/javascripts/components/hover_icon_button.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -22,9 +22,9 @@ export function HoverIconButton(props: HoverButtonProps) {
props.onMouseLeave(event);
}
};

return React.cloneElement(isMouseOver ? props.hoveredIcon : props.icon, {
...props,
const { hoveredIcon, ...restProps } = props;
return React.cloneElement(isMouseOver ? hoveredIcon : props.icon, {
...restProps,
onMouseEnter,
onMouseLeave,
});
Expand Down
213 changes: 121 additions & 92 deletions frontend/javascripts/libs/UpdatableTexture.ts
Original file line number Diff line number Diff line change
@@ -1,106 +1,135 @@
// @ts-nocheck
import * as THREE from "three";
import { document } from "libs/window";
import _ from "lodash";

function UpdatableTexture(
this: THREE.Texture,
width: number,
height: number,
format,
type,
mapping,
wrapS,
wrapT,
magFilter,
minFilter,
anisotropy,
encoding,
) {
THREE.Texture.call(
this,
null,
mapping,
wrapS,
wrapT,
magFilter,
minFilter,
format,
type,
anisotropy,
encoding,
);
const lazyGetCanvas = _.memoize(() => {
const canvas = document.createElement("canvas");
canvas.width = width;
canvas.height = height;
canvas.width = 1;
canvas.height = 1;
return canvas;
});

const getImageData = _.memoize((width: number, height: number): ImageData => {
const canvas = lazyGetCanvas();
const ctx = canvas.getContext("2d");
const imageData = ctx.createImageData(1, 1);
this.image = imageData;
this.magFilter = magFilter !== undefined ? magFilter : THREE.LinearFilter;
this.minFilter = minFilter !== undefined ? minFilter : THREE.LinearMipMapLinearFilter;
this.generateMipmaps = false;
this.flipY = false;
this.unpackAlignment = 1;
this.needsUpdate = true;
}
if (ctx == null) {
throw new Error("Could not get context for texture.");
}
const imageData = ctx.createImageData(width, height);

// Explicitly "release" canvas. Necessary for iOS.
// See https://pqina.nl/blog/total-canvas-memory-use-exceeds-the-maximum-limit/
canvas.width = 1;
canvas.height = 1;
ctx.clearRect(0, 0, 1, 1);

return imageData;
});

class UpdatableTexture extends THREE.Texture {
isUpdatableTexture: boolean;
renderer!: THREE.WebGLRenderer;
gl: any;
utils!: THREE.WebGLUtils;
width: number | undefined;
height: number | undefined;

UpdatableTexture.prototype = Object.create(THREE.Texture.prototype);
UpdatableTexture.prototype.constructor = UpdatableTexture;
UpdatableTexture.prototype.isUpdatableTexture = true;
constructor(
width: number,
height: number,
format?: THREE.PixelFormat,
type?: THREE.TextureDataType,
mapping?: THREE.Mapping,
wrapS?: THREE.Wrapping,
wrapT?: THREE.Wrapping,
magFilter?: THREE.TextureFilter,
minFilter?: THREE.TextureFilter,
anisotropy?: number,
encoding?: THREE.TextureEncoding,
) {
const imageData = getImageData(width, height);

UpdatableTexture.prototype.setRenderer = function setRenderer(renderer) {
this.renderer = renderer;
this.gl = this.renderer.getContext();
this.utils = THREE.WebGLUtils(this.gl, this.renderer.extensions, this.renderer.capabilities);
};
super(
// @ts-ignore
imageData,
mapping,
wrapS,
wrapT,
magFilter,
minFilter,
format,
type,
anisotropy,
encoding,
);

UpdatableTexture.prototype.setSize = function setSize(width, height) {
if (width === this.width && height === this.height) return;
if (!this.isInitialized()) return;
this.width = width;
this.height = height;
const activeTexture = this.gl.getParameter(this.gl.TEXTURE_BINDING_2D);
const textureProperties = this.renderer.properties.get(this);
this.gl.bindTexture(this.gl.TEXTURE_2D, textureProperties.__webglTexture);
if (!this.isInitialized()) this.width = null;
this.gl.texImage2D(
this.gl.TEXTURE_2D,
0,
this.utils.convert(this.format),
width,
height,
0,
this.utils.convert(this.format),
this.utils.convert(this.type),
null,
);
this.gl.bindTexture(this.gl.TEXTURE_2D, activeTexture);
};
this.magFilter = magFilter !== undefined ? magFilter : THREE.LinearFilter;
this.minFilter = minFilter !== undefined ? minFilter : THREE.LinearMipMapLinearFilter;
this.generateMipmaps = false;
this.flipY = false;
this.unpackAlignment = 1;
this.needsUpdate = true;
this.isUpdatableTexture = true;
}

setRenderer(renderer: THREE.WebGLRenderer) {
this.renderer = renderer;
this.gl = this.renderer.getContext();
this.utils = new THREE.WebGLUtils(
this.gl,
this.renderer.extensions,
this.renderer.capabilities,
);
}

UpdatableTexture.prototype.isInitialized = function isInitialized() {
return this.renderer.properties.get(this).__webglTexture != null;
};
setSize(width: number, height: number) {
if (width === this.width && height === this.height) return;
if (!this.isInitialized()) return;
this.width = width;
this.height = height;
const activeTexture = this.gl.getParameter(this.gl.TEXTURE_BINDING_2D);
const textureProperties = this.renderer.properties.get(this);
this.gl.bindTexture(this.gl.TEXTURE_2D, textureProperties.__webglTexture);
if (!this.isInitialized()) this.width = undefined;

UpdatableTexture.prototype.update = function update(src, x, y, width, height) {
if (!this.isInitialized()) {
this.renderer.initTexture(this);
this.gl.texImage2D(
this.gl.TEXTURE_2D,
0,
this.utils.convert(this.format),
width,
height,
0,
this.utils.convert(this.format),
this.utils.convert(this.type),
null,
);
this.gl.bindTexture(this.gl.TEXTURE_2D, activeTexture);
}

this.setSize(width, width);
const activeTexture = this.gl.getParameter(this.gl.TEXTURE_BINDING_2D);
const textureProperties = this.renderer.properties.get(this);
this.gl.bindTexture(this.gl.TEXTURE_2D, textureProperties.__webglTexture);
this.gl.texSubImage2D(
this.gl.TEXTURE_2D,
0,
x,
y,
width,
height,
this.utils.convert(this.format),
this.utils.convert(this.type),
src,
);
this.gl.bindTexture(this.gl.TEXTURE_2D, activeTexture);
};
isInitialized() {
return this.renderer.properties.get(this).__webglTexture != null;
}

update(src: Float32Array | Uint8Array, x: number, y: number, width: number, height: number) {
if (!this.isInitialized()) {
this.renderer.initTexture(this);
}
const activeTexture = this.gl.getParameter(this.gl.TEXTURE_BINDING_2D);
const textureProperties = this.renderer.properties.get(this);
this.gl.bindTexture(this.gl.TEXTURE_2D, textureProperties.__webglTexture);
this.gl.texSubImage2D(
this.gl.TEXTURE_2D,
0,
x,
y,
width,
height,
this.utils.convert(this.format),
this.utils.convert(this.type),
src,
);
this.gl.bindTexture(this.gl.TEXTURE_2D, activeTexture);
this.image = null;
}
}
export default UpdatableTexture;
Loading

0 comments on commit 5d85b9f

Please sign in to comment.