Add multi output Renderbuffer feature and snap pick normals with it

[Kurtil]

I am developing a custom measurement plugin but to do so, I need normals information on SnapPickResult. Unfortunately there is only worldPosition available...

This PR includes two main changes:

• RenderBuffer multi output API with no breaking changes ! Only new feature 🎉
• SnapPickResult normals : snappedWorldNormal & worldNormal

Implementation:

The snap pick normals are computed using partiale derivatives like the flat normal pick renderers. Also, instead of doing another render pass, normals are computed on the same pass as the depth init pass of the snap pick. The multi output RenderBuffer API update allows drawing outCoords on color attachment 0 and outNormal on color attachment 1.

Impact:

Normals on SnapPickResult may also be used on other use cases like adding section plan using snap picking or other measurement plugin like spacing measurement...

[ghost]

👇 Click on the image for a new way to code review

Legend

[xeolabs]

Great, thanks @Kurtil - would you be able to post a screen capture of your measurement tool? Just curious how you're using the normals.

[Kurtil]

I use the normals to correctly display the length on top of the surface.

[xeolabs]

I use the normals to correctly display the length on top of the surface.

Nice. Since I suppose the normal of an edge, derived using derivatives, would point in the direction of the corner rather than the surface, I guess you take the average of the normals at the two end points?

[xeolabs]

Also, how do you generate the line and text? If as a 3D Mesh, are you using createVertexGeometry, and maybe a dynamically-refreshed Geometry?

[Kurtil]

Nice. Since I suppose the normal of an edge, derived using derivatives, would point in the direction of the corner rather than the surface, I guess you take the average of the normals at the two end points?

Absolutely not. It seems to work fine without the average. I only take the normal the last snap pick returns. At the beginning, the depth init shader was a bit different with a uniform uNormalProvided and if yes, I took the normals of the state.normalsBuff instead of the derivative, but the difference was not significative at all. I don't really understand why but like I said the normal of the edge is quite good (perpendicular to the surface). It also picks the correct normal depending on the surface I am hovering:

Also, how do you generate the line and text? If as a 3D Mesh, are you using createVertexGeometry, and maybe a dynamically-refreshed Geometry?

I recreate a new mesh for every change. I try to update the buffer at first with gl.bufferSubData. It worked well but the WebGL buffer is not publicly exposed (something like mesh._geometry._state.buffer) so I decided to change for a new mesh at every change. The text is generated with buildVectorTextGeometry.

The code I added on top of the vbo autocompressd triangle example:

    const markerDiv = document.createElement('div');
    const canvas = viewer.scene.canvas.canvas;
    canvas.parentNode.insertBefore(markerDiv, canvas);

    markerDiv.style.background = "black";
    markerDiv.style.border = "2px solid blue";
    markerDiv.style.borderRadius = "20px";
    markerDiv.style.width = "10px";
    markerDiv.style.height = "10px";
    markerDiv.style.margin = "-200px -200px";
    markerDiv.style.zIndex = "100";
    markerDiv.style.position = "absolute";
    markerDiv.style.pointerEvents = "none";

    // Mouse input

    let lastPosition = null;
    let line = null;

    function updateCursorPosition(canvasPos) {
        const snapPickResult = viewer.scene.snapPick({
            canvasPos,
            snapRadius: 30,
            // snapToEdge: false, // Default is true
            // snapToVertex: true // Default is true
        });
        if (snapPickResult) {
            if (snapPickResult.snappedCanvasPos) {
                markerDiv.style.marginLeft = `${snapPickResult.snappedCanvasPos[0] - 10}px`;
                markerDiv.style.marginTop = `${snapPickResult.snappedCanvasPos[1] - 10}px`;
                markerDiv.style.border = "3px solid green";
                markerDiv.style.background = "greenyellow";
            } else {
                const canvasPos = viewer.scene.camera.projectWorldPos(snapPickResult.worldPos)

                markerDiv.style.marginLeft = `${canvasPos[0] - 10}px`;
                markerDiv.style.marginTop = `${canvasPos[1] - 10}px`;
                markerDiv.style.border = "3px solid blue";
                markerDiv.style.background = "blue";
            }

            lastPosition = snapPickResult?.snappedWorldPos ?? snapPickResult?.worldPos;

            lastPosition.worldNormal = snapPickResult.snappedWorldNormal ?? snapPickResult.worldNormal;

        } else {
            markerDiv.style.marginLeft = `${canvasPos[0] - 10}px`;
            markerDiv.style.marginTop = `${canvasPos[1] - 10}px`;
            markerDiv.style.background = "white";
            markerDiv.style.border = "1px solid black";

            lastPosition = null;
        }
    }

    let mouseDown = false;
    let dragging = false;

    function onMouseMove(event) {
        if (mouseDown) {
            dragging = true;
        }
        event.preventDefault();
        updateCursorPosition([event.clientX, event.clientY]);

        if (lastPosition) {
            line?.move(lastPosition);
        }
    }

    function onMouseDown () {
        mouseDown = true;
    }

    function onMouseUp(mouveEvent) {
        mouseDown = false;
        const wasDragging = dragging;
        dragging = false;
        if (wasDragging) return;

        if (!lastPosition) return;

        if (line) {
            line = null;
        } else {
            line = makeLine(lastPosition);
        }
    }

    const TEXT_SIZE_RATIO = 0.08;
    const ARROW_SIZE_RATIO = 0.04;
    const TEXT_BOTTOM_MARGIN_RATIO = 0.04;

    const makeLine = p1 => {
        let line = null;
        let text = null;
        return {
            p1,
            /** 
             *    p4___p1___p3 
             *         /|\
             *        / | \
             *       p8 | p7
             *          |
             *          |
             *          |
             *      p10 | p9
             *        \ | /
             *    p6___\|/___p5
             *         p2
            **/
            move(p2) {
                line?.destroy();

                // compute length and position
                const vec = math.subVec3(p2, p1, math.vec3());

                if (vec.every(p => p === 0)) return;

                const normalizedVec = math.normalizeVec3(vec, math.vec3());
                const opositeNormalizedVec = math.mulVec3Scalar(normalizedVec, -1, math.vec3());

                const len = math.lenVec3(vec);
                const halfWidthVec = math.mulVec3Scalar(vec, 0.5, math.vec3());
                const position = math.addVec3(p1, halfWidthVec, math.vec3());
                const normal = lastPosition.worldNormal; // TODO the pick normal as measure normal is correct only for measure parallel to the surface. 

                const cross = math.cross3Vec3(normal, math.normalizeVec3(vec, math.vec3()), math.vec3());
                const oppositeCross = math.mulVec3Scalar(cross, -1, math.vec3());
                const scaledCross = math.mulVec3Scalar(cross, len * ARROW_SIZE_RATIO, math.vec3());
                const opositeScaledCross = math.mulVec3Scalar(oppositeCross, len * ARROW_SIZE_RATIO, math.vec3());

                const p3 = math.addVec3(p1, scaledCross, math.vec3());
                const p4 = math.addVec3(p1, opositeScaledCross, math.vec3());

                const p5 = math.addVec3(p2, scaledCross, math.vec3());
                const p6 = math.addVec3(p2, opositeScaledCross, math.vec3());

                const p7Vec = math.mulVec3Scalar(math.normalizeVec3(math.addVec3(cross, normalizedVec, math.vec3())), len * ARROW_SIZE_RATIO * Math.SQRT2);
                const p7 = math.addVec3(p1, p7Vec, math.vec3());

                const p8Vec = math.mulVec3Scalar(math.normalizeVec3(math.addVec3(oppositeCross, normalizedVec, math.vec3())), len * ARROW_SIZE_RATIO * Math.SQRT2);
                const p8 = math.addVec3(p1, p8Vec, math.vec3());

                const p9Vec = math.mulVec3Scalar(math.normalizeVec3(math.addVec3(cross, opositeNormalizedVec, math.vec3())), len * ARROW_SIZE_RATIO * Math.SQRT2);
                const p9 = math.addVec3(p2, p9Vec, math.vec3());

                const p10Vec = math.mulVec3Scalar(math.normalizeVec3(math.addVec3(oppositeCross, opositeNormalizedVec, math.vec3())), len * ARROW_SIZE_RATIO * Math.SQRT2);
                const p10 = math.addVec3(p2, p10Vec, math.vec3());

                line = new LineSet(viewer.scene, {
                    positions: [...p1, ...p2, ...p3, ...p4, ...p5, ...p6, ...p7, ...p8, ...p9, ...p10],
                    indices: [0, 1, 0, 2, 0, 3, 0, 6, 0, 7, 1, 4, 1, 5, 1, 8, 1, 9],
                    color: [0, 0, 1],
                });

                text?.destroy();

                const { look, up, eye } = viewer.scene.camera;

                const cameraLookVector = math.normalizeVec3(math.subVec3(eye, look, math.vec3()));
                const cameraCross = math.cross3Vec3(up, cameraLookVector, math.vec3());
                const cameraRight = math.normalizeVec3(cameraCross, math.vec3());

                const dotX = math.dotVec3(cameraRight, normalizedVec);

                let mat = math.mat4([
                    ...normalizedVec, 0,
                    ...math.normalizeVec3(dotX >= 0 ? cross : oppositeCross, math.vec3()), 0,
                    ...normal, 0,
                    ...position, 1
                ]);

                const rotation = dotX < 0 ? Math.PI : 0;

                text = makeText(len.toFixed(2) + " m", len, mat, rotation);
            }
        }
    };

    const makeText = (content, size, matrix, rotation) => {
        const {
            primitive,
            positions,
            indices
        } = buildVectorTextGeometry({
            text: content,
            size: size * TEXT_SIZE_RATIO,
        });

        let min = positions[0];
        let max = positions[0];

        for (let i = 0; i < positions.length; i += 3) {
            const x = positions[i];
            if (x < min) min = x;
            if (x > max) max = x;
        }

        const width = max - min;
        const centerAlignX = - width / 2;

        const translationMatrix = math.translationMat4v([centerAlignX, size * TEXT_BOTTOM_MARGIN_RATIO, 0]);

        if (rotation) {
            const rotationMatrix = math.rotationMat4v(Math.PI, [0, 1, 0], math.mat4());
            math.mulMat4(matrix, rotationMatrix);
        }


        math.mulMat4(matrix, translationMatrix);

        return new Mesh(viewer.scene, {
            geometry: new ReadableGeometry(viewer.scene, {
                primitive,
                positions,
                indices,
            }),
            material: new PhongMaterial(viewer.scene, {
                emissive: [0.0, 0.0, 1],
                lineWidth: 2
            }),
            matrix
        });
    }

    document.addEventListener("mousemove", onMouseMove);
    document.addEventListener("mousedown", onMouseDown);
    document.addEventListener("mouseup", onMouseUp);

The entire file if you want to try:

vbo_batching_autocompressed_triangles.html.zip

[Kurtil]

Don't be afraid of the maths ... ^^ I am not very good at it and I did not figure out which method to use to correctly project the text on top of the line. That is why I recreate the transformation matrix needed by hand :

const mat = math.mat4([
    ...normalizedVec, 0,
    ...math.normalizeVec3(dotX >= 0 ? cross : oppositeCross, math.vec3()), 0,
    ...normal, 0,
    ...position, 1
]);

Also there is sill one thing missing. The measure normal may not be always the snap pick normal. In fact, there normals are the same only if the measure is parallel to the surface. That is why this kind of measure is not correctly displayed:

(There is a TODO in the code according to this issue until I figure out how to compute the correct measure normal :P)

[Kurtil]

I fixed the TODO... the cross vector just needed to be normalised:

const cross = math.normalizeVec3(math.cross3Vec3(normal, math.normalizeVec3(vec, math.vec3()), math.vec3()));