Geometric-error based point cloud attenuation and eye dome lighting #6069

likangning93 · 2017-12-21T02:03:19Z

This PR adds simple geometric-error-based attenuation and eye dome lighting for point clouds.

This trick only really works with replacement refinement point clouds right now, I'll follow up with some fixes for additive refinement.

cesium-concierge · 2017-12-21T02:03:20Z

@likangning93, thanks for the pull request! Maintainers, we have a signed CLA from @likangning93, so you can review this at any time.

⚠️ I noticed that CHANGES.md has not been updated. If this change updates the public API in any way, fixes a bug, or makes any non-trivial update, please add a bullet point to CHANGES.md and comment on this pull request so we know it was updated. For more info, see the Pull Request Guidelines.

I am a bot who helps you make Cesium awesome! Contributions to my configuration are welcome.

🌍 🌎 🌏

likangning93 · 2017-12-21T02:10:53Z

Awww who's a good bot? CHANGES.md updated.

pjcozzi · 2017-12-21T15:55:51Z

Possible to have a shorter name? 😄

pjcozzi · 2017-12-21T15:57:04Z

Can the Sandcastle example use better default parameters (and maybe camera view); currently, you can't really see the point cloud after load.

pjcozzi · 2017-12-21T15:58:43Z

For defaults, perhaps

decrease SSE
Set attenuation and EDL to true

pjcozzi · 2017-12-21T20:56:50Z

Maybe this is OK, but I found it strange that decreasing the max SSE - which should increase detail - forces the attenuation to become smaller.

pjcozzi · 2017-12-21T20:58:24Z

Maybe this is OK, but I found it strange that decreasing the max SSE - which should increase detail - forces the attenuation to become smaller.

Maybe the default max attenuation needs to be higher.

likangning93 · 2017-12-21T21:11:28Z

Maybe this is OK, but I found it strange that decreasing the max SSE - which should increase detail - forces the attenuation to become smaller.

In this case maximumAttenuation is the SSE instead of a constant, which was an attempt to reduce the amount of parameter tuning required. It also looks like there isn't any more detail getting called down, either the view is already of a leaf or server issues popped up while you were running the demo.

Maybe we could remove the link between maximumAttenuation and SSE and just add a note to recommend using the SSE as maximumAttenuation when tuning?

pjcozzi

Looks good, mostly trivial comments from me.

pjcozzi · 2017-12-21T21:01:28Z

Apps/Sandcastle/gallery/3D Tiles Point Cloud Attenuation and Eye Dome Lighting.html

+
+Cesium.knockout.getObservable(viewModel, 'geometricErrorScale').subscribe(
+    function(newValue) {
+        tileset.pointAttenuationOptions.geometricErrorScale = parseFloat(newValue);


Not sure about the "options" suffix for pointAttenuationOptions.

Perhaps pointShading?

This will also make eyeDomeLightingStrength and eyeDomeLightingRadius make more sense since it doesn't seem - at least from the user's perspective - that they should be a property on an attenuation object.

pjcozzi · 2017-12-21T21:02:59Z

Source/Scene/Cesium3DTileset.js

@@ -69,6 +71,8 @@ define([
        Cesium3DTilesetTraversal,
        Cesium3DTileStyleEngine,
        LabelCollection,
+        PointAttenuationOptions,


Same comment.

Throughout.

pjcozzi · 2017-12-21T21:05:39Z

Source/Scene/PointAttenuationOptions.js

+     * pointclouds using 3D Tiles.
+     *
+     * @param {Object} [options] Object with the following properties:
+     * {Boolean} [options.geometricErrorAttenuation=false] Perform point attenuation based on geometric error.


Why not just call this attenuation? The fact that is uses geometric error for sizing is an implementation detail and does not need to cloud the API and user's code.

I was hoping to distinguish this from Srinivas's post-process attenuation, since we might merge this in. It might also help to run both at the same time, with some tweaking.

I think it's also an important detail for the semi-technical user, since it works best based on the assumptions we're making for geometric error.

What is the one in the other branch called and what is the context?

In general, make the most common option the most obvious name. I suspect that this should be called attenuation and the other branch should be called specialAttenuationForWhateverReason.

pjcozzi · 2017-12-21T21:06:06Z

Source/Scene/PointAttenuationOptions.js

+     * {Number} [options.maximumAttenuation] Maximum attenuation in pixels. Defaults to the Cesium3DTileset's maximumScreenSpaceError.
+     * {Number} [options.baseResolution] Average base resolution for the dataset in meters. Substitute for Geometric Error when not available.
+     * {Boolean} [options.eyeDomeLighting=false] When true, use eye dome lighting when drawing with point attenuation.
+     * {Number} [options.eyeDomeLightingStrength=1.0] Increasing this value increases contrast on slopes and edges.


Here and below, what is the typical range?

eyeDomeLightingStrength looks best to me between 0.5 and 4.0, otherwise it starts getting too light or too dark..
eyeDomeLightingRadius is in pixels, so between 1.0 and 2.0 seems to be a good range.

OK, please document.

pjcozzi · 2017-12-21T21:06:39Z

Source/Scene/PointAttenuationOptions.js

+        var pointAttenuationOptions = defaultValue(options, {});
+
+        /**
+         * Perform point attenuation based on geometric error.


Throughout, include @default.

pjcozzi · 2017-12-21T21:17:23Z

Source/Shaders/PostProcessFilters/PointCloudEyeDomeLighting.glsl

+    vec2 depthAndLog2Depth = texture2D(u_pointCloud_ecAndLogDepthTexture, v_textureCoordinates + padding).zw;
+    if (depthAndLog2Depth.x == 0.0) // 0.0 is the clear value for the gbuffer
+    {
+        return vec2(0.0, 0.0); // throw out this sample


Just vec2(0.0) - it sets all components to zero.

pjcozzi · 2017-12-21T21:19:19Z

Source/Shaders/PostProcessFilters/PointCloudEyeDomeLighting.glsl

+        vec4 color = texture2D(u_pointCloud_colorTexture, v_textureCoordinates);
+
+        // sample from neighbors up, down, left, right
+        float padx = (1.0 / czm_viewport.z) * u_edlStrengthAndDistance.y;


Not 100% sure, but I think it is faster to make u_edlStrengthAndDistance a vec4 and then precompute these in the w and z channels.

pjcozzi · 2017-12-21T21:21:25Z

Specs/Scene/PointCloud3DTileContentSpec.js

@@ -436,6 +438,102 @@ defineSuite([
        });
    });

+    function countRenderedPixels(rgba) {


Instead of having this here, maybe add a new expectation function, e.g.,

expect(scene).toRenderPixelCount(5) expect(scene).toRenderPixelCountGreaterThan(5)

pjcozzi · 2017-12-21T21:22:08Z

Specs/Scene/PointCloud3DTileContentSpec.js

+        return count;
+    }
+
+    it('attenuates points based on geometric error', function() {


Should this be one test? Or should each call to expect... be a separate test so it is easier to track down test failures?

pjcozzi · 2017-12-21T21:22:46Z

Specs/Scene/PointCloudEyeDomeLightingSpec.js

+        scene.primitives.removeAll();
+    });
+
+    it('Adds a clear command and a post-processing draw call', function() {


Here and below it, use lowercase I think.

pjcozzi · 2017-12-21T21:26:06Z

Maybe we could remove the link between maximumAttenuation and SSE and just add a note to recommend using the SSE as maximumAttenuation when tuning?

Whatever you and @lilleyse think.

likangning93 · 2017-12-30T14:54:37Z

@pjcozzi updated!

pjcozzi · 2017-12-30T21:14:24Z

Thanks for the updates @likangning93!

@lilleyse please merge when ready.

No need for this to make the 1.41 release, but if it does, it should be merged on Jan 1 as I am starting the 1.41 release very early on Jan 2.

likangning93 · 2018-01-02T16:42:37Z

Source/Shaders/PostProcessFilters/PointCloudEyeDomeLighting.glsl

+        float shade = exp(-response * 300.0 * u_distancesAndEdlStrength.z);
+        color.rgb *= shade;
+        gl_FragColor = vec4(color);
+        gl_FragDepthEXT = czm_eyeToWindowCoordinates(vec4(ecAlphaDepth.xyz, 1.0)).z;


Just realized we shouldn't need this fragment depth write, since this still happens during rasterization and this data doesn't actually change...

since this still happens during rasterization and this data doesn't actually change

Or... not. I stand corrected. Looks like taking this out breaks depth picking. Whoops...

lilleyse · 2018-01-02T22:47:47Z

I haven't looked at the code yet, but it would be great to add the attenuation/EDL controls to its own panel in the 3D Tiles Inspector.

lilleyse · 2018-01-04T16:15:46Z

Also make sure to investigate the multi-frustum rendering bug.

likangning93 · 2018-01-08T18:14:42Z

Hmm, this doesn't currently work with shadows, which raises some concerns.

lilleyse · 2018-01-09T14:55:23Z

Ultimately it would be nice if we could move away from geometric error. One example is if we have really dense leaf tiles we might want to decrease their parent's geometric error so that they are not loaded as soon, but tweaking the geometric error like this has the side affect of making the parents render thinner. It's almost like we need a similar concept to geometric error but just for attenuation.

pjcozzi · 2018-01-09T23:18:05Z

It's almost like we need a similar concept to geometric error but just for attenuation.

Storing something like per-tile (or maybe maybe per-point) density is not out of the question. Adding to the 3D Tiles spec is OK.

One example is if we have really dense leaf tiles we might want to decrease their parent's geometric error so that they are not loaded as soon, but tweaking the geometric error like this has the side affect of making the parents render thinner

Thinkingly broadly here, maybe additional refinement parameters need to be considered so we aren't just fudging geometric error for cases like this.

likangning93 · 2018-01-23T14:46:21Z

One example is if we have really dense leaf tiles we might want to decrease their parent's geometric error so that they are not loaded as soon, but tweaking the geometric error like this has the side affect of making the parents render thinner

This also sounds like a case where we would want a few more LODs between the current parent and leaves in the tileset itself.

The current controls also incorporate a geometric error multiplier and attenuation limit that might help a little bit. I'll add [EDIT] to the sandcastle example showing/documenting how this can "kind of work" in cases where just straight-up geometric error doesn't produce good results.

likangning93 · 2018-01-23T18:17:15Z

@lilleyse updated! Post-merge TODOs:

open issue for shadows
open issue for Firefox draw buffer problem

pjcozzi · 2018-01-28T19:01:14Z

Source/Core/BoundingSphere.js

+     * Computes the radius of the BoundingSphere.
+     * @returns {Number} The radius of the BoundingSphere.
+     */
+    BoundingSphere.prototype.volume = function() {


Add this to CHANGES.md.

pjcozzi · 2018-01-28T19:02:17Z

Source/Core/BoundingSphere.js

+     */
+    BoundingSphere.prototype.volume = function() {
+        var radius = this.radius;
+        return (4.0 / 3.0) * CesiumMath.PI * radius * radius * radius;


Assign (4.0 / 3.0) * CesiumMath.PI to a file-scoped constant so that it is not recomputed each time this function is called. I'm not 100% sure the JIT will optimize it.

pjcozzi · 2018-01-28T19:06:18Z

Source/Scene/PointShading.js

+         * @type {Boolean}
+         * @default false
+         */
+        this.geometricErrorAttenuation = defaultValue(pointShading.geometricErrorAttenuation, false);


I know I've said this online and offline, but I still think we are going with a naming mistake here not just calling this attenuation for the same reason that Primitive is just called "Primitive" and the special cases have more precise names like GroundPrimitive. Always use the most obvious name for the common case.

Please reconsider here.

pjcozzi · 2018-01-28T19:08:37Z

@lilleyse are you able to review this soon so this can potentially make the 1.42 release on Thursday?

likangning93 · 2018-01-29T15:03:11Z

@lilleyse updated!

lilleyse

Awesome work @likangning93. The mechanics all look good - shaders, options, etc.

The point cloud shading object will also be helpful in exposing things like lighting and culling for point clouds with normals.

lilleyse · 2018-01-29T16:22:18Z

CHANGES.md

@@ -41,6 +41,8 @@ Change Log
 * Fixed camera movement and look functions for 2D mode [#5884](https://github.com/AnalyticalGraphicsInc/cesium/issues/5884)
 * Fixed discrepancy between default value used and commented value for default value for halfAxes of OrientedBoundingBox. [#6147](https://github.com/AnalyticalGraphicsInc/cesium/pull/6147)
 * Added `Cartographic.toCartesian` to convert from Cartographic to Cartesian3. [#6163](https://github.com/AnalyticalGraphicsInc/cesium/pull/6163)
+* Added geometric-error-based point cloud attenuation and eye dome lighting for point clouds using additive refinement.


Include the PR number like the other entries. Same below.

lilleyse · 2018-01-29T16:23:39Z

CHANGES.md

@@ -41,6 +41,8 @@ Change Log
 * Fixed camera movement and look functions for 2D mode [#5884](https://github.com/AnalyticalGraphicsInc/cesium/issues/5884)
 * Fixed discrepancy between default value used and commented value for default value for halfAxes of OrientedBoundingBox. [#6147](https://github.com/AnalyticalGraphicsInc/cesium/pull/6147)
 * Added `Cartographic.toCartesian` to convert from Cartographic to Cartesian3. [#6163](https://github.com/AnalyticalGraphicsInc/cesium/pull/6163)
+* Added geometric-error-based point cloud attenuation and eye dome lighting for point clouds using additive refinement.
+* Added volume computation to `BoundingSphere`.


Better to say what the actual function is:

Added BoundingSphere.volume for computing the volume of a BoundingSphere.

lilleyse · 2018-01-29T16:26:07Z

Source/Scene/Cesium3DTileset.js

@@ -124,6 +128,7 @@ define([
     * @param {Boolean} [options.debugShowRenderingStatistics=false] For debugging only. When true, draws labels to indicate the number of commands, points, triangles and features for each tile.
     * @param {Boolean} [options.debugShowMemoryUsage=false] For debugging only. When true, draws labels to indicate the texture and geometry memory in megabytes used by each tile.
     * @param {Boolean} [options.debugShowUrl=false] For debugging only. When true, draws labels to indicate the url of each tile.
+     * @param {Object} [options.pointShading] Options for constructing a PointShading object to control point attenuation based on geometric error and lighting.


Use {@link PointShading}

lilleyse · 2018-01-29T16:31:51Z

CHANGES.md

@@ -41,6 +41,8 @@ Change Log
 * Fixed camera movement and look functions for 2D mode [#5884](https://github.com/AnalyticalGraphicsInc/cesium/issues/5884)
 * Fixed discrepancy between default value used and commented value for default value for halfAxes of OrientedBoundingBox. [#6147](https://github.com/AnalyticalGraphicsInc/cesium/pull/6147)
 * Added `Cartographic.toCartesian` to convert from Cartographic to Cartesian3. [#6163](https://github.com/AnalyticalGraphicsInc/cesium/pull/6163)
+* Added geometric-error-based point cloud attenuation and eye dome lighting for point clouds using additive refinement.


using additive refinement

Should this be replacement?

lilleyse · 2018-01-29T16:36:08Z

Apps/Sandcastle/gallery/3D Tiles Point Cloud Shading.html

@@ -0,0 +1,274 @@
+<!DOCTYPE html>


This demo might look better with terrain on.

I gave it a quick try, but it looks like Chappes church is below terrain, and the terrain also doesn't perfectly align with the Mt. St. Helens points, it punches through in some views.

lilleyse · 2018-01-29T18:38:36Z

Specs/Scene/PointCloud3DTileContentSpec.js

+        });
+    });
+
+


Remove whitespace.

lilleyse · 2018-01-29T18:38:44Z

Specs/Scene/PointCloud3DTileContentSpec.js

+
+            scene.destroyForSpecs();
+        });
+    });


Given that all these tests are really similar, there could be a common function that all these tests call. Each test would pass in a callback function that makes the change (e.g. in here, the callback would contain scene.morphTo2D(0);.

lilleyse · 2018-01-29T18:39:00Z

Specs/Scene/PointCloudEyeDomeLightingSpec.js

+    PointCloudEyeDomeLighting,
+    Cesium3DTilesTester,
+    createScene) {
+    'use strict';


Fix indentation on includes.

lilleyse · 2018-01-29T18:39:23Z

Specs/Scene/PointCloudEyeDomeLightingSpec.js

+
+    beforeAll(function() {
+        scene = createScene();
+        scene.frameState.passes.render = true;


Is this line necessary?

lilleyse · 2018-01-29T18:44:36Z

Specs/addDefaultMatchers.js

@@ -241,6 +241,40 @@ define([
                };
            },

+            toRenderPixelCount : function(util, customEqualityTesters) {


Instead could this and below be a single function that just returns the number of the pixels? The comparison logic (equals and greater than) seems like it should belong in the individual tests.

@pjcozzi suggested adding these as new matchers: #6069 (comment)

I think the new matchers idea is good, but I think we can get away with just a single new matcher.

likangning93 · 2018-01-29T22:47:52Z

@lilleyse updated!

lilleyse · 2018-01-30T15:58:25Z

Source/Scene/PointCloudShading.js

+        this.eyeDomeLightingRadius = defaultValue(pointCloudShading.eyeDomeLightingRadius, 1.0);
+    }
+
+    PointCloudShading.isSupported = PointCloudEyeDomeLighting.isSupported;


Add doc for this. Example: https://github.com/AnalyticalGraphicsInc/cesium/blob/master/Source/Scene/GroundPrimitive.js#L365-L371

For consistency perhaps have PointCloudShading.isSupported take the scene instead of context, and pass scene.context through to PointCloudEyeDomeLighting.isSupported.

lilleyse · 2018-01-30T16:02:41Z

Looks great, just that one comment. Fix the merge conflict and this should be ready.

add simple geometric error based point cloud attenuation

501de27

added eye dome lighting for point clouds

095e152

likangning93 force-pushed the attenuation-and-EDL branch from 932663c to 095e152 Compare December 21, 2017 02:08

updated changes.md

dccefeb

add webgl extension support check to spec and requirements blurb to doc

aad2105

pjcozzi requested changes Dec 21, 2017

View reviewed changes

lilleyse mentioned this pull request Dec 22, 2017

Simple point cloud attenuation #5824

Closed

PR comments

9659688

pjcozzi assigned lilleyse Dec 30, 2017

likangning93 mentioned this pull request Jan 2, 2018

Firefox doesn't support a Framebuffer with colorTextures of different pixelDatatype #6076

Open

likangning93 commented Jan 2, 2018

View reviewed changes

merge master into attenuation-and-EDL

b77963d

merged master into attenuation-and-EDL

d3367bf

likangning93 added 2 commits January 23, 2018 11:01

added another tileset to point cloud shading sandcastle

e754963

added point cloud attenuation and EDL options to 3d tiles inspector

9f9ebd1

pjcozzi reviewed Jan 28, 2018

View reviewed changes

likangning93 added 2 commits January 29, 2018 09:59

geometricErrorAttenuation -> attenuation

4c7231b

merged master into attenuation-and-EDL

0750c83

change access token for mt. st. helens

4e765ac

lilleyse reviewed Jan 29, 2018

View reviewed changes

likangning93 added 2 commits January 29, 2018 16:54

PR comments

6041751

add check for points with color, fix for points with normal lighting

314355c

lilleyse reviewed Jan 30, 2018

View reviewed changes

likangning93 added 2 commits January 30, 2018 11:28

merged in master

1484470

comments for PointCloudShading.isSupported

c4a3f6b

likangning93 force-pushed the attenuation-and-EDL branch from 6b1879f to c4a3f6b Compare January 30, 2018 17:09

Update PointCloudShading.js

4400531

lilleyse merged commit f1fad41 into CesiumGS:master Jan 30, 2018

lilleyse deleted the attenuation-and-EDL branch January 30, 2018 18:31

Geometric-error based point cloud attenuation and eye dome lighting #6069

Geometric-error based point cloud attenuation and eye dome lighting #6069

Conversation

likangning93 commented Dec 21, 2017

cesium-concierge commented Dec 21, 2017

likangning93 commented Dec 21, 2017

pjcozzi commented Dec 21, 2017

pjcozzi commented Dec 21, 2017

pjcozzi commented Dec 21, 2017

pjcozzi commented Dec 21, 2017

pjcozzi commented Dec 21, 2017

likangning93 commented Dec 21, 2017

pjcozzi left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

likangning93 Dec 29, 2017 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

pjcozzi commented Dec 21, 2017

likangning93 commented Dec 30, 2017

pjcozzi commented Dec 30, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

lilleyse commented Jan 2, 2018

lilleyse commented Jan 4, 2018

likangning93 commented Jan 8, 2018

lilleyse commented Jan 9, 2018

pjcozzi commented Jan 9, 2018

likangning93 commented Jan 23, 2018 • edited Loading

likangning93 commented Jan 23, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

pjcozzi commented Jan 28, 2018

likangning93 commented Jan 29, 2018

lilleyse left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

likangning93 commented Jan 29, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

lilleyse commented Jan 30, 2018

likangning93 Dec 29, 2017 •

edited

Loading

likangning93 commented Jan 23, 2018 •

edited

Loading