Geometry and Appearances

[pjcozzi]

Also see the discussion on the forum.

This will introduce a new generic Primitive (name TBA) that is higher-level than Renderer, but lower-level than the current primitives. It is meant for batching of heterogeneous static geometry. Static means changing the geometry, i.e., the vertex data, is expensive, but changing the modelMatrix, is, of course, still cheap. Dynamic geometry batching is a separate topic, and will require direct, or near direct, access to vertex buffers.

This decouples primitives into geometry and appearance, making it easy to add new geometry types, e.g., boxes, cylinders, etc., and new appearances, e.g., full GLSL shader + render state + material. The design is nothing new; it is what is done in pretty much every 3D engine and book out there, and similar to - but more general than - what I did in Insight3D years ago.

We'll use this to add features for KML, and may rewrite existing primitives - most likely Polygon and EllipsoidPrimitive - to use it. The Geometric Algorithms Google Summer of Code proposal is a subset of this effort.

Example code

Today we write:

var extent = new Polygon();
extent.configureExtent(
    new Extent(
        Math.toRadians(-180.0),
        Math.toRadians(50.0),
        Math.toRadians(180.0),
        Math.toRadians(90.0)));
primitives.add(extent);

In the future, we would write:

var geometry = new ExtentGeometry(
    new Extent(
        Math.toRadians(-180.0),
        Math.toRadians(50.0),
        Math.toRadians(180.0),
        Math.toRadians(90.0)));
var appearance = new SurfaceApperance();

var extent = new Primitive(geometry, appearance);
primitives.add(extent);

It's more code. Don't worry.

(1) it allows for batching:

var geometry = new ExtentGeometry(
    new Extent(
        Math.toRadians(-180.0),
        Math.toRadians(50.0),
        Math.toRadians(180.0),
        Math.toRadians(90.0)));
var otherGeometry = new ExtentGeometry(
    new Extent(
        Math.toRadians(-180.0),
        Math.toRadians(20.0),
        Math.toRadians(180.0),
        Math.toRadians(30.0)));
var apperance = new SurfaceApperance();

var extent = new Primitive([geometry, otherGeometry], appearance);
primitives.add(extent);

(2) it is easy to draw new geometry or with new appearances

var geometry = // something read from a file, computed procedurally, taken from user input, etc.
var appearance = new AppearanceIWrote();

var extent = new Primitive(geometry, appearance);
primitives.add(extent);

(3) we can write nice (slow) wrappers on top of it:

var extent = new ExtentPrimitive(
    new Extent(
        Math.toRadians(-180.0),
        Math.toRadians(50.0),
        Math.toRadians(180.0),
        Math.toRadians(90.0)));
primitives.add(extent);

Now it's less code. Magic.

The new Primitive allows for individual geometry picking, and high-precision rendering with GPU RTE. In theory, I think I can make it work for Columbus view too.

Details

@bagnell

• Tech blog post - Web workers
• Tech blog post - "A WebGL Batching Pipeline."

Primitives

• Automatically match geometry and appearance VertexFormat.
  • In the primitive's geometry pipeline, we can procedurally add normals, binormals, and tangents if the geometry doesn't already have them. However:
    • This is not as efficient as having the geometry compute them since Primitive needs to use generic algorithms.
    • To implement this for when an Appearance/Material changes (which can change the VertexFormat), we would have to keep the geometry in system memory. Ugh. This could be an "auto" option I suppose.
    • We can't use generic algorithms to compute texture coordinates; instead, each Geometry would need a function to add texture coordinates.
    • At the batching-level of abstraction, requiring the user to match vertex formats may be reasonable so we might not do anything.

Geometries

• Outline and polyline interop
• Sector geometry (pie slice)
• Provide progress update from worker. Include onComplete (for primitives built on geometries too) so, for example, we can destroy a primitive once the new one is ready.
• Should WallGeometry just be an extrusion object for SimplePolylineGeometry?
• Per vertex normals for PolylineVolumeGeometry and friends so the polygon can roll, i.e., it is not always oriented north.

Appearances

• Pass uniforms to appearances
• What new materials should we add that work well with these new geometries?
• Ray-casted ellipsoid appearance
• Ray-casted cylinder appearance

Scripting

• Scriptable (JSON) geometry
• Scriptable appearances like materials

Performance

• Use several workers for loading.
• Store pickColor attribute in a separate buffer and have a separate vertex array for the pick pass; this will reduce memory bandwidth and cache pollution for the color pass.
• Use texture atlas with geometry?
• Only introduce new vertices for normals for walls and extruded geometries based on the adjacent face angles.
• Optimize fitToUnsignedShortIndices to use tighter bounding spheres per-geometry.

Done

@bagnell

• Add binormal and tangent attributes to BoxGeometry.
• Finish EllipsoidGeometry. It only computes positions right now.
• Replace ExtentTessellator with a new ExtentGeometry.
• Compute all attributes for wall.
• Compute all attributes for polygon.
• Add new PolygonGeometry that works with and without holes. It will not use the full pipeline used in the current Polygon primitive because some of that is in the generic Primitive.
• Add new CircleGeometry and EllipseGeometry.
• Add new WallGeometry. Meet the needs of KML and Sky.
• Profile to decide if we should change all geometries/tessellators to use typed arrays and Float64Array, which will also require some WebGL-related changes.
• Figure out what to do with HeightmapTessellator.
• Add Columbus view support (notes below)
• How to implement show per geometry, i.e., how to change vertex data? We should be able to change color too. (notes below)
• Finish tests
• Fix Sandcastle examples:
  • http://localhost:8080/Apps/Sandcastle/index.html?src=Materials.html
  • http://localhost:8080/Apps/Sandcastle/index.html?src=Custom%20Rendering.html
  • http://localhost:8080/Apps/Sandcastle/index.html?src=Animations.html
  • http://localhost:8080/Apps/Sandcastle/index.html?src=Polygons.html
• Use a dirty list for updating per-instance attributes.
• Extent and ellipse geometries should be able to rotate texture coordinates.
• Optimize bounding sphere created from GeometryPipeline.combine.
• Profile geometry pipeline. What is the optimal order of pipeline stages?
• Investigate moving geometry pipeline to a web worker.
• Back-and-front passes for translucent volumes.  See @pjcozzi.
• Create geometry and appearances for the existing polyline so we can render static polylines efficiently.
• Fix missing triangles in cylinder.  Cylinder geometry #949 

@hpinkos

• Generic function to compute binormal and tangent attributes given a geometry with texture coordinates. See Computing Tangent Space Basis Vectors for an Arbitrary Mesh. @tfili has a C++ implementation.
• Generic function to compute normal attributes given a geometry with positions. I can dig up a C++ implementation.
• Add new ExtrudedExtentGeometry (with bottom/top) - or make this part of the ExtentGeometry.  Meet the needs of KML.
• Add new ExtrudedPolygonGeometry (with bottom/top) - or make this part of the PolygonGeometry.  Meet the needs of KML.
• SphereGeometry based on EllipsoidGeometry.
• Ability to extruded EllipseGeometry and CircleGeometry.
• Add CylinderGeometry. @pjcozzi has C++ code.
• Make WallGeometry subsample so it matches the curvature of the ellipsoid.
• Ability to create outlines for PolygonGeometry, ExtentGeometry, EllipseGeometry, CircleGeometry, WallGeometry, BoxGeometry, and EllipsoidGeometry. Go over the design with @pjcozzi beforehand.
• CorridorGeometry. Option to miter or bevel corners.
• CorridorOutlineGeometry.
• Generalize corridors and tubes as extruded shapes along a polyline.
• Tubes for aircraft keepout zones.

@pjcozzi

• Add color to VertexFormat.
• Add Surface (procedurally compute normal/binormal/tangent) appearance.
• Customize geometry pipeline in Primitive, e.g., optimize vertex cache.
• Support OES_element_index_uint
• Support OES_vertex_array_object
• Decouple geometry from modelMatrix, color, and pickData, perhaps call it NodeGeometry or Spatial. This will allow us to "instance" the geometry, but we need to be careful because many pipeline operations modify the geometry in place.
• Rename mesh/meshes to geometry/geometries throughout.
• Finalize Geometry format. In particular: do we need more than one indexList and modify combine to work without indices.
• Include doc for well-known vertex names
• Finish reference doc
• Match VertexFormat and Appearance, e.g., add normals if needed.
• SimplePolyline geometry and appearance.
• Implement Polygon based on geometry and appearance.
• Add Extent primitive.
• Add back lighting.
• Two-sided lighting, e.g., for walls.
• Sandcastle example showing different geometry types.

[pjcozzi]

@bagnell for per-instance show/color above, let's:

• Add two properties to the Primitive constructor options. I think these should default to false.
  • mutableShow - allows updating show on a per-instance basis.
  • mutableColor - allows updating color on a per-instance basis when a per-instance color appearance is used.
• When either mutable property is true, a separate vertex buffer per-property should be used to store the show or color. Show should be a single component unsigned byte unless it hits a driver slow path, in which case, it should be a single component float (0.0 or 1.0).
• When mutableShow is true, we should dynamically edit the vertex shader similar to how we edit the fragment shader for picking. This way each appearance does not not need to implement it. It should be simple to just have an if statement that sets gl_Position so the vertex is clipped.
• Add show to GeometryInstance so it can provide an initial value and to mirror the workflow with color.
• To give the user an interface to modify show/color per-instance without having to keep the instance in memory (with a reference to the BIG geometry), let's:
  • Rename pickData to id.
  • Add getMutableInstance to Primitive, which would work like this:

var instance = new GeometryInstance({
  geometry : geometry,
  color : color,
  show : false,
  id : 'my instance id' // also used for picking
});

var primitive = new Primitive({
  instances : [instance, anotherInstance, …],
  mutableShow : true,
  mutableColor : true
});

primitive.getMutableInstance('my instance id').show = true;
primitive.getMutableInstance('my instance id').color = new Color();

The mutable instance uses property getter/setters (performance will not be significant here), and maps the instance to the vertex buffer range that needs to be updated. Like the BillboardCollection, use a heuristic to determine when to do sub-buffer updates vs. just rewriting the entire buffer. If your heuristic is better then the billboards (which should be easy to beat), also update them.

[pjcozzi]

@bagnell for Columbus view / 2D above, let's

• Add a allowColumbusView property to the options of all geometries, which defaults to true. When this is true:
  • Make all geometries clip their triangles to the IDL like PolygonGeometry. Recall that we have the generic IntersectionTests.trianglePlaneIntersection function in addition to the more specific functions in PolygonPipeline and PolylinePipeline.
• Add a allowColumbusView property to the Primitive constructor options, which defaults to true.
  • In the geometry pipeline, after transformToWorldCoordinates and GeometryPipeline.combine, compute position2D from position, and then encode to have:
    • position3DHigh / position3DLow
    • position2DHigh / position2DLow
  • How should we deal with normal/binormal/tangent?
  • Can we procedurally modify the appearance's vertex shader to support Columbus view. It would be similar to PolygonVS.glsl (that if/else block is needed to avoid jitter). How far can we take this? Does an appearance vertex shader need to output gl_Position at all? Or can that be appended by Primitive? We'll have a better idea when we do a few more appearances.
  • Can we compute the Columbus-view bounding-sphere directly from the geometry bounding sphere? Or do we need to use the transformed positions to account for the projection and IDL? I believe the later.

[pjcozzi]

@bagnell here's a new plan for per-instance show/color based on discussion with @kring. It's only a bit more work, but it is a lot more general, in that it will allow passing any per-instance data to an appearance instead of just show/color.

• Remove color from GeometryInstance and replace it with generic per-instance attributes (think of it as a uniform for the instance, but it is really stored as a vertex attribute).

var instance = new GeometryInstance({
  geometry : geometry,
  attributes : {
    color : new GeometryInstanceAttribute({
      componentDatatype : ComponentDatatype.UNSIGNED_BYTE,
      componentsPerAttribute : 4,
      normalize : true,
      value : color.toTypedArray() // function doesn't exist but you get it
    }),
    show : new GeometryInstanceAttribute({
      componentDatatype : ComponentDatatype.UNSIGNED_BYTE,
      componentsPerAttribute : 1,
      normalize : true,
      value : 0
    }),
  }
});

Specific GeometryInstanceAttribute types will simplify this:

var instance = new GeometryInstance({
  geometry : geometry,
  attributes : {
    color : new ColorGeometryInstanceAttribute(color),
    show : new ShowGeometryInstanceAttribute(false)
  }
});

Show should be a single component unsigned byte unless it hits a driver slow path, in which case, it should be a single component float (0.0 or 1.0).

• In Primitive, verify that all provided instances have the same (or no) attributes. Throw if they don't or maybe just find the common subset. Not sure which is better.
• For each attribute, create a separate vertex buffer with a dynamic update hint. We can add other hints later, e.g., to interleave them all into a static buffer.
• We'll support well-known attribute names, i.e., semantics, just like we already do for geometry attributes. When the show attribute is present, we should dynamically edit the vertex shader similar to how we edit the fragment shader for picking. This way each appearance does not not need to implement it. It should be simple to just have an if statement that sets gl_Position so the vertex is clipped.
• To give the user an interface to modify per-instance attributes without having to keep the instance in memory (with a reference to the BIG geometry), let's:
  • Rename pickData to id.
  • Add getGeometryInstanceAttribues to Primitive, which would work like this:

var instance = new GeometryInstance({
  geometry : geometry,
  attributes : {
    color : new ColorGeometryInstanceAttribute(color),
    show : new ShowGeometryInstanceAttribute(false)
  },
  id : 'my instance id' // also used for picking
});

var primitive = new Primitive({
  instances : [instance, anotherInstance, …]
});

var attributes = primitive.getGeometryInstanceAttribues('my instance id');
attributes.color = ColorGeometryInstanceAttribute.toValue(new Color());
attributes.show = ShowGeometryInstanceAttribute.toValue(true);

The attributes use property getter/setters (performance will not be significant here), and map the instance to the vertex buffer range that needs to be updated. Like the BillboardCollection, use a heuristic to determine when to do sub-buffer updates vs. just rewriting the entire buffer. If your heuristic is better then the billboards (which should be easy to beat), also update them.

[pjcozzi]

@bagnell if you didn't already notice, to map per-instance attributes to a buffer range, we should optimize for the vertex caches before combing instances; otherwise, maintaining the mapping will be a pain. This will lead to a slightly less optimal ordering since there will be a compulsory miss at the start of each instance, but it will not matter at all in practice. In addition, this should improve the overall performance of the geometry pipeline.

[pjcozzi]

Only introduce new vertices for normals for walls and extruded geometries based on the adjacent face angles.

@bagnell are we already doing this? Or do we always introduce new vertices?

[bagnell]

We always introduce new vertices.

[ggetz]

Closing this. Having this roadmap is no longer very useful for tracking issues or enhancements with Geometry and Appearances. Thanks!