Don't reproject Web Mercator imagery tiles unnecessarily #4339

kring · 2016-09-19T06:09:12Z

Previously, Cesium would reproject all Web Mercator imagery tiles to Geographic on load. This had two costs:

It took some time (not much, probably).
It introduced some blurring in the imagery because destination pixels were not aligned with source texels.

And then, of course, Cesium would reproject those Geographic images onto the globe with the normal 3D perspective projection. So effectively, Cesium reprojected imagery tiles twice.

After this PR, Cesium includes Web Mercator texture coordinates with the terrain tiles, so Web Mercator tiles can be projected directly onto the screen without going via Geographic in most cases. Some notes:

The imagery looks a bit sharper after this change.
I can't think of any reason that interpolating texture coordinates linearly in Web Mercator space is any less accurate than interpolating in Geographic space. In fact I think interpolating in Web Mercator should be a bit better because it's conformal-ish.
Terrain meshes are now bigger; there's a an extra 12-bit compressed float per vertex, which currently means an extra whole float per vertex. I spent way more time than I should have trying to avoid this. I thought I could eliminate the height attribute in most cases, especially when using 3D only. And I wrote some really hairy code to support this (in the terrain3 branch if you want to gaze upon the horror). But in the end it was just too hard and I gave up.
The Web Mercator coordinates are currently only used for terrain tiles that fall entirely within the Web Mercator bounds (+/- ~85 degrees latitude). For tiles that cross that line, we still reproject to geographic. We could avoid this by making sure that our terrain meshes always include vertices at +/- 85 degrees latitude (i.e. add them when we create the mesh) so a triangle never crosses that line. Triangles that cross the Web Mercator boundary cause problems when interpolating texture coordinates for fragments.

Its pretty hacky right now, but working.

It looks terrible in current versions of Bing Maps.

pjcozzi · 2016-09-20T01:55:44Z

The imagery looks a bit sharper after this change.

Do you have a nice before/after (maybe with an image diff if useful) we could use for twitter and the release blog?

pjcozzi · 2016-09-20T01:57:54Z

Terrain meshes are now bigger; there's a an extra 12-bit compressed float per vertex, which currently means an extra whole float per vertex...

Ouch. @bagnell and I will brainstorm since the trend is to use less memory, not more. Maybe it is unavoidable. Maybe we can optimize something else. Maybe this is the best we can do.

pjcozzi · 2016-09-20T02:00:03Z

CHANGES.md

@@ -3,6 +3,7 @@ Change Log

 ### 1.26 - 2016-10-03

+* Removed an unnecessary reprojection of Web Mercator imagery tiles to the Geographic projection on load.  This should improve both visual quality and performance slightly.


Are you sure we can make the performance claim in practice? In the steady state, this is probably worse.

pjcozzi · 2016-09-20T02:01:28Z

Source/Core/Rectangle.js

+     * @param {Rectangle} [result] The object onto which to store the result.
+     * @returns {Rectangle|undefined} The modified result parameter, a new Rectangle instance if none was provided or undefined if there is no intersection.
+     */
+    Rectangle.simpleIntersection = function(rectangle, otherRectangle, result) {


Add to CHANGES.md please.

pjcozzi · 2016-09-20T02:03:03Z

Source/Core/TerrainEncoding.js

+
+            stride = (numCompressed0 + numCompressed1) * sizeInBytes;
+
+            return [


Did you run this through the formatter?

I don't think we're using the same editor, so running it through "the formatter" probably doesn't make sense. But I moved the open curly to the same line as the open square bracket and stuff.

pjcozzi · 2016-09-20T02:06:11Z

Specs/Scene/ImageryLayerSpec.js

@@ -156,7 +156,7 @@ defineSuite([
        });
    }

-    it('reprojects web mercator images', function() {
+    it('reprojects web mercator images when necessary', function() {


Are any other new unit tests needed, e.g., request completely inside +/- ~85 latitude and completely outside?

pjcozzi · 2016-09-20T02:09:29Z

The Web Mercator coordinates are currently only used for terrain tiles that fall entirely within the Web Mercator bounds (+/- ~85 degrees latitude). For tiles that cross that line, we still reproject to geographic. We could avoid this by making sure that our terrain meshes always include vertices at +/- 85 degrees latitude (i.e. add them when we create the mesh) so a triangle never crosses that line. Triangles that cross the Web Mercator boundary cause problems when interpolating texture coordinates for fragments.

This could be done at runtime with quick reject tests, right? Do you think that would be cleaner than the current approach, which is...complicated. Or are you suggesting a breaking terrain spec change and doing this offline? The later is not out of the question, it would just have to be longer-term.

pjcozzi · 2016-09-20T02:09:49Z

@bagnell can you please also review this when you are available?

kring · 2016-09-21T03:23:25Z

Ouch. @bagnell and I will brainstorm since the trend is to use less memory, not more. Maybe it is unavoidable. Maybe we can optimize something else. Maybe this is the best we can do.

Here are some ideas:

Include the vertical geographic and/or web mercator texture coordinate, but not both unless we actually have two different imagery layers with two different projections. Aside from the complexity of managing optional vertex attributes like this, we also need to worry about other things that use the texture coordinates, like the water effect.
Eliminate the height. It's used for upsampling, and it's used in 2D and CV. We could handle the upsampling case by storing the heights separately (they don't need to be sent to the GPU). And of course 2D and CV don't matter if you're using the 3D-only option. But this is a lot of complexity and most applications won't benefit because they don't disable 2D and CV. We could at least let every app get some benefit by creating meshes for 2D and CV on the CPU (when displaying those views) rather than using vertex shader displacement.

kring · 2016-09-21T03:24:18Z

We could at least let every app get some benefit by creating meshes for 2D and CV on the CPU (when displaying those views) rather than using vertex shader displacement.

Or maybe by using a separate buffer for the heights when rendering in this view.

kring · 2016-09-21T03:25:41Z

This could be done at runtime with quick reject tests, right? Do you think that would be cleaner than the current approach, which is...complicated. Or are you suggesting a breaking terrain spec change and doing this offline? The later is not out of the question, it would just have to be longer-term.

I think it can be reasonably done at runtime. For heightmap terrain it's easy, for quantized mesh it's a bit of work, but not too hard. I'm not sure the overall complexity would go down here though. ;)

kring · 2016-09-21T07:27:41Z

Before:

After:

Diff (used https://huddle.github.io/Resemble.js/):

It's most obvious in contrasty bits, so the labels mostly in these screenshots. Save the two images and flip back and forth between them to really see it. On contrastier maps (like Bing Maps Roads for example), lots of map features look noticeably sharper. Overall, it's admittedly fairly subtle, though.

kring · 2016-09-22T05:27:30Z

Ok I think this is ready. Of course, it's up to you guys if you think the improvement is worth the extra float. I tried and failed to measure any performance or memory usage difference with this change. Actually, the memory usage of the GPU process in Chrome was consistently 10 meg (out of 500+!) lower after this change, possibly just because there's less texture creation now and perhaps Chrome doesn't clean up old ones very aggressively?

I'd be interested in the options I mentioned above for eliminating the height or the two sets of texture coordinates in order to reduce the amount of vertex data, but it's not straightforward and I think it's more important to do some other things first, like improve the tile load process.

denverpierce · 2016-09-22T16:15:40Z

Gif of both versions:

denverpierce · 2016-09-27T15:04:34Z

I modified the Path example to fly over the continental U.S. with the camera pointed straight down to test performance. I ran MSI Afterburner in the background to watch results, keeping as many other things constant as I could (no other windows open, opening from an empty chrome, etc.).

VRAM usage was ~10mb lower on the new version, but RAM usage was ~100mb higher. GPU and CPU usage was about the same.

1.25:

noReproject:

pjcozzi · 2016-10-15T15:12:03Z

@bagnell could you please evaluate this at the bug bash?

bagnell · 2016-10-19T13:36:56Z

This looks good to me. For the extra float, I'll add the ideas from @kring above to an issue.

pjcozzi · 2016-10-19T13:55:20Z

Thanks again @kring. I moved the CHANGES.md update to 1.27 in master.

kring added 22 commits August 1, 2016 21:35

Switch to depth first traversal, add SSE to Sandcastle example.

294a83f

Traverse and load near-to-far.

9ab3fd1

Remove manual tile sorting code.

918c59a

Don't reproject web mercator to geographic unnecessarily.

ca333b3

Its pretty hacky right now, but working.

Remove default Bing Maps gamma correction.

0fd3f21

It looks terrible in current versions of Bing Maps.

Fix a bug that prevented zooming past the terrain level.

bcf6f81

Change max SSE to 4/3, add wireframe option to Terrain example.

e2a4119

Fix heightmap based terrain.

e7c65b6

Add clarifying comment.

3cf3faa

Fix tiles with 12 bit attributes.

fb269ad

Fix quantized mesh terrain with normals.

205a043

Remove TODOs.

8eb8e96

Cleanup.

0c09753

Revert changes accidentally pulled from 3d-tiles.

7195f28

Merge remote-tracking branch 'origin/master' into noReproject

acdf984

Fix computation of veryCloseX and veryCloseY.

bd42b71

Fix test failure.

5f1888b

Merge remote-tracking branch 'origin/master' into noReproject

ce0e458

Remove unneeded changes.

6b04032

Update CHANGES.md.

8480d01

Fix jshint warning.

0534ee6

Y -> T

bd6a1f4

pjcozzi reviewed Sep 20, 2016

View reviewed changes

Merge remote-tracking branch 'origin/master' into noReproject

1bbe7ca

kring added 6 commits September 22, 2016 12:26

Update CHANGES.md, fix formatting.

091bca3

Fix formatting better.

783ea9a

Merge remote-tracking branch 'origin/master' into noReproject

5dd34a3

Add some specs.

1f22ed7

Tweak wording of CHANGES.md.

9b79102

Fix jshint warning.

23dc61d

Merge remote-tracking branch 'origin/master' into noReproject

32c4117

kring mentioned this pull request Oct 13, 2016

Smeared imagery on some mobile devices #2541

Open

pjcozzi added the bug bash label Oct 15, 2016

bagnell merged commit a8f0d0f into master Oct 19, 2016

bagnell deleted the noReproject branch October 19, 2016 13:37

bagnell mentioned this pull request Oct 19, 2016

Reduce terrain memory usage #4447

Open

This was referenced Oct 20, 2016

Fix ShadowMap NaNs #4470

Merged

Terrain Shadows Broken #4479

Closed

pjcozzi mentioned this pull request Nov 10, 2016

EllipsoidTerrainProvider with WebMercatorTilingScheme #4610

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Don't reproject Web Mercator imagery tiles unnecessarily #4339

Don't reproject Web Mercator imagery tiles unnecessarily #4339

kring commented Sep 19, 2016

pjcozzi commented Sep 20, 2016

pjcozzi commented Sep 20, 2016

pjcozzi Sep 20, 2016

pjcozzi Sep 20, 2016

pjcozzi Sep 20, 2016

kring Sep 22, 2016

pjcozzi Sep 20, 2016

pjcozzi commented Sep 20, 2016

pjcozzi commented Sep 20, 2016

kring commented Sep 21, 2016

kring commented Sep 21, 2016

kring commented Sep 21, 2016

kring commented Sep 21, 2016

kring commented Sep 22, 2016

denverpierce commented Sep 22, 2016

denverpierce commented Sep 27, 2016

pjcozzi commented Oct 15, 2016

bagnell commented Oct 19, 2016

pjcozzi commented Oct 19, 2016

		@@ -3,6 +3,7 @@ Change Log

		### 1.26 - 2016-10-03

		* Removed an unnecessary reprojection of Web Mercator imagery tiles to the Geographic projection on load. This should improve both visual quality and performance slightly.


		stride = (numCompressed0 + numCompressed1) * sizeInBytes;

		return [

Don't reproject Web Mercator imagery tiles unnecessarily #4339

Don't reproject Web Mercator imagery tiles unnecessarily #4339

Conversation

kring commented Sep 19, 2016

pjcozzi commented Sep 20, 2016

pjcozzi commented Sep 20, 2016

pjcozzi Sep 20, 2016

Choose a reason for hiding this comment

pjcozzi Sep 20, 2016

Choose a reason for hiding this comment

pjcozzi Sep 20, 2016

Choose a reason for hiding this comment

kring Sep 22, 2016

Choose a reason for hiding this comment

pjcozzi Sep 20, 2016

Choose a reason for hiding this comment

pjcozzi commented Sep 20, 2016

pjcozzi commented Sep 20, 2016

kring commented Sep 21, 2016

kring commented Sep 21, 2016

kring commented Sep 21, 2016

kring commented Sep 21, 2016

kring commented Sep 22, 2016

denverpierce commented Sep 22, 2016

denverpierce commented Sep 27, 2016

pjcozzi commented Oct 15, 2016

bagnell commented Oct 19, 2016

pjcozzi commented Oct 19, 2016