Remaining issues with depth testing precision/clamping

[thedax]

A lot of games still have depth problems, so let's gather them here for a meta-issue, but not ones that need OpenGL 3 (Jeanne, Danball, Saint Saiya), since that's a limitation of hardware.

Cars - Race O Rama: #4112
Kingdom Hearts: BBS (Android only? I never noticed any problems with it on Win32): #3058
Phantom Brave: #4064
Phantasy Star Portable 1: #3777
Phantasy Star Portable 2: #1788
Puyo Puyo Fever 2: #3663
NBA 2K13 (possibly? We need someone to look at it): #6603
Ratchet & Clank: #6105
Dream Club Portable (possibly?): #6497
Blazing Soul Accelate: #4665
Pilot Academy: No issue created yet (NPC planes can be seen crossing in front of buildings they should be going behind)

I've marked some games as "possibly?", as I'm not entirely sure if their problems are depth related or not. We can discuss that in the comments.

Note: This is not an issue to ask for fixes, ETAs, etc. I know some games are quite desirable to be fixed, but please, let's keep it on topic.

[unknownbrackets]

So, here's a few things about depth based on testing:

1. The minz/maxz registers apply a depth range test after the viewport and presumably other transforms. GLES doesn't support these.

   1.1. Note that this is not applied in throughmode.

   1.2. Note that this is applied in clearmode.

   1.3. Seems like it ignores the clipping flag.

2. Like with x and y, viewport z2 is the center, and viewport z1 is the range. We treat it this way in GLES. (working)

3. The viewport z1/z2 can actually exceed the range [0, 65535] although the current GLES code assumes otherwise.

   3.1. The center could be zero, and the range 65535. In this configuration, a vertex z of 0 would result in 0.0, and a vertex z of 1.0 would result in 65535. However, the GLES code would see this as glDepthRange(0 - 1.0, 0 + 1.0);. Window coordinates in GLES seem to be clamped to [0, 1] so this would incorrectly scale depth.

   3.2. As far as I can tell, it seems like dividing the viewport values by 65535.0f is correct.

   (working now)

4. Fixed point depth coordinates are meant to be in the scale of (max + 1) / 2, i.e. 128 and 32768.

   4.1. Note that this means it's not possible to reach the 1.0 value of the depth range, since inputs will be [-128, 127] and [-32768, 32767].

   4.2. Positions are handled essentially the same way (Final Fantasy 4 - dialog boxes have "holes" #4617.) We may be best off converting everything to floats in the vertex decoder (wonder if this will have perf or other accuracy impact...)

   (working now)

5. The PSP's depth buffer has a precision of only 16 bits.

   5.1. In GLES, we store usually as 24-bit fixed point, which has higher precision. This can cause trouble with >= tests and such where the result ought to have been rounded.

   5.2. I haven't yet figured out how rounding works, exactly.

6. Vertex depth coordinates outside [-1.0, 1.0] are always clipped afaict. GLES also has this behavior.

   6.1. Note that this means, even if you set a depth range like 32767.5f +/- 16384.0f, that won't make 1.5f a valid coordinate.

   6.2. This is also true in clearmode.

   6.3. Note that in throughmode, it clamps and seems to convert float -> integer [0, 65535].

7. Throughmode 0 - 65535 is meant to map similarly to -1 - 1 in non-through.

   7.1. I think this is handled by the ortho matrix (u_proj_through) in GLES.

   (working now)

8. The depth buffer is stored in PSP RAM swizzled (although + 6MB deswizzles it automatically on access.) If any games depend on this, it will be difficult to support.

   8.1. I think GLES (2 and 3) lack any support for downloading the depth buffer.

   8.2. Technically a game could download or upload a depth buffer, or even use it via a CLUT. We currently have hacks to attempt to handle depth buffer reuse (GLES 3+, GL 3+) only.

-[Unknown]

[hrydgard]

@unknownbrackets

1. We can simulate it with discard() but it will be VERY expensive. May also be able to do it with strange projection matrix hacks but not sure. I don't think ES has user clip planes, unfortunately, otherwise those would be an option.

8.1 You can actually download the depth buffer with glReadPixels, and you can theoretically upload it by drawing a quad and writing values to depth in the fragment shader. In ES 3.0 you can texture from depth buffers too.

Very nice post!

[unknownbrackets]

Right, desktop GL only can download AFAIK. We already do it in the GE debugger. But that doesn't help mobile...

Yeah, I was also thinking about trying to use the [-1, 1] clipping and then projecting with depth range trickiness, but I have a feeling there will be complications. But then I'm not sure we can avoid some messiness there if games are using greater magnitude z than 65535...

-[Unknown]

[hrydgard]

No, glReadPixels exists on mobile. It's glDrawPixels that doesn't.

Yeah, I think if we do that the depth buffer will be scaled wrong and if a game uses several minz/maxz settings per frame, results will be very unexpected...

[unknownbrackets]

I meant, it doesn't work for GL_DEPTH_COMPONENT on mobile, even on GLES 3.

-[Unknown]

[hrydgard]

Hm, I thought it did for some reason. Oh well, can work around it with a depth-texture-to-color pass in ES 3.0 at least...

[ofry]

Kingdom Hearts: BBS (Android only? I never noticed any problems with it on Win32): I got it in Win7 x64.

[ofry]

Kingdom Hearts: Birth by Sleep Final Mix

Now my issue with buffered rendering has solved after v1.0.1-769-gc4ea4e3

Please test it on Android smartphones...

[ofry]

#7877 Blazblue Continuum Shift 2 (ULUS10579)

[DerekTurtleRoe]

I have a similar issue as well on both v1.1.1 and the latest Git build. Here is the screenshot:

[unknownbrackets]

Random thought: if clipping is disabled, I wonder if depth can go outside the range? Like if I set the range to 32767.5 +- 1.0f. Or my 1.5f example above. Hmm.

-[Unknown]

[unknownbrackets]

I wonder if we could use GL_DEPTH_CLAMP in throughmode? I think I found it clamping depth.

-[Unknown]

[hrydgard]

GL_DEPTH_CLAMP is not universally supported though, would be nice to do without it..

[unknownbrackets]

So did some more testing, and found something interesting.

If I set the depth range to 32769.0 +- 32767.5, then that should set my range to [1.5, 65536.5], right? Except that any fragments outside [0, 65535] will get discarded based on the minz/maxz test.

Anyway, in this situation, z = -1 maps to 0002, and z = 1 gets discarded. As expected. But, z = (-32767.5 - 1) / 32767.5, or in other words a z less than -1, does NOT get clipped. Instead if maps to 0001.

So then I realized I had not sent a sceGuEnable(GU_CLIP_PLANES);. After sending that... unexpectedly, depth was not clipped at all. Instead, it was clamped. Now, all my draws (which are all flat by the way) are succeeding, changing the color, and setting depth values.

In other words, using the numbers above, z = -1 still maps to 0002, but z = 1 maps to FFFF now instead of getting discarded. Note again that I have maxz set to FFFF, I'll test with it different... confirmed, if I set maxz to FFFE, then z = 1 does get discarded even with GU_CLIP_PLANES on.

So... yeah.

-[Unknown]

[unknownbrackets]

Okay, so after thinking about this, here are my ideas:

1. When using a "full" depth precision range, take a slice 1/4 the size in the middle, by scaling and offsetting appropriately in the projection matrix. This means ~22 bits of precision which is still good.

2. If (clipEnable & 1) == 0, then we would set the depth range based on minz/maxz only, ignoring viewport. This will make GL clip for us. Viewport will get handled by altering the projection matrix.

3. If (clipEnable & 1) == 1, then we check minz and maxz.

   3.1. If minz > 0, we use that as the depth range. Otherwise, we use 0 (300% of our depth range away.)

   3.2. If maxz < 65535, we use that as the depth range. Otherwise, we use 1 (300% of our depth range away.)

   3.3. Values < -4 and > 4 would still be clipped, instead of clamped, but most games already seem to work, so this may be fine.

   3.4. Where supported, we can also use GL_DEPTH_CLAMP, which will work around the 3.3 issue.

4. Perhaps we add an ini-only setting to do per-fragment clamping. I'm thinking we can ignore it, and just let the values lay outside our "slice". Not accurate, but not slow either and will probably match most scenarios.

5. There's also a rounding issue. In general, [-0.5, 65536) seems like it might be allowed for standard [0, 65535] clamping. We could try adjusting the depth range slightly to account for this. Need more testing to be sure.

6. All this needs accounting for with depth rounding.

7. Pretty sure the software renderer doesn't handle some of this right either.

-[Unknown]

[unknownbrackets]

Valkyria Chronciles 2 is also affected by the rounding / clamping issue:

http://forums.ppsspp.org/showthread.php?tid=433&pid=116239#pid116239
#13462

-[Unknown]

[unknownbrackets]

I added some additional notes in #12058 based on further testing.

-[Unknown]

[ghost]

This still an issue?

[unknownbrackets]

I think the depth behavior on the PSP is mostly simulated now (just that some devices can't handle it), and working well on a range of devices. We still have some issues but they're separate already and I think the issues this was tracking are generally all fixed.

So, going to close it.

-[Unknown]