Process rendered CLUTs on the GPU

[unknownbrackets]

Unfortunately, this is not ready for merge yet. There are some issues.

Improvements:

• OpenGL and D3D9 upload indexed textures correctly.
• OpenGL and D3D9 properly processes CLUTs for regular textures - fixes Brave story - graphics glitches (debugged) #6754.

Not working:

• Rendered CLUT with rendered texture -> colors are totally wrong, but shape is right in the only testcase I have.

I debugged through D3D9 with debugging enabled, and couldn't find anything. If I set the indexed texture, I get grey, so it does create that texture right. But it always renders transparent black... even if I change the shader.

As for the OpenGL rendered issue (Kurohyou 2 demo), it flickers the wrong colors, which is really weird. I'm thinking it's because I need to copy the framebuffer in LoadClut, e.g. maybe the game is drawing over it.

-[Unknown]

[unknownbrackets]

Hmm. Using a temp FBO for the CLUT affects Kurohyou, but makes it worse (mostly just black.) Also rachets down Brave Story's performance from ~620-650% to 550% (still better than downloading the CLUT)...

-[Unknown]

[daniel229]

The shadows show up agains,but still not right in Heroes' VS.

[daniel229]

The weird shadows is missing in silent hill shattered memories.

[daniel229]

Almost 50% speed down under water in ikuze gen san

[daniel229]

FF Type 0 missing shadows.

[unknownbrackets]

Hmm. Obviously I impacted render-to-texture in some way I did not intend. Can you bisect?

-[Unknown]

[daniel229]

Heroes' VS， silent hill shattered memories， FF Type 0 are affected by unknownbrackets@86fb081

slow down in ikuze gen san is affected by unknownbrackets@17675ca

[daniel229]

White screen, even slow and memory leaking in Sonic rivals 2,up to 2GB ram usage and increasing.
Also affected by unknownbrackets@86fb081 and unknownbrackets@17675ca

[unknownbrackets]

Duh, I didn't consider that obviously clutRenderAddress_ could be 0 if I set it to fb_address. Arg.

-[Unknown]

[unknownbrackets]

That fixes that problem, I think, but not any of the problems listed in the first post...

-[Unknown]

[daniel229]

Still memory leaking.

[unknownbrackets]

Oops, see it now. Should be fixed as well.

-[Unknown]

[daniel229]

Yeah,memory leak is fixed.
another games Burnout Dominator gets red artifact.

59:14:273 idle0        I[G3D]: GLES\ShaderManager.cpp:153 Linked shader: vs 91 fs 92
59:17:512 idle0        W[G3D]: Common\FramebufferCommon.cpp:781 Block transfer download 04000000 -> 04170000
59:17:518 idle0        I[G3D]: GLES\ShaderManager.cpp:153 Linked shader: vs 16 fs 94
59:17:520 idle0        I[SCEGE]: GLES\Framebuffer.cpp:682 Resizing FBO for 00000000 : 512 x 272 x 3
59:17:523 idle0        I[G3D]: GLES\ShaderManager.cpp:153 Linked shader: vs 42 fs 96
59:17:524 idle0        W[G3D]: Common\FramebufferCommon.cpp:830 Block transfer upload 04170000 -> 04000000
59:17:526 idle0        W[G3D]: GLES\TextureCache.cpp:331 Render to texture with different formats 3 != 0
59:17:526 idle0        I[SCEGE]: Common\FramebufferCommon.cpp:417 Creating FBO for 00176000 : 512 x 272 x 0
59:17:527 idle0        E[G3D]: GLES\TextureCache.cpp:389 4 and 8-bit CLUT format not supported for framebuffers
59:17:527 idle0        W[G3D]: GLES\TextureCache.cpp:405 Render to texture with incompatible formats 5 != 3 at 00000000
59:17:527 idle0        W[G3D]: GLES\StateMapping.cpp:318 Unsupported RGB mask: r=ff g=1f b=07
59:17:552 idle0        I[SCEGE]: GLES\Framebuffer.cpp:682 Resizing FBO for 00088000 : 512 x 272 x 3
59:17:744 idle0        I[SCEGE]: GLES\Framebuffer.cpp:1885 Decimating FBO for 00000000 (480 x 272 x 3), age 6
59:17:777 idle0        I[SCEGE]: GLES\Framebuffer.cpp:1885 Decimating FBO for 00088000 (480 x 272 x 3), age 6
59:18:317 idle0        I[G3D]: GLES\ShaderManager.cpp:153 Linked shader: vs 16 fs 101

[unknownbrackets]

Does it improve if you change this:

        } else if (entry->format == GE_TFMT_CLUT8 || entry->format == GE_TFMT_CLUT4) {
            ERROR_LOG_REPORT_ONCE(fourEightBit, G3D, "4 and 8-bit CLUT format not supported for framebuffers");
        }

To:

        } else if (entry->format == GE_TFMT_CLUT8 || entry->format == GE_TFMT_CLUT4) {
            ERROR_LOG_REPORT_ONCE(fourEightBit, G3D, "4 and 8-bit CLUT format not supported for framebuffers");
            return;
        }

I wonder if this is trying to use 4/8 bit...

-[Unknown]

[daniel229]

Yes, the glitch is gone.
It seems it has to return 0 or 1,otherwise complie failed.return 0 or 1 did not help the glitch.

[unknownbrackets]

In the last commit in TextureCache.cpp, what if you change:

        VirtualFramebuffer *clutVfb = nullptr;
        for (size_t i = 0, n = fbCache_.size(); i < n; ++i) {
            auto clutFramebuffer = fbCache_[i];
            if (clutFramebuffer->fb_address == clutRenderAddress_) {
                clutVfb = clutFramebuffer;
            }
        }

To:

        VirtualFramebuffer *clutVfb = nullptr;
        for (size_t i = 0, n = fbCache_.size(); i < n; ++i) {
            auto clutFramebuffer = fbCache_[i];
            if (clutFramebuffer->fb_address == clutRenderAddress_ && clutFramebuffer->drawnFormat == clutFormat) {
                clutVfb = clutFramebuffer;
            }
        }

-[Unknown]

[daniel229]

Still no change.

[hrydgard]

This one will be nasty to rebase :)

[unknownbrackets]

Yeah, I'll look at it later. I guess it might end up being faster on Vulkan?

-[Unknown]

[hrydgard]

Depends on how it's implemented but I'm sure it's possible to get it to perform well.

[finansar222]

/ *สร้าง hash แบบเดียวโดยใช้ข้อมูลและโหมดที่ระบุ
ข้อมูลและข้อมูลที่สอดคล้องกันต้องเป็นทวีคูณของ 0x10
cryptkey เป็น NULL สำหรับ savedata * /
int build_hash ( unsigned char * output,
unsigned char * data,
unsigned int len,
unsigned int alignedLen,
โหมดint ,
unsigned char * cryptkey)

[finansar222]

/ *สร้าง hash แบบเดียวโดยใช้ข้อมูลและโหมดที่ระบุ
ข้อมูลและข้อมูลที่สอดคล้องกันต้องเป็นทวีคูณของ 0x10
cryptkey เป็น NULL สำหรับ savedata * /
int build_hash ( unsigned char * output,
unsigned char * data,
unsigned int len,
unsigned int alignedLen,
โหมดint ,
unsigned char * cryptkey)

[ghost]

This pr can help burnout dominator and brave story lag issue?

[hrydgard]

It can help some things yes (though I don't think Burnout is one of them), but it had issues and is now very old - it'll need quite some work to bring it up to date so it can be merged.

[unknownbrackets]

I think if anything the approach will change significantly. I'm going to finally close this.

-[Unknown]