Importing huge texture (100 Mpx) as ETC2 causes excessive ram usage

[RevoluPowered]

Godot version:
3.1 - Current released version

OS/device including version:
Windows 10 Pro

Hardware info
Intel i7 6700 @ 3.4GHz
Asus Strix Radeon R9 390X
DDR4 24 GB RAM

Issue description:
Consumption of ram is extremely high, CPU and system hangs to the point browsers and mouse stop working when importing the large asset.

Expected results: import time should take longer since the size of the image is large, but 60~% system memory should not be consumed during import of a single image.

Steps to reproduce:

• started with a clean project.

1. Load sample image attached into project.
2. Make new 3D scene
3. Make new 3D sprite in scene
4. Use sample image supplied on 3D sprite.
   SAMPLE IMAGE: https://mega.nz/#!hz4iXAbb!oA0y9FswwJR1_90onAVXsqgjAe888dnsMMXm3n-s3AA
5. your editor should now hang, system freeze and ram will be consumed.

**Ram usage featured is the single large image from the minimal reproduction project **

Minimal reproduction project:
https://mega.nz/#!87oEFQSQ!AqgfSyddq4IE5YzU0gb4lzaSy2wCQcxVztT5GsVzgwk

[volzhs]

for more information, the png file has 13210x7430 dimension.

[OvermindDL1]

So that's 3926*1200 just to calculate it out, which is just shy of 400 megs worth of raw RGBA image data, definitely doesn't seem like it should take up near 12 gigs... o.O

[RevoluPowered]

Yes, I thought that haha. The PNG is hand painted by a guy who doesn't want to lower the resolution.

I know the image is huge.

Note: issue only happens when the item is used in 3D space - this triggers the re-import. I read briefly in the engine docs that this is expected for it to re-import for 3D prep, but wouldn't expect it to create so much ram unless it was making vertexes for each pixel or something insane.

[volzhs]

after a quick test with the attached image file, I got this result.

Compress mode - Lossless / Filter on / Mipmaps off : 190MB
Compress mode - Video RAM / Filter on / Mipmaps off : (while re-importing 9GB) -> 4.5GB
Compress mode - Video RAM / Filter on / Mipmaps on : (while re-importing 12GB) -> 6.4GB

need to wait for 40~70 seconds for re-importing it with using 100% of 8 core / 16 thread cpu.

[volzhs]

you can manually set compress mode to lossless while fixing it on import dock.

[RevoluPowered]

OK I understand the compress mode can be changed, but why is it consuming so much ram, doesn't seem right, regardless of the import settings?

[Calinou]

Make sure to disable ETC2 compression in the Project Settings if you only intend to export to desktop platforms. ETC2 is compressed using etc2comp, which is pretty slow compared to S3TC.

[RevoluPowered]

I'd like it if we could improve the ram usage so it doesn't eat my system memory like that though, it's not as if that should ever be expected behavior, I understand that etc2comp is used for porting to android, but this game will be used on android as it's one of our planned platforms for the game we are working on.

additionally, etc2comp looks like it's been abandoned(2-3 years no commits), perhaps its time that the component is replaced?

[volzhs]

There is a pending PR for it. #26886

[RevoluPowered]

Later on I'll test if turning this option off relieves ram usage, then if #26886 is merged I'll close the ticket.

Might not be today though

[RevoluPowered]

Just tested this out. Confirmed that it is fixed if etc2comp is disabled.

Anyone know when #26886 will be ready?

[RevoluPowered]

Same image, ram usage info - during re-import

[Anutrix]

Any updates.

[RevoluPowered]

Hi @Anutrix just spoke to @fire about this PR. We're not entirely sure when or if we will get any time to finish #26886

[jitspoe]

I wonder if this is at all related: #10059

[Zireael07]

Is this still the case after #37488 was merged?

[RevoluPowered]

I believe after ETC2 was merged this is resolved closing!