Godot 4 port

[Manik2607]

Is there going to be any Godot 4 port Any time soon?????

[Zylann]

Duplicate of #279.

It will eventually be ported when Godot 4 will be stable.

[Manik2607]

Duplicate of #279.

It will eventually be ported when Godot 4 will be stable.

It's very stable now btw.
🤗

[Zireael07]

"Stable" here means "no longer a beta"

[miketgv]

now is RC2

[Duroxxigar]

Guys - I'm pretty sure Zylann knows if Godot 4 is stable for them or not.

[Zireael07]

@Duroxxigar You misunderstand. "Stable" does not refer to whether it crashes or not. It refers to the final version of a program, the actual release, not alpha, not beta, nor candidate

[Zylann]

I gave a try at porting the plugin. There are a number of issues, but a big one is Godot 4.0 does not support 2D HDR viewports, which the plugin requires to edit heightmaps. Partial texture update is also no longer available. This will have to wait at least to Godot 4.1...

[ElsAr4e]

Should be no longer needed if you are using future releases of Godot 4?

godotengine/godot-proposals#6121 (comment)

[Zylann]

AFAIK terrain in Godot 4 is still planned to be plugins.

[ElsAr4e]

You're probably right ... godotengine/godot-proposals#6121 (comment)

[Megalomaniak]

There are a number of issues, but a big one is Godot 4.0 does not support 2D HDR viewports, which the plugin requires to edit heightmaps.

Perhaps it's worth rethinking the plugin architecture altogether for 4.0 and on wards tho? I mean aren't compute shaders a thing now? Admittedly that would mean vulkan only I suppose.

[Zylann]

The current brush system did not need to be rethought so soon, it was working quite well. I'm considering compute shaders in the future, but they aren't exactly enough here, because the brush system uses the renderer to draw/rasterize things. Also they still can't interact with renderer resources (they would require the heightmap to be uploaded twice, downloaded back and reuploaded for the renderer to get the changes, it's worse), and indeed that would mean Vulkan is required. I'm not keen in having to rewrite this stuff so soon for the third time just for a missing feature Godot is even supposed to have.

I wonder if instead 3D orthographic viewports could be used to keep the current logic working with less changes, if HDR works in 3D? But I'm not sure if the "No Blend" blending mode is supported there, which I rely on.

I'm still considering to continue porting when I have time, and eventually will create a branch with what could be made to work, then will see what to do about the rest (I might find other blocking issues)

Also speaking of rethinking architecture, I currently mostly work in C++ modules for large projects, and I wish I could do the same for this plugin. But even today GDExtension isn't good enough to me (bugs and missing features).

[Zylann]

Btw if someone knows/has time to implement 16-bit and 32-bit HDR viewports 2D rendering in Godot that would help too.
(Source code for reference https://github.com/godotengine/godot/blob/master/servers/rendering/renderer_rd/storage_rd/texture_storage.cpp#L2570)

Although it seems devs ideas are different, based on this PR and comment godotengine/godot#61667 (comment), not sure I understand what the plan is but it was said for 4.1.

[Zylann]

Since there doesn't seem to be plans to add back HDR in 2D in a way that can be used by the plugin, there is also this proposal: godotengine/godot-proposals#5272

The only other alternative right now is compute shaders, but that means a full rewrite [of the brush system and the generator], and really that means the editor will no longer support users who don't have Vulkan.

[ElsAr4e]

One question: What is different when it comes to this terrain editor: https://github.com/ozzr/godot_terrain ?
It's also based on GDScript?

And even works with GLES2.

Which terrain editor is better if you want to have high quality output (of course until now you can only use both in Godot 3)?
Yours or the other?

The other terrain editor (https://github.com/ozzr/godot_terrain) should be released for Godot 4?

[Zylann]

One question: What is different when it comes to this terrain editor: https://github.com/ozzr/godot_terrain ? It's also based on GDScript?

This one doesn't seem to be done using a plugin (I can't find any in the repo), it's probably done with an engine module and uses C++ to edit the terrain. I used C++ in the past too using GDNative but it had bugs and was also slower than shaders.

And even works with GLES2.

AFAIK the only way to make this work in GLES2 is to use an RG8 heightmap, which requires different shaders and algorithms to decode/encode heights (making stuff a bit slower as well), and cannot be filtered natively. I didn't choose it because a long time ago I didn't know GLES2 didnt support HDR textures, and it was easier to support RH/RF. Also in Godot 4 there is no GLES2, and HDR textures should be widely supported now (the choice of Godot in 2D is not really a hardware limitation I'm afraid).

Which terrain editor is better if you want to have high quality output (of course until now you can only use both in Godot 3)? Yours or the other?

This is quite subjective and depends a lot on your assets.

The other terrain editor (https://github.com/ozzr/godot_terrain) should be released for Godot 4?

I don't know, maybe. You may ask ozzr.

[Valeryn4]

Good day! Is there a port planned for godot 4 in the near future?

[Zylann]

I'm already attempting to port it. But as you may read in earlier posts, there are some blockers. It will require some degree of downgrade from what the editor could do in Godot 3.

[Valeryn4]

I'm already attempting to port it. But as you may read in earlier posts, there are some blockers. It will require some degree of downgrade from what the editor could do in Godot 3.

i found this plugin. This plugin works in godot 4 with two small fixes.
this plugin makes working with brushes more convenient.
I was thinking of taking it, rewriting it and using lod from the devmar guides.

[Zylann]

After doing some tests, I think I can carry on supporting heightmap editing by switching to the RGB8 format (before heightmaps were using the RH format). Filtering surprisingly works well. Heightmaps will then be a regular PNG:

Hopefully RGB8 allows to edit heightmaps with 2D viewports as long as no weird color management occurs, so it won't need too many changes and both Vulkan and GLES3 will be supported.

Switching to RGB8 will also solve #345 and #334, because it has higher precision than RH.

Ironically, it would also make it easier to support GLES2, however it is no longer available in Godot 4. That said, it could be backported if someone needs it.

An option may be added to the import menu in order to import EXR and old .RES heightmaps.

The absence of partial texture update isn't a blocker but will reduce performance significantly when editing large terrains. I can't tell yet by how much, will see when this part works again.

RGB8 will require conversion to float in order to build a heightmap collider, which can be very slow. It can be worked around by doing that conversion when the terrain is saved, and save the converted collider in addition to the heightmap so it can be loaded faster. One downside is that the game will be heavier due to the extra file.
Alternatively if format options are added, one could convert the heightmap to RF (float) in the first place, once they are done with editing. Due to absence of HDR viewports it wont be editable, but at least it will remove the need to convert at the expense of a bit more VRAM usage (since float is 4 bytes per pixels, while RGB8 is 3 bytes per pixel).
For those reasons it would still be really nice of Godot supported a compatible way to edit high-depth images on the GPU using shaders, or supporting RGB8 in collider data.

Once the plugin is ported, it would be interesting to introduce a new shader based on this technique: https://github.com/cdxntchou/IndexMapTerrain (thanks to outobugi for the reference) since it seems to solve a lot of limitations previous shaders have.

I opened a new godot4 branch. It isn't working yet, but I'm doing some progress.

[Zylann]

This is hitting hard once again: godotengine/godot-proposals#1943
I'm trying to port the part of the plugin that automatically packs Albedo+Bump+Normal+Roughness maps into imported textures. I was re-implementing Godot's texture importers to have that workflow complete, but porting this got even harder in Godot 4, too much unexposed stuff. Will have to give up on that and do it differently. Instead I may fallback on feeding Godot's stock importers with the kind of images and .import files they want (I had problems doing that in the past, but we'll see how it goes). This might affect usability a bit...
(if you only ever used Albedo and Normal without texture arrays it probably wont affect you) nevermind, it looks like it will be fine.

[Zylann]

You can try the ported version of the plugin in the godot4 branch: https://github.com/Zylann/godot_heightmap_plugin/tree/godot4
It might have some bugs remaining (often due to the dynamic nature of GDScript, stuff doesnt fail until code actually runs), if you find some you can create a new issue.

Some known issues:

• If you select a terrain, turn off the plugin and then close the scene, Godot will crash The editor crashes after turning off a plugin and closing the current scene godotengine/godot#75329
• After installing the plugin, you might need to restart Godot at least once to make sure it imports some resources (was already a problem in Godot 3. There are workarounds in place but who knows)
• There might be errors when alternating selection between terrain node and detail layer nodes: Godot errors/crashes after inspecting two different objects handled by the same plugin godotengine/godot#73650
• Edition performance might be slower than in Godot 3 due to missing features of Godot 4, however I haven't noticed yet.
• After importing textures using the import tool, sometimes all editor popup menus become unresponsive. Appears to be a Godot issue Context Menu stops capturing mouse input after some unknown trigger godotengine/godot#61758 (comment). Workaround is to use arrow and Enter keys, or restart the editor

[Valeryn4]

• Edition performance might be slower than in Godot 3 due to missing features of Godot 4, however I haven't noticed yet.

What kind of functions?
maybe I have a solution.

[MGilleronFJ]

What kind of functions? maybe I have a solution.

It was explained in earlier posts and a few places of this proposal godotengine/godot-proposals#6121

The removal of 2D HDR viewports usable for fast 32-bit GPU editing means the heightmap had to use a different format which requires encoding and decoding everywhere heights are used or modified, which isnt bad on GPU, but has to happen on CPU too in some cases (technically in Godot 3 there was already some encoding occurring on the CPU because it was using half-precision floats, but that conversion was built-in and occurred in C++ instead of GDScript).

Partial texture update is also not available so everytime the plugin modifies the terrain it has to upload the entire map instead of just the modified area.

[Redwarx008]

Edition performance might be slower than in Godot 3 due to missing features of Godot 4, however I haven't noticed yet.

When editing terrain above 2k, the stutter is obvious.

[Redwarx008]

I thought of a method as a workaround in case there is no partial update method for textures.
We can prepare a texture of the corresponding size for each lod block (cut from the original texture) or cut the original texture into a texture no larger than 1024, which can temporarily solve the performance problem.

[MGilleronFJ]

While cutting stuff in chunks would work, it's not as simple as it sounds. Cutting maps in chunks requires to handle painting on multiple chunks at a time (I also wonder if it requires to keep a copy of neighboring pixels on every chunk just to avoid filtering issues in shaders, especially when normals are computed at chunk borders!). Then other tools need to be aware of it (even the generator, which has the option to add the existing terrain to the generated one), then the details system needs to be aware of the trick as well, the API needs to reflect that by adding "tile coordinate" parameters etc... there cannot even be an "original texture" stored somewhere because it would have to get updated at the end regardless, and terrain needs to be able to render the same while you edit.
I thought of doing that for a while because it's a great idea for other reasons, but it's a lot of work that was supposed to come with new features and workflow differences, worth a major version, rather than a temporary in-editor workaround for a basic feature Godot was supposed to have.

[Hobart2967]

Nice work @Zylann ! Will try this out right now :)

[Hobart2967]

Works like a charm on godot4 <3

[Kinwailo]

You can use unpackUnorm4x8(floatBitsToUint(f)) and uintBitsToFloat(packUnorm4x8(rgba)) to cast between R32F and RGBA8. The data in Image.get_data() exact match, so you can directly use data.to_float32_array() to get the float data used in physical. If you want to support gles3, you can do manual pack/unpack.

here is my test code:

shader_type canvas_item;
render_mode blend_disabled;

uint packUnorm(vec4 rgba) {
    return uint( uint(rgba.r * 255.0) << 00u
               | uint(rgba.g * 255.0) << 08u
               | uint(rgba.b * 255.0) << 16u
               | uint(rgba.a * 255.0) << 24u );
}

vec4 unpackUnorm(uint v) {
    return vec4( float((v >> 00u) & 255u) / 255.0
               , float((v >> 08u) & 255u) / 255.0
               , float((v >> 16u) & 255u) / 255.0
               , float((v >> 24u) & 255u) / 255.0);
}

void fragment() {
	float v = texture(TEXTURE, UV).r;
	uint i = floatBitsToUint(v);
//	vec4 c = unpackUnorm4x8(i);
	vec4 c = unpackUnorm(i);
//	uint i2 = packUnorm4x8(c);
	uint i2 = packUnorm(c);
	float v2 = uintBitsToFloat(i2);
	COLOR = v == v2 ? vec4(1.0) : vec4(0.0);
}

[Zylann]

I can't use that because sampler filtering would destroy it in the cases filter is used, it would interpolate between RGBA components but because floats work differently it might be an issue

Note: your code isn't applicable for heights of this plugin, it encodes normalized values, while this plugin uses actual heights. Also it converts to vec4 for some reason, which isn't necessary. But floatBitsToUint may be usable with some adjustments.

[TokisanGames]

@Kinwailo Can you elaborate on what one would use this for? If the height data is 32F, why would I want it as RGB8A?

If you wish to discuss it on discord, you can join my server where Zylann, me, and 2 other devs of 4 separate terrain systems for Godot are under development. https://tokisan.com/discord

Thanks

[Kinwailo]

@TokisanGames Because godot4 don't support getting R32F texture from viewport for sculpting, it is a workaround until godot4 support R32F viewport texture.

[TokisanGames]

I see. So In the scenario where one is using the viewport to paint on the GPU in float you could instead paint in int. I thought that Zylann had found a workaround for hterrain. We are painting in float on the CPU in C++ for now.

Thanks for the suggestion.

[Zylann]

Maybe I'll give a try to RF later, with manual filtering in all the places that use viewports to edit the heightmap. RGB8 works well, but RF could be even better. Also I thought RGB8 took less space than RF, but this made me doubt, though it depends on the graphics API and OpenGL/Vulkan have it.
Saving it to disk will however require again to use .res uncompressed, which will actually take more memory than the current solution, or save it as .exr in such a way that it gets loaded back as RF and not RGBF

[Redwarx008]

Actually, in the vulkan port, any rgb is converted to rgba when rendering. So rgb8 can't save space compared to Rf

[Zylann]

I changed the plugin to work with RF (float 32-bit) heightmaps locally.

It works well, but Godot is still throwing a wrench at it:

It seems that reading the alpha channel of a transparent viewport with hint_screen_texture always returns 1.0, even if that viewport is set to never clear.
That means the terrain generator is unable to compute normalmaps, cannot perform erosion or any other iterative process. It needs to read the last written heights, and it cannot do that if only RGB can be read and not Alpha, since all 4 components are needed to encode a float...

Now, there is a potential workaround, but it's ugly, slow and inconvenient: download the viewport texture at every pass, then re-upload it as an ImageTexture, and pass it as shader uniform instead. Shaders and scripts have to be hacked, it sucks.
The issue doesn't seem to have been reported in Godot yet, the closest I could find was godotengine/godot#35230 but seems slightly different.
Another workaround could be to rewrite the code with two ping-pong viewports, instead of just one...

Made an issue: godotengine/godot#78207

[Zylann]

I pushed the change to the godot4 branch. Now the terrain uses 32-bit floats.
If you created terrains with the former RGB8 format, it will automatically convert.

[Ophiolith]

Hey, just a quick heads-up, Godot 4.1 breaks the current Godot 4 branch of the editor. Enabling the plugin generates the following errors:

res://addons/zylann.hterrain/tools/preview_generator.gd:11 - Parse Error: The function signature doesn't match the parent. Parent signature is "_generate(Resource, Vector2i, Dictionary) -> Texture2D".
res://addons/zylann.hterrain/tools/preview_generator.gd:20 - Parse Error: The function signature doesn't match the parent. Parent signature is "_generate_from_path(String, Vector2i, Dictionary) -> Texture2D".
res://addons/zylann.hterrain/tools/plugin.gd:110 - Invalid call. Nonexistent function 'new' in base 'GDScript'.

[Zylann]

Aaaand that pretty much means it won't work in Godot 4.0 anymore if I fix this :(
It's not the first time it happens unfortunately... that's a compatibility breakage I cannot work around.

[Ophiolith]

@Zylann I love Godot, but the unpredictability of whether point updates will break plugins is a huge pain point tbh, and must be a nightmare for plugin maintainers! No worries eitherway, I'm fairly sure I should be able to roll back to 4.0 without too much issue, but thought it was worth letting you know. Thanks for all your hard work on the plugin <3

[Zylann]

I pushed the change to make it work in Godot 4.1.

[Ophiolith]

Welp, didn't expect an update so quickly! Works like a charm. Much appreciated!

[Redwarx008]

Now godot4.2beta already has 2d hdr, no need to convert to rgba8

[Zylann]

I switched to floats already, and I didnt need HDR for that