Automatic mesh upgrade tool breaks ext_resource reference and embeds mesh data in scene file

[lostminds]

Godot version

4.2beta3

System information

macOS 13.6.1

Issue description

After running the new automatic mesh conversion tool choosing to "Upgrade and save", one of my scene files has been broken. It appears that something has gone wrong importing one (or both) of the imported meshes the scene references:

[ext_resource type="PackedScene" uid="uid://bq61epp0aqoyf" path="res://Models/Blob.glb" id="2_53eok"]
[ext_resource type="PackedScene" uid="uid://c5h71rygd5fkg" path="res://Models/BlobConnectionCover.dae" id="4_dnxis"]

These are referenced in the scene as nodes with some overrides, like:

[node name="BlobMesh" parent="Model" instance=ExtResource("2_53eok")]
transform = Transform3D(0.8, 0, 0, 0, 0.8, 0, 0, 0, 0.8, 0, -0.425, 0)

[node name="Blob" parent="Model/BlobMesh" index="0"]
transform = Transform3D(1, 0, 0, 0, 1, 0, 0, 0, 1, 0, 0.5, 0)
material_override = SubResource("ShaderMaterial_13t46")
instance_shader_parameters/Glow = 0.0

[node name="ConnectionCoverLeft" parent="Model" instance=ExtResource("4_dnxis")]
visible = false

[node name="ConnectionCover" parent="Model/ConnectionCoverLeft" index="0"]
transform = Transform3D(0.8, 0, 0, 0, 0.8, 0, 0, 0, 0.8, 0, -0.025, 0)
material_override = SubResource("ShaderMaterial_in6u2")
cast_shadow = 0
instance_shader_parameters/Glow = 0.0

After the conversion tool, while the references in scene nodes remain unchanged (except for new resource ids), the ext_resources referencing the meshes are now empty:

[ext_resource type="PackedScene" path="" id="9_yxliy"]
[ext_resource type="PackedScene" path="" id="10_yiqah"]

These references with no uids and no paths, cause the scene to fail loading due to these broken references, and of course they no longer reference the intended imported meshes.
Additionally, the converter tool (or importer?) has instead tried to embed the entire meshes and the packed scene in the scene file, in the form of big subresources like:

[sub_resource type="ArrayMesh" id="ArrayMesh_g37c8"]
resource_name = "Blob_Sphere"
_blend_shape_names = PackedStringArray("Left", "Right", "Up", "Down")
_surfaces = [{
"aabb": AABB(-0.797799, -0.497474, -0.797602, 1.59541, 0.995045, 1.59541),
"attribute_data": PackedByteArray(0, 0, 64, 63, 240, 255, 12 
...

Abbreviated here since this seems to contain the full mesh data for the model, increasing the scene file size from 16KB to 725KB. There's then an embedded packed scene, which I guess is supposed to be a replacement for the external resource:

[sub_resource type="PackedScene" id="PackedScene_b5yox"]
_bundled = {
"conn_count": 0,
"conns": PackedInt32Array(),
"editable_instances": [],
"names": PackedStringArray("Blob2", "Node3D", "Blob", "mesh", "skeleton", "blend_shapes/Down", "blend_shapes/Left", "blend_shapes/Right", "blend_shapes/Up", "MeshInstance3D"),
"node_count": 2,
"node_paths": [],
"nodes": PackedInt32Array(-1, -1, 1, 0, -1, 0, 0, 0, 0, 9, 2, -1, 6, 3, 0, 4, 1, 5, 2, 6, 2, 7, 2, 8, 2, 0),
"variants": [SubResource("ArrayMesh_g37c8"), NodePath(""), 0.0],
"version": 3
}

But this is then not referenced anywhere in the scene file that I can find. So if it's intended as a fallback, it's not used? Also interestingly, I see now that it contains a "Blob2" node name, that looks like the import name change issue from #83429 perhaps this could be what is causing the automatic importer to fail as well (but in a more hard to fix way)?

So, to sum it up, the converter seems to have failed in some way, resulting in the expected ext_resource type="PackedScene" reference being left empty, instead of getting the new converted mesh reference. On top of that it seems to have instead embedded all the mesh and imported PackedScene data into the scene file instead, but then not used this embedded data.

It also appears that the process failed to convert or save the meshes, as running the scene (after restoring the scene file to its pre-broken state) still causes the warnings from the renderer about converting meshes from an old format at runtime.

Steps to reproduce

Unfortunately I've been unable to find a way to trigger the import tool again, as it seems to only run once. But perhaps there's a way to do this? I noticed that it did output errors when it ran, but at the time I did not make note of them as I expected I would be able to try again if it failed and check the errors then if I would need to.

But I've supplied the complete scene file (before and after conversion-versions) along with the offending mesh files and their import file settings, in the hope that these will be helpful in tracking this issue down. The scene references a lot of other resources, but these I'm pretty sure are irrelevant to this issue. But in order not to remove anything that would actually be relevant to this bug I've left the files as is instead of trying to "clean them up".

I've also tried just making a new project in 4.2beta3 and importing the mesh there, which worked fine. So it seems to be something related to the converting process somehow.

Minimal reproduction project

While I can't share the complete project you can access the relevant files as described above here: MeshConverterIssueFiles.zip

[clayjohn]

What version were you upgrading from? i.e. was this a 4.0.x project or a 4.1.x project?

[lostminds]

This project was last saved in 4.2b2 I think, and I had it working after fixing some imports that were broken due to the naming issue #83429 so the meshes should all already have been updated since I thought I had re-imported them all already (I went through all my meshes and hit re-import, and then fixed the scenes references that were broken due to name changes).

However, there was still some mesh that I was never able to find that wasn't converted, as I still got the old mesh format conversion warning at runtime. So this is probably the mesh the automatic tool also found. I guess it could be that the offending mesh was one of these, and that the re-import just silently failed to convert the mesh when I pressed the re-import button, so it was still there in the old format and now it has failed again with this automatic tool?

Is there a way to trigger the conversion tool again by re-setting some hidden project flag or such? If I could get it to run again I could report any errors it would print out and see if it gets the same result when run again.

[clayjohn]

@lostminds The conversion tool only runs if a mesh that needs to be converted is detected. All the tool does is open every scene and save it again. It doesn't do anything fancy.

You can force it to run again by opening your project in 4.1.x and reimporting a single mesh. But I don't recommend that as there may be other unintended side effects.

I'm seeing if I can recreate the conditions that led to this issue in the first place. I'm wondering if it has to do with having the scene instantiated with "editable" children. I'm testing that theory now

[lostminds]

conversion tool only runs if a mesh that needs to be converted is detected

But only once, or each time the project is loaded? If I open the project now I do not get the prompt to run the tool, even if the scene has been reverted to it's pre-converted state again. And I still get the warning (twice)

W 0:00:00:0442   fix_surface_compatibility: A surface uses an old surface format and needs to be upgraded. The upgrade happens automatically at load time every time until the mesh is saved again or re-imported. Once saved (or re-imported), this mesh will be incompatible with earlier versions of Godot.
  <C++ Source>   servers/rendering_server.cpp:2062 @ fix_surface_compatibility()

when I run the project, which I guess means the mesh hasn't been converted/saved?

[clayjohn]

But only once, or each time the project is loaded? If I open the project now I do not get the prompt to run the tool, even if the scene has been reverted to it's pre-converted state again. And I still get the warning (twice)

Hmmm, that really shouldn't be happening. If somehow a mesh was missed then it should re-open the popup. I have a theory about what might cause this to happen. Let me validate and make a quick PR

The popup will run each time the project is opened, the setting to disable warnings does not persist across runs.

edit: I can't quite figure this out. The warning should not be printed if you click "Upgrade & Re-save" as it disables warnings for the rest of the time that the engine is running.

[warriormaster12]

Not sure if I should create a separate issue but restarting an editor after upgrading meshes causes the popup to reappear.

On top of that upgrading seems to break animation libraries (for this probably I should open a new issue)

[warriormaster12]

2023-10-26.14-43-23.mp4

[viksl]

I was here just to report this too. The pop up shows every time, does it convert the meshes and it shouldn't pop up or does it not save it and that's why it pops up everytime, or what? It seems to keep converting the same meshes everytime.

[lostminds]

edit: I can't quite figure this out. The warning should not be printed if you click "Upgrade & Re-save" as it disables warnings for the rest of the time that the engine is running.

The warnings appear when I run the project, not in the editor, so it's not so strange that the warnings appear. I just meant it's an indication that the meshes haven't been converted (or saved), as otherwise the warning wouldn't be shown when the game runs and uses these meshes. This lack of saving the converted meshes (in case of some failure?) could I guess also be why others are reporting the tool popping up each time they load the project, trying to convert the same meshes again.

[clayjohn]

edit: I can't quite figure this out. The warning should not be printed if you click "Upgrade & Re-save" as it disables warnings for the rest of the time that the engine is running.

The warnings appear when I run the project, not in the editor, so it's not so strange that the warnings appear. I just meant it's an indication that the meshes haven't been converted (or saved), as otherwise the warning wouldn't be shown when the game runs and uses these meshes. This lack of saving the converted meshes (in case of some failure?) could I guess also be why others are reporting the tool popping up each time they load the project, trying to convert the same meshes again.

Oh okay, thanks for that clarification. It would be really helpful to know what meshes are being missed. But the error message can only detect the source of the mesh in some cases. And it seems like the case you are bumping up against isn't one of those cases

[chrisl8]

When I open a 4.1 project in the latest build from master, it imports a lot of things on the first run, but also I never stop getting these messages on every run, even if I close and open the editor many times, these messages spam my debug log every time I run the game.

This happens even if I just clone my game right off of github and edit it in 4.2 immediately, so that there is no .godot folder.

This is the repo, if it helps at all https://github.com/chrisl8/space-game

It seems related to this issue I think.

Here are the individual errors. The first one only once, and then the others repeat hundreds of times:

W 0:00:00:0786   RenderingServer::fix_surface_compatibility: A surface uses an old surface format and needs to be upgraded. The upgrade happens automatically at load time every time until the mesh is saved again or re-imported. Once saved (or re-imported), this mesh will be incompatible with earlier versions of Godot.
  <C++ Source>   servers\rendering_server.cpp:2062 @ RenderingServer::fix_surface_compatibility()

W 0:00:00:0847   RenderingServer::fix_surface_compatibility: A surface of res://spaceship/room_03/room_03.tscn::ArrayMesh_2f21o uses an old surface format and needs to be upgraded. The upgrade happens automatically at load time every time until the mesh is saved again or re-imported. Once saved (or re-imported), this mesh will be incompatible with earlier versions of Godot.
  <C++ Source>   servers\rendering_server.cpp:2064 @ RenderingServer::fix_surface_compatibility()

W 0:00:00:0857   RenderingServer::fix_surface_compatibility: A surface of res://spaceship/room_03/room_03.tscn::ArrayMesh_u6f87 uses an old surface format and needs to be upgraded. The upgrade happens automatically at load time every time until the mesh is saved again or re-imported. Once saved (or re-imported), this mesh will be incompatible with earlier versions of Godot.
  <C++ Source>   servers\rendering_server.cpp:2064 @ RenderingServer::fix_surface_compatibility()

W 0:00:00:0865   RenderingServer::fix_surface_compatibility: A surface of res://spaceship/hallway_-1/Hallway_-1.tscn::ArrayMesh_it8u3 uses an old surface format and needs to be upgraded. The upgrade happens automatically at load time every time until the mesh is saved again or re-imported. Once saved (or re-imported), this mesh will be incompatible with earlier versions of Godot.
  <C++ Source>   servers\rendering_server.cpp:2064 @ RenderingServer::fix_surface_compatibility()

W 0:00:00:0866   RenderingServer::fix_surface_compatibility: A surface of res://spaceship/hallway_-1/Hallway_-1.tscn::ArrayMesh_i3s2f uses an old surface format and needs to be upgraded. The upgrade happens automatically at load time every time until the mesh is saved again or re-imported. Once saved (or re-imported), this mesh will be incompatible with earlier versions of Godot.
  <C++ Source>   servers\rendering_server.cpp:2064 @ RenderingServer::fix_surface_compatibility()

Update: If I go into the hallway_-1 scene and the room_03 scene and save, the multiple errors about those scenes do go away. So a resave fixes them, it just never happens automatically.

However, I still get this error on every run:
W 0:00:00:0896 RenderingServer::fix_surface_compatibility: A surface uses an old surface format and needs to be upgraded. The upgrade happens automatically at load time every time until the mesh is saved again or re-imported. Once saved (or re-imported), this mesh will be incompatible with earlier versions of Godot. <C++ Source> servers\rendering_server.cpp:2062 @ RenderingServer::fix_surface_compatibility()

[clayjohn]

@chrisl8 you need to open a tscn that contains a mesh in the editor to trigger the popup. Once triggered, you can select "Upgrade & Re-save" which will automatically re-save every scene, re-import every mesh, and re-save every mesh stored in a .res or .tres. This will get rid of the errors. I tested on your repo and can confirm that the upgrade tool appears to work as expected.

That being said after upgrading, upon running the project I get errors relating to failing loading some scenes. Its unclear if the errors come from this same issue or not as some scenes (e.g. player.tscn) appear to be corrupted even when running with 4.1.1.

[lostminds]

I got the importer to trigger again, like you wrote above @clayjohn the converter only seems to run if you open the project with a scene open that has a mesh (this was why it wouldn't re-run for me, so that mystery is solved at least). Opening files I had not previously opened caused re-imports that again broke scenes in the same way as above. It seemed that if had a scene open with the mesh loaded that the converter tool was trying to convert it would fail and breaks that open scene, like the example up top.

However, after trying to repeat the process again I couldn't get the same result. Now I get the same situation as the others above, it just pops up again each time I open the project, trying to convert all meshes again. The scenes aren't changed (at least no diffs, but timestamp on file is changed) and I still get the warning on running.

So perhaps the initial issue was/is related to some cached mesh data of open files that was conflicting with the tool re-imports? That eventually was cleared away with closing the scenes, relaunching and opening them again trying to get the issue to trigger again?

[clayjohn]

I got the importer to trigger again, like you wrote above @clayjohn the converter only seems to run if you open the project with a scene open that has a mesh (this was why it wouldn't re-run for me, so that mystery is solved at least). Opening files I had not previously opened caused re-imports that again broke scenes in the same way as above. It seemed that if had a scene open with the mesh loaded that the converter tool was trying to convert it would fail and breaks that open scene, like the example up top.

However, after trying to repeat the process again I couldn't get the same result. Now I get the same situation as the others above, it just pops up again each time I open the project, trying to convert all meshes again. The scenes aren't changed (at least no diffs, but timestamp on file is changed) and I still get the warning on running.

So perhaps the initial issue was/is related to some cached mesh data of open files that was conflicting with the tool re-imports? That eventually was cleared away with closing the scenes, relaunching and opening them again trying to get the issue to trigger again?

That might be it. I wonder how closely related this is to the glb name change thing as well.

It certainly sounds like the tool is missing some meshes. Using @chrisl8's test project above, Akien found a scene where the tool isn't triggered at all (neither is the warning). So that's a start to help find what is being missed.

Do you think you could identify the scenes that have meshes that aren't getting fixed? I.e. the scenes that contain meshes that are triggering the warning even after re-saving?

[chrisl8]

@chrisl8 you need to open a tscn that contains a mesh in the editor to trigger the popup. Once triggered, you can select "Upgrade & Re-save" which will automatically re-save every scene, re-import every mesh, and re-save every mesh stored in a .res or .tres. This will get rid of the errors. I tested on your repo and can confirm that the upgrade tool appears to work as expected.

Thank you.
I followed your advice and I do get the popup when opening a scene for the first time, but now it just locks up until I go kill Godot in task manager.

UpgradeLocksUp.mp4

That being said after upgrading, upon running the project I get errors relating to failing loading some scenes. Its unclear if the errors come from this same issue or not as some scenes (e.g. player.tscn) appear to be corrupted even when running with 4.1.1.

That is odd as I get no issues with corrupt scenes and the project runs fine for me in 4.1.2 and 4.2 after a fresh pull from the repo.
I just tried it about four times and no issues, but this may not be a Godot issue if it works for me and not you.
Sure enough, there is a corrupt scene file there. One of those "move or delete a file and Godot can't open it anymore" things. Just took me a while to find it, and somehow it works if I open things in the right order, but not the wrong order.

Edit: If I open it in 4.1.2 and try to open many scenes, they are listed as corrupt, but if I select Project->Reload current project, then they work. However, there are no files outside of .godot edited by that operation, so I'm not sure what to do to fix it for first time users of the project. Seems like a problem unrelated to this issue, but it might be causing some of the hiccups I'm having with the mesh conversion code.

I'm glad my project is broken enough to provide useful testing.

---

Edit, Edit: This seems to be the path:

1. Clone the repo.
2. Open it in 4.1.2
3. Try to open spaceship->spacehsip.tscn
4. It is reported as corrupt.
5. Project->Reload Current Project (Godot will close, but not reopen by itself)
6. Open Godot 4.1.2 and the scene again to see that it now opens
7. Close Godot 4.1.2
8. Open the project in 4.2 latest build from Master

At this point it will never bring up the dialog about upgrading meshes, regardless of what scene is opened, but it will spam the debug log with warnings about converting meshes whenever the project is run in debug.

Edit, Edit, Edit: Regarding step 5 above, really in 4.1.2 or 4.2, after initially opening the project, one MUST close Godot and open it again, otherwise many scenes will report as corrupt. Once closing and reopening, then they no longer report as corrupt.
However, I also have not been able to make the Mesh Import dialog open after doing this either. I only manage to see it if I open a scene on the first time I open Godot with a project, but then, some scenes are not loading then either, so it is a bit of a catch 22.

[viksl]

In my case the pop up shows when I first time open godot project and it shows every time and everytime goes through reimporting the same assets, it also reimports assets which are not in the default scene. When I open any scene afterwards it does not pop up so it reimports everything even if it's not in my opened scene but for some reason does it everytime I open the godot project.

I wonder if it's the fact that my models are FBX using the fbx importer (linked in project settings) but I also have gltf files there and these also get reimported everytime?

[dc1248]

@clayjohn I can confirm it missed a few meshes/scenes in my project, and also corrupted one scene (besides breaking the tscn file, it added stuff that belongs in linked scenes). I quit the editor, restored the corrupt scene from git, opened one of the missed scenes, was prompted to the mesh upgrade tool, and that time it didn't corrupt anything, but I had to manually open and resave a few scenes.

The scenes that failed to convert are very simple: a root Node3D with one mesh+staticbody+collisionshape and one linked scene (added with Instantiate Child Scene) which is just a mesh+staticbody+collisionshape, which DID convert just fine. These are all embedded meshes copypasted from a GLTF import. The linked scene seems to be the issue - as if the converter says "this is already converted" and mistakenly flags the parent scene as already-converted.

When beta4 comes out I'll rollback to my pre-beta3 commit, retest the converter, and submit more feedback if these issues persist.

edit: tested in beta5, no problems!

[lostminds]

The original issue here, with the broken scenes after import, seems to be related to some cache issue of previously opened scenes. But it's not as easy as being the scenes you have open when you open the project as I suspected at first, making it difficult to track down. The tool also appears to be triggered to re-run the check if any of these cached (not currently open) scenes contain meshes, since it runs even if the only scene I have open when I launch the project has no meshes. In these cases it also outputs errors from trying to parse these scenes (that are not open), like:

 scene/resources/resource_format_text.cpp:284 - res://Scenes/Game.tscn:67 - Parse Error: 
  Failed loading resource: res://Scenes/Game.tscn. Make sure resources have been imported by opening the project in the editor at least once.

But these scene files on disk are unchanged and opening them works fine.

As for the other issue of files that trigger the converter warning, but are missed or not saved by the converter tool I found the scene that was causing the problem in my case: Special_ZapEffect.tscn.zip
Opening this scene in the editor and saving it removed the warning, and updates the file, changing the format of the embedded mesh data. And the converter no longer triggers when I open the project. It appears this scene file has previously been caused to embed the mesh data of the imported mesh into the scene file, but in this case this has correctly overridden the mesh instances to reference the embedded data unlike in the cases above. And this embedded sub_resource ArrayMesh apparently is in the old format, so it triggers the conversion, but it's not changed/saved properly by the auto-update tool.

[clayjohn]

Is anyone here willing to share their project before upgrading to Beta3? I am having trouble reproducing the issues above and I have a feeling I will need to be able to reproduce them in order to get closer to a solution.

Right now I am seeing a few clues:

1. Some GLBs are missed entirely by the upgrade tool (as seen in @chrisl8's project) and so they don't trigger an upgrade
2. It seems that some meshes are already cached (or partially loaded) by the time the UpgradeTool runs leading to them continuing to load afterwards (and maybe not capturing save data)

Edit: I discussed this with people who are more knowledgeable than I am about importer and have some ideas for what might fix this. I'll start working on a PR. In the meantime, a test scene would be very helpful

[viksl]

@clayjohn Here's a test project I just made. Just to be clear, I tested to create a new project in beta beta 2 and earlier in 4.2 but none of them showed my problem. The issue only shows with projects I made in 4.1.y (or I assume earlier versions too but I haven't tested it).

I'm also uploading here the full test project including the .godot directory just in case it matters (I know usually I should delete it before uploading).

For me any time I open this project in 4.2 beta3 I get the pop up, I press the "do it and save it" button and then I can close the editor open the editor again, open the project and the pop up is back and it will reimport the exactly same meshes everytime (all of them).

Sometimes I get some permission error with one file for some reason (apparently it's used but I don't understand by what considering I don't use it elsewhere and only godot uses it in that directory) - this error is pretty common too, is it possible that there's some race condition messing this up because otherwise I don't know what else could be holding it back. Especially considering it's the same directory as all the other files and the other ones work just fine.

Is this ok?

TestBeta2Mesh.zip

Edit: I'm on win 11 btw.

[dc1248]

@clayjohn Here's a minimal test project created in 4.2-beta2 /without/ any gltf/glb files. test-broken-mesh-upgrade.zip

Notably, ONLY when I run the converter in beta3, I get a 'parse error' on line 44 of sm_gravestone_02.tscn
[node name="sm_gravestone_01" parent="." instance=ExtResource("3_32dt8")]
which looks fine to me and never causes issues when I open this scene in beta2 or beta3 (and decline to run the converter).

edit: I'm on linux

[chrisl8]

Sometimes I get some permission error with one file for some reason (apparently it's used but I don't understand by what considering I don't use it elsewhere and only godot uses it in that directory) - this error is pretty common too, is it possible that there's some race condition messing this up because otherwise I don't know what else could be holding it back. Especially considering it's the same directory as all the other files and the other ones work just fine.

When I went through my project (the one still up on github in the pre-4.2 format) and opened every scene and then closed and opened Godot 4.2 "latest build" to force it into doing a full conversion I got this error over and over

I fixed it by rebooting and trying again.

My theory is that maybe Godot was tripping on itself, or maybe my external VSCode editor had grabbed it, or maybe it was just Windows being Windows.

It is odd that it was always the same file though, like maybe Godot was trying to open it twice?

It eventually worked though for no clear reason so I don't have a clear path to reproduce it.

I only mention it since you also mentioned getting a permission issue, so maybe this will be a common issue for users.

[clayjohn]

For those of you able to build locally, please test out #84200 and let me know if it resolves the issues you have reported here.

I have run all the provided projects and I can't reproduce the issues mentioned in any of them. That alone is no surprise if my theory is correct as it is most likely a threading issue from running the upgrade tool while the engine is still loading. I hope that #84200 is enough to resolve the problems, but I can't be sure until it is tested on the affected systems.

I hope to provide "official" testing builds as well for those that can't build locally. But that may take a couple days to be ready.

[chrisl8]

Tangentially related note, if you clone my project and open it, you must then close it and open it again, otherwise Godot will claim that most scenes are corrupt. My guess is that Godot doesn't fully recognize its own import files until it restarts. This isn't new or really a complaint, just FYI if you try to use my project for testing and it is complaining about corrupt scenes.

#84200 definitely seems like an improvement. However, I am now aware of needing to open every .tscn file in the Godot editor before trying to use it which I wasn't before.

Until I did that I had this error when running:
W 0:00:00:0784 RenderingServer::fix_surface_compatibility: At least one surface uses an old surface format and needs to be upgraded. The upgrade happens automatically at load time every time until the mesh is saved again or re-imported. Once saved (or re-imported), this mesh will be incompatible with earlier versions of Godot. <C++ Source> servers\rendering_server.cpp:2058 @ RenderingServer::fix_surface_compatibility()

After opening every scene though, it went away. I didn't even have to explicitly save them.

I never got the popup, I guess that is just gone now? It apparently just performed the fixes silently and without any complaints or errors or permissions issues.

[clayjohn]

Tangentially related note, if you clone my project and open it, you must then close it and open it again, otherwise Godot will claim that most scenes are corrupt. My guess is that Godot doesn't fully recognize its own import files until it restarts. This isn't new or really a complaint, just FYI if you try to use my project for testing and it is complaining about corrupt scenes.

#84200 definitely seems like an improvement. However, I am now aware of needing to open every .tscn file in the Godot editor before trying to use it which I wasn't before.

Until I did that I had this error when running: W 0:00:00:0784 RenderingServer::fix_surface_compatibility: At least one surface uses an old surface format and needs to be upgraded. The upgrade happens automatically at load time every time until the mesh is saved again or re-imported. Once saved (or re-imported), this mesh will be incompatible with earlier versions of Godot. <C++ Source> servers\rendering_server.cpp:2058 @ RenderingServer::fix_surface_compatibility()

After opening every scene though, it went away. I didn't even have to explicitly save them.

I never got the popup, I guess that is just gone now? It apparently just performed the fixes silently and without any complaints or errors or permissions issues.

That is extremely odd. The popup should still come up. And the upgrade tool doesn't run unless you select the option to run it in the popup. In this case it seems like the upgrade tool didn't run at all for you.

You shouldn't have to open every tscn individually either. That should be done for you by the upgrade tool.

To clarify, did you test this on a clean project, or one that had already been used with beta3?

[warriormaster12]

Seems to have upgrade everything just fine although even animations although interestingly I got a dependency erros for two animation.fbx files that were resolved by rebooting the project again.

[chrisl8]

My Godot build version: v4.2.beta.custom_build [d036f8aa9]

Here is a quick video to demonstrate what I'm doing and what I'm seeing.
https://github.com/godotengine/godot/assets/6188278/37f88f6f-62c8-49f3-8337-ab1f1fec0c33

Note that from a clean clone of the repo, I initially have to close and reopen to open the scenes.
After that, when I run there are a lot of errors, but just opening the scene and running again fixes it.
You can see in the git diff that changes were made to the .tscn files.

[dc1248]

That is extremely odd. The popup should still come up. And the upgrade tool doesn't run unless you select the option to run it in the popup. In this case it seems like the upgrade tool didn't run at all for you.

I've noticed this too. The popup only seems to come up when you've opened a scene that has meshes in the old format. Maybe it should come up whenever someone opens a project last saved in a prior Godot version.

I really think there should be a "File > Resave Everything" (scenes, materials, anims, etc) command that basically opens this popup so I can run it anytime and do a git diff to verify that Godot is loading and saving everything consistently. The "old mesh format detected" warning message can suggest to use it.

[acegiak]

I'm also getting the tool popping up every time i open the project with 4.3 beta 4

[clayjohn]

I have pushed updated to #84200 in the hopes that we finally have something that works in all cases.

I really think there should be a "File > Resave Everything" (scenes, materials, anims, etc) command that basically opens this popup so I can run it anytime and do a git diff to verify that Godot is loading and saving everything consistently. The "old mesh format detected" warning message can suggest to use it.

We might do that. In theory we should be able to get the tool to always detect when it is needed and only ever popup when needed. For us that is ideal so that we aren't cluttering the editor with a tool that is only needed in the specific context of updating your game from 4.1 to 4.2

[YuriSizov]

@lostminds and others, with beta 5 out, could you retest your projects and report back if they are now imported into Godot 4.2 without issues?

https://godotengine.org/download/archive/4.2-beta5/

[lostminds]

@lostminds and others, with beta 5 out, could you retest your projects and report back if they are now imported into Godot 4.2 without issues?

I tried just opening the project in beta 5 and this worked fine (as I had previously resolved the mesh importing issues). I then quit and put back in the scene file that caused issues before and launched the project. This activated the conversion tool (but now with "Upgrade and restart") so I ran it and it restarted and processed all the files. According to the git diff it didn't modify any scene files on disk, (other than the offending scene with embedded mesh data which it seems to have fixed). But it added "gltf/naming_version=0" to the all the mesh .import files. However, this seems to have broken all the mesh imports again in my project, since I'm guessing they now produce the old "2" mesh names, and not the correct new mesh names I had fixed all my scenes to reference. So in this case I'm back to the original issue of the re-import breaking all the references again. With a lot of errors when running the game like:

W 0:00:00:0345   instantiate: Node 'Model/Block/Block' was modified from inside an instance, but it has vanished.
  <C++ Source>   scene/resources/packed_scene.cpp:231 @ instantiate()

But this is probably just because I had "fixed" it manually before, and I guess this won't affect anyone importing projects that are pre-4.2.
I fixed this my removing the "gltf/naming_version=0" line from all the import files and then manually again re-importing the meshes. This instead put a "gltf/naming_version=1" in the import files, and everything now seems to work fine.

I am still seeing another warning each time I launch the project that I did not see previous to 4.2 beta versions:

W 0:00:00:0111   _load_pipeline_cache: Invalid pipelines cache header.
  <C++ Source>   drivers/vulkan/rendering_device_vulkan.cpp:9197 @ _load_pipeline_cache()

But I'm guessing that's not related to this issue?

[YuriSizov]

But it added "gltf/naming_version=0" to the all the mesh .import files. However, this seems to have broken all the mesh imports again in my project, since I'm guessing they now produce the old "2" mesh names, and not the correct new mesh names I had fixed all my scenes to reference.

That's unrelated to the mesh format. We have added compatibility code so that existing scenes from Godot 4.1 keep using the old naming scheme (with "2" and such), which removes the need to fix scenes.

Since your project is already half fixed, I don't think we can offer an automatic solution. You'll have to update your assets to use the new naming version manually (or revert scene changes).

I am still seeing another warning each time I launch the project that I did not see previous to 4.2 beta versions

That's probably unrelated, yeah. Worth a dedicated bug report.

[YuriSizov]

I'm going to close this as resolved based on the discussion above.

[Joy-less]

This is still an issue in 4.3-stable, running the mesh upgrade tool removed most meshes in my project irrecoverably (lucky I'm using version control). Judging by the errors and changes to files, it seems to be the same issue.

[ozggit]

Hey @Joy-less or anyone running into this issue. If you run the Upgrade tool and it breaks your meshes (happen to me), then try deleting the .import files in the same directory where the meshes (fbx) were. This prompted an automatic re-import (didn't work manually!) that fixed everything.