[Mono] Networking not working with 3.4 on Windows with library binding both IPv4 and IPv6

[zaksnet]

Godot version

3.4-Stable

System information

Windows 11

Issue description

This is probably not related to networking at all but here is the case:
I'm using MQTTNet library with Godot as my communication framework and it seems to work good with v3.3.4 but with v3.4 of Godot MQTTNet does not connect with the server.
There is not error at the console and even when debugging with JetBrains Rider everything seems to run smoothly (besides that it does not connect with the server), MQTT client options are created properly, server IP:Port are correct (It doesn't look so far that it could be my error).
The only thing left to check in order to resolve this would be to keep testing commits and see what breaks this, any tips on what commit that would be is appreciated!

Steps to reproduce

No easy way! If someone can point out which commit might be responsible for this i could do the testing.

Minimal reproduction project

UPDATE
3.4-beta1: Doesn't even start the project, gives System.TypeInitializationException: The type initializer for 'Sys' threw an exception. (I'm guessing that was a known issue and was fixed)
3.4-beta2 to 3.4-stable: Same issue. So the issue has been there since 3.4-beta1 (or beta2)

Possibly related to #54725 ?

I have attached a sample project to demonstrate the issue, the project connects with a public MQtt broker. If you run the project with 3.3.4 it will print Connected with the server! in godot log window. If you run it with 3.4 it with print Disconnected from the server! in the log window.
SampleMqttNetProject.zip

[akien-mga]

CC @godotengine/network @godotengine/mono

[juse4pro]

We also had kind of this problem since Godot 3.4
Our problem was that the network library that we use (LiteNetLib) is trying to bind IPv4 and IPv6 sockets at the same time (which broke the network thread). We disabled IPv6 socket creation and now everything works again.

Why the library is doing so we don't know. But this only happens with Godot 3.4 (3.3.2 works fine).

Maybe this info helps a bit.

[zaksnet]

Thank you @juse4pro 🙏! It seems that after i enforce IPV4 the project works again with 3.4+ versions of Godot. This seems to be more of a Mono issue and if i had to guess its probably caused by a mono upgrade in Godot 3.4 but i'm not certain if i should close this issue or not. @akien-mga

[akien-mga]

Mono was upgraded from 6.12.0.147 to 6.12.0.158 in 3.4, but that only spans a handful of upstream commits, which don't seem related to networking: mono/mono@mono-6.12.0.147...mono-6.12.0.158

On the other hand there were some changes in godot-mono-builds (which we use to build Mono) related to the BTLS library on Windows, which could impact this:
godotengine/godot-mono-builds#47
godotengine/godot-mono-builds#49

It's good that there is a known workaround but it still sounds like a bug that should be fixed in future Godot versions.

Would also be good to confirm if 3.4.1-stable is also affected by the issue, and then I could make a test build of 3.4.1-stable with the 3.3.x containers (using Mono 6.12.0.144 built before the BTLS changes) to confirm if either of those changes (mono version or build script changes) are indeed the root of the problem.

For the record I don't reproduce the issue with 3.4-stable on Linux with the project attached in OP, so this might well be Windows specific.

[akien-mga]

Would also be good to confirm if 3.4.1-stable is also affected by the issue, and then I could make a test build of 3.4.1-stable with the 3.3.x containers (using Mono 6.12.0.144 built before the BTLS changes) to confirm if either of those changes (mono version or build script changes) are indeed the root of the problem.

Here's a test build of 3.4.1-stable using the mono 6.12.0.147 containers used for 3.3.x. Please test both the official 3.4.1-stable and this one to see if there's a difference: https://downloads.tuxfamily.org/godotengine/testing/Godot_v3.4.1-stable_mono_win64-mono6.12.0.147.zip

[zaksnet]

Thank you for the test build @akien-mga 🙏! I can confirm that the issue persists with both Godot 3.4.1 Stable and your test build. So the problem is probably with godot-mono-builds.

[akien-mga]

Thank you for the test build @akien-mga 🙏! I can confirm that the issue persists with both Godot 3.4.1 Stable and your test build. So the problem is probably with godot-mono-builds.

No actually this confirms that the problem is not with godot-mono-builds either, since the container I used for the test build doesn't have those changes.

So it's a regression in Godot itself, which happened between 3.3-stable and 3.4-beta2 (as you pointed out the beta1 mono builds are not usable to test).

[jfaz13]

Any updates on this regression? Is it still slated for 3.5 or still under investigation? This unfortunately breaks compatibility with iOS since they demand IPv6.

[mhilbrunner]

No updates on this yet I'm afraid. I'll try to find the responsible breaking change.

[mhilbrunner]

So... something more complicated is going on here. I tried to bisect this, and failed: while I can reproduce the issue with the provided example project using the official 3.4-beta2 build (https://downloads.tuxfamily.org/godotengine/3.4/beta2/mono/), I can't reproduce it if I build 3.4-beta2 myself on Windows (using commit a71169c, which should be 3.4-beta2 according to the Release Notes).

So the official build fails, but a custom one with the same commit succeeds. Any ideas? Maybe the cause is in how the official build is produced?

This is with mono-6.12.0.107-x64-0.msi (https://download.mono-project.com/archive/6.12.0/windows-installer/mono-6.12.0.107-x64-0.msi), which seems to be the latest stable Windows release (albeit noticeably older than the one in Akien's comments above).
The preview version (https://download.mono-project.com/archive/6.12.0/windows-installer/mono-6.12.0.165-x64-0.msi) works too, doesn't seem to make a difference.

[jfaz13]

I've no clue but if you're using the same commit, it sounds like the issue has to be in the way it's built. Is that the same mono version used in the official build?

[mhilbrunner]

So, I've now built 3.4-beta2 (a71169c) with the following Mono versions on Windows with MSVC:
6.12.0.107 (latest stable on Windows)
6.12.0.144 (preview version)
6.12.0.158 (which seems to be used in official Godot 3.4-beta2)
6.12.0.165 (nightly)

All of these work fine with the provided test project for me, whereas the official Godot 3.4-beta2 build does not.
Provided I didn't miss anything, there seems to be a difference between my builds and the official ones.
Not sure how the official ones are built (e.g. which compiler, are they cross compiled from Linux?), need to research.

[akien-mga]

See my debugging above which has information about the official builds, but also seemed to show that it's not a regression from the build environment, at least according to test reports.

[mhilbrunner]

Hm. But a regression from code also seems weird if 3.4-beta2 works for me if I compile it myself, but not if I use the official build.

@jfaz1 @juse4pro Are you on Windows 11 like the OP or Windows 10?

[jfaz13]

Hm. But a regression from code also seems weird if 3.4-beta2 works for me if I compiled it myself, but not if I use the official build.

@jfaz1 @juse4pro Are you on Windows 11 like the OP or Windows 10?

Yes, I am on Windows 11. I also tested on Windows 10, though.

[mhilbrunner]

Yeah, I can reproduce the issue with the official build on Windows 10 too - and my own build works. Weird.

Edit:

Here's a test build of 3.4.1-stable using the mono 6.12.0.147 containers used for 3.3.x. Please test both the official 3.4.1-stable and this one to see if there's a difference: https://downloads.tuxfamily.org/godotengine/testing/Godot_v3.4.1-stable_mono_win64-mono6.12.0.147.zip

This one reproduces the error for me too. Next step: Having a look at
https://github.com/godotengine/build-containers
https://github.com/godotengine/godot-mono-builds

[jferdelyi]

Same problem with Godot 3.5.1 on Windows11