Unable to establish WebRTC peer connection outside local network when using C#

[TheWrightDev]

Godot version:
3.2.3-stable_mono_win64

OS/device including version:
Windows

Issue description:
When attempting to establish a webrtc peer connection if local or inside the current network all works as expected. But if outside, if file containing the WebRTCMultiplayer and PeerConnection was written in C# it cannot join, but the same file written in gdscript works great (even if host is C#).

Can view debug logs for webrtc here:
C# Host - C# Join
GDScript Host - GDScript Join

Some minor observations from looking over the debug logs when connecting:
The gdscript version contains this line fairly early on
(basicportallocator.cc:1355): AllocationSequence: UDPPort will be handling the STUN candidate generation.
And while I don't know where exactly that's coming from, searching around for that file seems to indicate that potentially its absence in the C# version might be coming from the "PORTALLOCATOR_ENABLE_SHARED_SOCKET" flag somehow not being set?

Second observation is that the C# log for the joining client won't contain any instances of Received STUN ping response.

Steps to reproduce:

1. Download full setup: https://www.filedropper.com/simplewebrtc (includes webrtc_debug and both builds already) OR Download SimpleWebRTC.zip, add webrtc_debug, set AutoLoad to MultiplayerClient.cs and export, then set to MultiplayerClientGD (but keep name as MultiplayerClient) then build and export as well.

2. From the Server directory npm install followed by node server.js

3. Launch two instances locally of either version and play. In one instance click Host, copy down the room number from the upper left and enter it into the other client's "Game" input and click Join. Feel free to mix and match, 2 x gds, 2 x C#, or either type can host for the other, just verify that everything works great.

4. Get the server setup outside your current network as well as a copy of the builds. The gdscript builds will still be able to connect and play, but the C# builds will no longer be able to join. They can host, and the gds build can still actually join and play with a C# build, but a C# build cannot be the joining player.

Minimal reproduction project:
SimpleWebRTC.zip

Credit to u/Slyka whose minimal project I used for this

[akien-mga]

CC @neikeq @Faless

[jonbonazza]

Since this seems to only be an issue with c# and the network and multiplayer apis are the same regardless of scripting language, i think this is more likely an issue with the c# bindings than the network/mp api itself.

[ASavchenkov]

So I'm a bit confused as to where I would even look for the source of this issue, since I don't see any documentation on how it is the mono module works. Afaik , the c# source files are automatically generated during the build process, but there isn't any c# code that actually deals with ICE servers as far as I can tell, so is the issue happening when initialize is called on WebRTCPeerconnection?

I'd love to help resolve this since it affects me, but it feels like it's way above my pay grade. Would love some guidance as to where I should look at least.

[jonbonazza]

Is STUN supported by c#? Im not up to date on the current state of the world irt NAT punch through

[ASavchenkov]

Is STUN supported by c#? Im not up to date on the current state of the world irt NAT punch through

It doesn't happen in c#. IIRC c# code calls into Godot's WebRTC implementation. That's why this is quite confusing. There's really not that much c# that actually does anything with respect to ICE.

[jonbonazza]

Is STUN supported by c#? Im not up to date on the current state of the world irt NAT punch through

It doesn't happen in c#. IIRC c# code calls into Godot's WebRTC implementation. That's why this is quite confusing. There's really not that much c# that actually does anything with respect to ICE.

If that's the case then it is certainly weird, as the internal webrtc apis should work the exact same way regardless of scripting language.

Im not sure if either @neikeq or @Faless has an ideas here, but im admittedly not really familiar with Godot's webrtc implementation or the NAT punch through facilities available.

[ASavchenkov]

OK so I decided to go back to tackling this, but with my own "copy-paste" signalling system, and weirdly enough, it sometimes works? Here's the source, though I've linked a fully set up project below.

edit: Earlier example project wasn't adding ICE candidates. This one does, and the bug persists.
WebRTCMonoTest.zip

I've yet to see it fail on a local machine. It works between my wired desktop and a computer connected to the internet by a mobile tether. (edit: Testing by having friends on different networks run the binary results in silent failure.) Unfortunately, I don't have access to enough varied computers and internet connections to debug.

edit: Again, this seems like the kind of thing you can only really test if you have multiple machines on multiple connections. I can't seem to find any testing software that can simulate that kind of environment, so I welcome any suggestions.

[ASavchenkov]

OK so I've managed to reproduce this issue on my personal machines. It seems that when IPV6 is an option, WebRTC can indeed bypass ICE servers, hence my trouble reproducing the issue earlier. Disabling IPV6 on either machine causes failure.

It seems to be that when using WebRTC from c#, using ICE servers just doesn't work. Why? I don't know (yet).

[Faless]

run the binary results in silent failure.

Try using the debug build of the webrtc plugin, you should get a much more verbose log in console.

[ASavchenkov]

run the binary results in silent failure.

Try using the debug build of the webrtc plugin, you should get a much more verbose log in console.

So now that I've reproduced this at home this is much easier to do! :)

Notable things that I've found:

1. The initiator spits this out when getting an answer: PeerConnection cannot create an answer in a state other than have-remote-offer or have-local-pranswer.
   I double checked and SetLocalDescription is being called prior to this so afaik it shouldn't even be trying to create an answer...
2. After setting the local description, the peer connection is still in the "NEW" state.

I confirmed that the connection is first being created, then added to an instance of WebRTCMultiplayer. Only then are "CreateOffer", and subsequently "SetLocalDescription" called (in my reproduction project).

Note: I checked just now and these two points actually both occur even when things "work" over ipv6" or on a local network, so perhaps I just misunderstand what the states are supposed to represent.

Other than that, the only difference I've found between a failed connection and a successful one are the failed STUN pings that come afterwards.

[ectucker1]

OK so I've managed to reproduce this issue on my personal machines. It seems that when IPV6 is an option, WebRTC can indeed bypass ICE servers, hence my trouble reproducing the issue earlier. Disabling IPV6 on either machine causes failure.

What was the method you used to recreate this locally? I'd like to do some testing on it with my own project.

[ASavchenkov]

What was the method you used to recreate this locally? I'd like to do some testing on it with my own project.

Unfortunately I did not recreate it locally. One machine ran on my home network, and another ran off of my phone's USB tether.

The thing is, even with that setup, I was unable to reproduce the error because both had IPV6 support, and didn't need STUN. I had to disable IPV6 on my home machine's network adapter to reproduce the issue.

[ASavchenkov]

So perhaps someone with a better understanding of how the c# bindings work can direct me here, because I'm about to have to fix this one way or another, and I'd rather not go about it bumbling around like a toddler.

Where do I start looking for the cause here? Is it most likely in the WebRTCModule, or in the WebRTC GDNative library? Is this related to the mono glue generation? I'm not finding any documentation on how this stuff works in general.

[mhilbrunner]

Things that could be relevant:
#54758
#36958