Networking mirrored can't work on WSL2

[yqs112358]

Windows Version

Microsoft Windows [Version 10.0.22631.2500]

WSL Version

2.0.14.0

Are you using WSL 1 or WSL 2?

• WSL 2
• WSL 1

Kernel Version

5.15.133.1-1

Distro Version

Ubuntu 20.04

Other Software

None

Repro Steps

• Set .wslconfig:

[experimental]
networkingMode=mirrored
dnsTunneling=true
firewall=true

• wsl --shutdown to stop WSL
• Allow tcp port 8000 in Windows firewall settings
• Restart WSL

Expected Behavior

Both WSL and Windows can successfully access 127.0.0.1:8000 by curl

Actual Behavior

My laptop's IP is 192.168.3.11 in WLAN

1. Test HTTP server on port 8000 in WSL

Execute python -m http.server in WSL. Output:
Serving HTTP on 0.0.0.0 port 8000 (http://0.0.0.0:8000/) ...

• Execute curl in WSL   (WSL curl -> WSL server)
  • curl http://192.168.3.11:8000 succeeded.
  • curl http://localhost:8000 failed. Cannot connect. Reason: Timeout
  • curl http://127.0.0.1:8000 failed. Cannot connect. Reason: Timeout
• Execute curl in Windows   (Windows curl -> WSL server)
  • curl http://192.168.3.11:8000 failed. Cannot connect. Reason: Timeout
  • curl http://localhost:8000 failed. Cannot connect. Reason: Timeout
  • curl http://127.0.0.1:8000 failed. Cannot connect. Reason: Timeout

2. Test HTTP server on port 8000 in Windows

Execute python -m http.server --bind 0.0.0.0 in Windows. Output:
Serving HTTP on 0.0.0.0 port 8000 (http://0.0.0.0:8000/) ...

• Execute curl in WSL   (WSL curl -> Windows server)
  • curl http://192.168.3.11:8000 failed. Cannot connect. Reason: Connection refused
  • curl http://localhost:8000 failed. Cannot connect. Reason: Timeout
  • curl http://127.0.0.1:8000 failed. Cannot connect. Reason: Timeout
• Execute curl in Windows   (Windows curl -> Windows server)
  • curl http://192.168.3.11:8000 succeeded.
  • curl http://localhost:8000 succeeded.
  • curl http://127.0.0.1:8000 succeeded.

Diagnostic Logs

/etc/resolv.conf

nameserver 127.0.0.42

ifconfig in WSL

br-d0766515769e: flags=4099<UP,BROADCAST,MULTICAST>  mtu 1500
        inet 172.18.0.1  netmask 255.255.0.0  broadcast 172.18.255.255
        ether 02:42:03:7d:ec:8b  txqueuelen 0  (Ethernet)
        RX packets 0  bytes 0 (0.0 B)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 0  bytes 0 (0.0 B)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

docker0: flags=4099<UP,BROADCAST,MULTICAST>  mtu 1500
        inet 172.17.0.1  netmask 255.255.0.0  broadcast 172.17.255.255
        ether 02:42:98:0b:74:7d  txqueuelen 0  (Ethernet)
        RX packets 0  bytes 0 (0.0 B)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 0  bytes 0 (0.0 B)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

eth1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 192.168.3.11  netmask 255.255.255.0  broadcast 192.168.3.255
        inet6 fe80::1c95:c4bd:2c0b:12fd  prefixlen 64  scopeid 0x20<link>
        ether 40:ec:99:cf:f2:a6  txqueuelen 1000  (Ethernet)
        RX packets 280  bytes 21218 (21.2 KB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 158  bytes 21856 (21.8 KB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

lo: flags=73<UP,LOOPBACK,RUNNING>  mtu 65536
        inet 127.0.0.1  netmask 255.0.0.0
        inet6 ::1  prefixlen 128  scopeid 0x10<host>
        loop  txqueuelen 1000  (Local Loopback)
        RX packets 86  bytes 10468 (10.4 KB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 86  bytes 10468 (10.4 KB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

loopback0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        ether 00:15:5d:d8:86:a8  txqueuelen 1000  (Ethernet)
        RX packets 20  bytes 1416 (1.4 KB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 12  bytes 888 (888.0 B)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

[keith-horton]

Hi there.
Are you setting up the server within the Docker container in Linux? Note that Docker has its own IP address on its own network device. How did you configure it in Linux to forward requests to that address within Docker?

If you setup a listener within Docker within Linux, you'll want to let WSL know that the port is reserved and used by the WSL container by adding this to the .wslconfig file under experimental:

ignoredPorts=8000

[yqs112358]

Thanks for your remind😂
But this test is actually done in real WSL, not in a docker container.
I have removed docker-related content in this issue to avoid confusion.

[sf467]

The mirrored network feature of wsl doesn't work on my machine either.
my win11 is dev channal, build 23570, but......

[FlowerBirds]

Only one lo network card：

update wsl 1.x to 2.0.4 and window to 23H2.

what's wrong with me ?

[keith-horton]

If you are seeing issues, please collect traces so we can investigate what is going on: https://github.com/microsoft/WSL/blob/master/diagnostics/collect-wsl-logs.ps1

[FlowerBirds]

@keith-horton https://github.com/FlowerBirds/wsl2/blob/main/WslLogs-2023-10-24_08-49-22.zip

Thanks

[shigenobuokamoto]

@yqs112358
can you try this?

curl --noproxy \* http://127.0.0.1:8000

[keith-horton]

@keith-horton https://github.com/FlowerBirds/wsl2/blob/main/WslLogs-2023-10-24_08-49-22.zip

Thanks

Thank you. I tried 4 different tools to open the trace, but all were able to only resolve a couple of lines of a trace. Sorry. Could we try again and see if we get a more usable trace?

We have a new script to collect networking logs, could you run this? https://github.com/microsoft/WSL/blob/master/diagnostics/collect-networking-logs.ps1

Thanks!

[FlowerBirds]

@keith-horton use collect-networking-logs.ps1 script and upload logs at https://github.com/FlowerBirds/wsl2/blob/main/WslNetworkingLogs-2023-10-25_08-29-04.zip.

Thanks!

[yqs112358]

@yqs112358 can you try this?

curl --noproxy \* http://127.0.0.1:8000

Sorry, but the same: cannot connect. (Both in Windows and WSL)
I think it's not the problem of proxy because I have not set $http_proxy

[yqs112358]

I have upgraded to WSL preview 2.0.6 and tried again, but the problem still exists.
Actual Behavior section is updated. I have tested creating server in Windows too and posted the result above.

[keith-horton]

@FlowerBirds , thank you for the trace.

I'm seeing multiple errors unfortunately. Can you verify that WPA is installed and accessible from where you're running your script? e.g. it doesn't seem to be able to find wpr.exe:

• wpr.exe -start $logProfile -filemode 2>&1 >> $wprOutputLog

•   + CategoryInfo          : NotSpecified: (:String) [], RemoteException
    + FullyQualifiedErrorId : NativeCommandError
  
    The system cannot find the file specified.
    
  
  

Also, trying to run the shell script in Linux also failed. Can you run this from an administrator command prompt? Maybe that was the issue.

wsl.exe : <3>WSL (17) ERROR: CreateProcessEntryCommon:511: execvpe ./networking.sh failed 2
所在位置 C:\Users\merit\collect-networking-logs.ps1:126 字符: 1

• & wsl.exe -e ./networking.sh 2>&1 > $folder/linux_network_configurati ...

•   + CategoryInfo          : NotSpecified: (<3>WSL (17) ERR...ing.sh failed 2:String) [], RemoteException
    + FullyQualifiedErrorId : NativeCommandError
  
  

<3>WSL (17) ERROR: CreateProcessEntryCommon:514: Create process not expected to return

Thanks!

[yqs112358]

Updated to WSL pre-release 2.0.9.0, the situation above has not changed.

Some discovery attached:
When execute python -m http.server --bind 0.0.0.0 in Windows first, then try to execute python -m http.server in WSL, python in WSL will report:

OSError: [Errno 98] Address already in use

So I think the port-sharing works well. The problem is only about why the network traffic cannot arrive the target port.

[yqs112358]

If you are seeing issues, please collect traces so we can investigate what is going on

@keith-horton
I have recorded the trace by collect-wsl-logs.ps1 and attached here. Hope to be helpful for you 😊
WslLogs-2023-11-14_18-06-19.zip

In this record, I have done these actions step by step:

1. Start WSL
2. Create an http server by python -m http.server --bind 0.0.0.0 in Windows
3. Execute curl localhost:8000 in WSL, trying to connect to the server. The connection failed.
4. Shutdown the server above, and create another http server by python -m http.server in WSL
5. Execute curl localhost:8000 in Windows to connect to the server. The connection still failed.

[keith-horton]

Thank you. I'll look at the trace.
Can you verify in your scenario if curl 127.0.0.1:8000 works? localhost will usually default to the IPv6 localhost ::1 address. And due to some complex reasons, we don't currently support connectivity from the host <-- --> container using the ::1 address (just 127.0.0.1).

[yqs112358]

Can you verify in your scenario if curl 127.0.0.1:8000 works?

@keith-horton
Okay, I have recorded again with 127.0.0.1 and attached here.
WslLogs-2023-11-15_10-53-30.zip

In this record, I have done these actions step by step:

1. Start WSL
2. Create an http server by python -m http.server --bind 0.0.0.0 in Windows
3. Execute curl 127.0.0.1:8000 in WSL, trying to connect to the server. The connection failed.
4. Shutdown the server above, and create another http server by python -m http.server --bind 0.0.0.0 in WSL
5. Execute curl 127.0.0.1:8000 in Windows to connect to the server. The connection failed.

These two tests are still all failed, just the same as the behavior in Actual Behavior section

[grscottjo]

I was facing this same issue but seem to have worked around it.

What I did that seems to have fixed it: I added hostAddressLoopback=true to my .wslconfig
Differences between the issue's setup and mine:

• I'm not using the dnsTunneling=true and firewall=true settings in .wslconfig
• I did not do any manual port rules for port 8000 in Windows

> wsl --version
WSL version: 2.0.9.0
Kernel version: 5.15.133.1-1
WSLg version: 1.0.59
MSRDC version: 1.2.4677
Direct3D version: 1.611.1-81528511
DXCore version: 10.0.25131.1002-220531-1700.rs-onecore-base2-hyp
Windows version: 10.0.22631.2715

This page says hostAddressLoopback defaults to false.
https://learn.microsoft.com/en-us/windows/wsl/wsl-config
The description sounded close enough so I took a stab at it, and was able to access :8000 from the windows host, and from the windows host and other machines on my network if I bind to 0.0.0.0.

Caveats:

• I didn't always use curl, I used a Python script that just puts together some http.client calls to connect and report the response status

import http.client

connection = http.client.HTTPConnection('192.168.1.118', 8000, timeout=10)
connection.request("GET", "/")
response = connection.getresponse()
print("Status: {} and reason: {}".format(response.status, response.reason))

connection.close()

• I probably didn't test (windows client)->(wsl server) and (wsl client)->(windows server) as thoroughly as I could have. I think technically I did both with my python script, but then when I thought it would be good to post here, I tried (wsl client)->(windows server) with curl, and when I went to retry with (windows client)->(wsl server) with curl, the wsl side got an error about the port being used, so I gave up on being thorough.
• I also happened to set ignoredPorts=80 at the same time as I set hostAddressLoopback=true. I forgot at the time that the http.server was using 8000 and was just trying the things that seems most likely from the .wslconfig page I linked above. So I think it's irrelevant because it was specified for 80, but I want to be thorough in my caveats so I can get out of follow up questions asking me if I can test more thoroughly for you guys. ;-) I'm not at all an expert in any of this, but who knows, maybe setting any ignored port somehow kicked it into a different mode where the connections started working...

[yqs112358]

@grscottjo Thanks for testing. However, according to Microsoft's documentation, the ability to connect from a LAN IP with hostAddressLoopback=true is expected behavior:

When set to True, will allow the Container to connect to the Host, or the Host to connect to the Container, by an IP address that's assigned to the Host.

Note that the 127.0.0.1 loopback address can always be used - this option allows for all additionally assigned local IP addresses to be used as well.

I have tried hostAddressLoopback=true. With this option turned on, 192.168.3.11 is accessible in both directions via curl. However 127.0.0.1 is still not accessible. I think this problem has not been solved yet 😊

[Jokcy]

@grscottjo Thank you for you sharing! It seems work for me now, I have tried a lot of times.

[yqs112358]

Update 2024.03.27:

According to https://learn.microsoft.com/en-us/windows/wsl/wsl-config#wslconfig, when set localhostForwarding=true in .wslconfig, the host can access WSL correctly in both localhost:port and 127.0.0.1:port. This function works well.

But I have tested networkMode=mirrored and it still cannot work.
I'm now starting to think that maybe there's something wrong with my WSL configuration or my Ubuntu somewhere. I've heard of some other people reinstalled WSL and then the mirrored networking back to normal. Have you guys had similar experiences?

Versions:

• WSL: 2.2.1.0
• Linux Core: 5.15.150.1-2
• Windows: 10.0.22631.3296

[keith-horton]

@yqs112358 , you can connect to WSL from the host with mirrored mode, but you need allow loopback traffic through Hyper-V Firewall, and you need to create a Hyper-V Firewall rule to allow the port you want into the WSL container.

See https://learn.microsoft.com/en-us/windows/security/operating-system-security/network-security/windows-firewall/hyper-v-firewall - look for LoopbackEnabled and New-NetFirewallHyperVRule.

[yunsii]

#11172 May be the same problem

[wangyw6716]

Try executing the command

sudo systemctl disable network-manager.service

followed by running wsl.exe --shutdown in PowerShell, and then restart WSL again.

[keith-horton]

Yes. SystemD running its own Network Manager service is highly likely to hit compatibility issues with WSL.

[yunsii]

@wangyw6716 Cool, it works like a charm.

[keith-horton]

@yqs112358 , can you see if you have NetworkManager running? that's not going to work well with WSL.

[yunsii]

@keith-horton Before I disable network-manager.service, I checked status by systemctl status network-manager.service, and network-manager.service is running. It works after I disabled network-manager.service.

[keith-horton]

Thanks!

[yqs112358]

Solved:
I have uninstall WSL and Ubuntu thoroughly, and then re-install again.
After a complete reinstall, networkingMode=mirrored works well. Both directions are accessiable by curl 127.0.0.1 or localhost.
In new Ubuntu, I have found that network-manager.service does not exist. Maybe this service does have some what of effect, or maybe there was some problem with the network configuration of my old WSL. Anyway, this feature is now fully functional. Thanks for your hard work 😊 @keith-horton @wangyw6716

Env:

WSL Version： 2.2.1.0
Core Version： 5.15.150.1-2
WSLg Version： 1.0.60
MSRDC Version： 1.2.5105
Direct3D Version： 1.611.1-81528511
DXCore Version： 10.0.25131.1002-220531-1700.rs-onecore-base2-hyp
Windows Version： 10.0.22631.3374

[yqs112358]

By the way, does Docker currently support network=mirrored well?
Some of my work dependents on this and it would be great if mirrored could be supported effectively.

[keith-horton]

We have been working closely with Docker - they have made positive changes for new WSL features such as mirrored mode and DNS tunneling :)

I would recommend using their latest updates.

[RomanZhakhovTR]

Disabling network-manager.service helped,
Thanks a lot :)

[jahanarun]

@yqs112358 can you try this?

curl --noproxy \* http://127.0.0.1:8000

This works!
Now, how do I use this noproxy mode in other apps?

[FrankFang]

@RomanZhakhovTR How could I disable network-manager.service, I could not find it.