[WSL2][systemd][interop] Unable to Execute Windows Binary when systemd enabled

[thelastlin]

Version

0.67.6.0

WSL Version

• WSL 2
• WSL 1

Kernel Version

Linux version 5.15.62.1-microsoft-standard-WSL2 (oe-user@oe-host) (x86_64-msft-linux-gcc (GCC) 9.3.0, GNU ld (GNU Binutils) 2.34.0.20200220) #1 SMP Wed Aug 24 22:24:20 UTC 2022

Distro Version

Ubuntu 22.04

Other Software

No response

Repro Steps

1. Edit /etc/wsl.conf, enable systemd support. (https://devblogs.microsoft.com/commandline/systemd-support-is-now-available-in-wsl/)
2. Run wsl.exe --shutdown in wsl2 shell, and reopen the shell.
3. Run wsl.exe --version or some other PE binaries, and shell will warn you that the "exec format error" or "MZ: not found"

Expected Behavior

Able to execute any valid PE binaries.

Actual Behavior

"exec format error" or "MZ: not found"

Diagnostic Logs

$ execve("/mnt/c/WINDOWS/system32/calc.exe", ["calc.exe"], 0x7ffc12cdd038 /* 37 vars */) = -1 ENOEXEC (Exec format error)
strace: exec: Exec format error
+++ exited with 1 +++

$ echo $WSL_INTEROP
/run/WSL/410_interop

$ ll /run/WSL
total 0
drwxrwxrwx  2 root root 100  9月 22 12:29 ./
drwxr-xr-x 26 root root 760  9月 22 12:29 ../
srwxrwxrwx  1 root root   0  9月 22 12:29 2_interop=
srwxrwxrwx  1 root root   0  9月 22 12:29 410_interop=
srwxrwxrwx  1 root root   0  9月 22 12:29 549_interop=

$ ll /proc/sys/fs/binfmt_misc/
total 0
drwxr-xr-x 2 root root 0  9月 22 12:29 ./
dr-xr-xr-x 1 root root 0  9月 22 12:29 ../
-rw-r--r-- 1 root root 0  9月 22 12:29 llvm-12-runtime.binfmt
-rw-r--r-- 1 root root 0  9月 22 12:29 llvm-14-runtime.binfmt
-rw-r--r-- 1 root root 0  9月 22 12:29 python2.7
-rw-r--r-- 1 root root 0  9月 22 12:29 python3.10
--w------- 1 root root 0  9月 22 12:29 register
-rw-r--r-- 1 root root 0  9月 22 12:29 status

$cat /proc/sys/fs/binfmt_misc/WSLInterop
cat: /proc/sys/fs/binfmt_misc/WSLInterop: No such file or directory

[WitherZuo]

The interoperability of WSL and Windows relies on WSL init. After systemd is turned on, WSL init will also run under systemd, which may be one of the reasons for breaking the interoperability function.

[Re4son]

I don't think it is by design @WitherZuo. I have both systemd and interop working temporarily in Ubuntu but it stops working when I start another WSL distro.
Once I shutdown WSL and start Ubuntu on its own it works again.

[reynoldsa]

The binfmt_misc handler for WSL2 to run Windows binaries either is getting overwritten by systemd or is never set:

systemd enabled

adam@ADAM-HOMEDESK /p/s/f/binfmt_misc> ls -lah
total 0
drwxr-xr-x 2 root root 0 Sep 22 12:20 ./
dr-xr-xr-x 1 root root 0 Sep 22 12:20 ../
-rw-r--r-- 1 root root 0 Sep 22 12:20 llvm-13-runtime.binfmt
-rw-r--r-- 1 root root 0 Sep 22 12:20 llvm-14-runtime.binfmt
-rw-r--r-- 1 root root 0 Sep 22 12:20 python3.10
--w------- 1 root root 0 Sep 22 12:20 register
-rw-r--r-- 1 root root 0 Sep 22 12:20 status

systemd disabled

adam@ADAM-HOMEDESK /p/s/f/binfmt_misc> ls -lah
total 0
drwxr-xr-x 2 root root 0 Sep 22 12:21 ./
dr-xr-xr-x 1 root root 0 Sep 22 12:21 ../
--w------- 1 root root 0 Sep 22 12:21 register
-rw-r--r-- 1 root root 0 Sep 22 12:21 status
-rw-r--r-- 1 root root 0 Sep 22 12:21 WSLInterop

[cerebrate]

Systemd does that, as those of us who've futzed with alternative systemd solutions in the past have come to know. The solution-slash-workaround is to put this file:

https://github.com/arkane-systems/bottle-imp/blob/master/othersrc/usr-lib/binfmt.d/WSLInterop.conf

in your /usr/lib/binfmt.d .

[Re4son]

Good pointer @cerebrate, thanks heaps! That helped.

I compared the distro that worked out of the box and the distro that didn't and the difference was existing config files in "/usr/lib/binfmt.d"

• Stock Ubuntu out of the box with nothing in "/usr/lib/binfmt.d" works fine
• Kali Linux with plenty of packages installed and thus several config files in "/usr/lib/binfmt.d" does not work

Ubuntu on its own works fine.
Kali doesn't work and also breaks Ubuntu.
Deleting everything under "/usr/lib/binfmt.d" in the Kali distro fixes Kali (and stops breaking Ubuntu).
Adding your file to "/usr/lib/binfmt.d" alongside the existing files fixes Kali (and stops breaking Ubuntu).

Thanks for that @cerebrate, much appreciate it.

[benhillis]

@Re4son - thanks for the additional detail. This boils down to Linux not supporting namespacing of binfmt_misc interpreters. Any fix we make is going to be problematic until that is resolved.

It looks like we need to handle the case where the distro has binfmt interpreters installed by adding our own .conf file though.

[dbbao]

有什么解决方案吗，我也碰到这个问题了

[PhyX-Meow]

Systemd does that, as those of us who've futzed with alternative systemd solutions in the past have come to know. The solution-slash-workaround is to put this file:

https://github.com/arkane-systems/bottle-imp/blob/master/othersrc/usr-lib/binfmt.d/WSLInterop.conf

in your /usr/lib/binfmt.d .

@lhrbest 这里已经给了 workaround 了

[dbbao]

我是Ubuntu 20.04，会报如下的错误：
``root@lhrxxt:/mnt/f/pwsh# cmd.exe
<3>WSL (996) ERROR: UtilConnectUnix:511: connect failed 2
root@lhrxxt:/mnt/f/pwsh#
root@lhrxxt:/mnt/f/pwsh# ll /usr/lib/binfmt.d/
total 12
drwxr-xr-x 2 root root 4096 Sep 26 15:30 ./
drwxr-xr-x 122 root root 4096 Sep 14 10:57 ../
-rw-r--r-- 1 root root 27 Sep 26 15:30 WSLInterop.conf
root@lhrxxt:/mnt/f/pwsh# cmd.exe
<3>WSL (1000) ERROR: UtilConnectUnix:511: connect failed 2
root@lhrxxt:/mnt/f/pwsh#
root@lhrxxt:/mnt/f/pwsh#

[PhyX-Meow]

@lhrbest Try restart wsl. Normally there should be a file named WSLInterop.

[kpko]

Hi,

I'm using https://github.com/yuk7/ArchWSL and enabling systemd reliably broke the ability to execute Windows applications from WSL. I resolved the issue by generating a file in /usr/lib/binfmt.d/WSLInterop.conf with the following content:

:WSLInterop:M::MZ::/init:PF

Also I removed mono.conf from the same directory as they might create a conflicting mapping. After running sudo systemctl restart systemd-binfmt, my executables are working now.

I've tried several binfmt entries but most of the entries I found online did not include the 'PF' flags and resulted in an error.

[romaluca]

Hi, i have the same problem and the file:
https://github.com/arkane-systems/bottle-imp/blob/master/othersrc/usr-lib/binfmt.d/WSLInterop.conf
not exists anymore.
Where i can find it?
Thasnk

[PhyX-Meow]

@romaluca It is here, https://github.com/arkane-systems/bottle-imp/blob/v0.13.0/othersrc/usr-lib/binfmt.d/WSLInterop.conf

[nicoandmee]

Hi,

I'm using yuk7/ArchWSL and enabling systemd reliably broke the ability to execute Windows applications from WSL. I resolved the issue by generating a file in /usr/lib/binfmt.d/WSLInterop.conf with the following content:

:WSLInterop:M::MZ::/init:PF

Also I removed mono.conf from the same directory as they might create a conflicting mapping. After running sudo systemctl restart systemd-binfmt, my executables are working now.

I've tried several binfmt entries but most of the entries I found online did not include the 'PF' flags and resulted in an error.

Thank you so much for posting this! ❤️ I am in the same situation. My executables are now working, though the appendWindowsPath directive still isn't adding them to the system PATH. But using a fully qualified path like /mnt/c/Windows/System32/clip.exe works 🤔

[cvernooy23]

Hi,

I'm using https://github.com/yuk7/ArchWSL and enabling systemd reliably broke the ability to execute Windows applications from WSL. I resolved the issue by generating a file in /usr/lib/binfmt.d/WSLInterop.conf with the following content:

:WSLInterop:M::MZ::/init:PF

Also I removed mono.conf from the same directory as they might create a conflicting mapping. After running sudo systemctl restart systemd-binfmt, my executables are working now.

I've tried several binfmt entries but most of the entries I found online did not include the 'PF' flags and resulted in an error.

I tried using this solution (for Kali) and while the window launches it is black. Is there anything else that needs to be done?

[ststefa]

Thanks a lot, this solved the problem also for me on Debian bullseye. Learned something new today ;)

[freckhard]

Hello everyone, thanks to your comments, I was able to solve the problem that my Windows executables were not working after the activation of systemd, just by adding the above mentioned entry to the WSLInterop.conf.

Here is a one-liner for you:

sudo sh -c 'echo :WSLInterop:M::MZ::/init:PF > /usr/lib/binfmt.d/WSLInterop.conf'

After wsl --shutdown and logging back in, now next to systemd, explorer.exe and the like are finally working again! ✓ 🥳

(tested on WSL2 @ Ubuntu22.04 and Debian bookworm/sid)

[mikerosile]

Thanks @freckhard your solution solved it for me, as well! I was using a custom Fedora distro.

[mwoodpatrick]

Systemd does that, as those of us who've futzed with alternative systemd solutions in the past have come to know. The solution-slash-workaround is to put this file:
https://github.com/arkane-systems/bottle-imp/blob/master/othersrc/usr-lib/binfmt.d/WSLInterop.conf
in your /usr/lib/binfmt.d .

@lhrbest 这里已经给了 workaround 了

I'm seeing this issue in #9778 however:

However, I get

wget https://github.com/arkane-systems/bottle-imp/blob/master/othersrc/usr-lib/binfmt.d/WSLInterop.conf
--2023-03-14 11:47:13-- https://github.com/arkane-systems/bottle-imp/blob/master/othersrc/usr-lib/binfmt.d/WSLInterop.conf
Resolving github.com (github.com)... 192.30.255.112
Connecting to github.com (github.com)|192.30.255.112|:443... connected.
HTTP request sent, awaiting response... 404 Not Found
2023-03-14 11:47:13 ERROR 404: Not Found.

Any pointers on the best way to resolve would be appreciated

[cerebrate]

Sorry, file moved in the 1.0 release. Try here:

https://github.com/arkane-systems/bottle-imp/blob/v0.13.0/othersrc/usr-lib/binfmt.d/WSLInterop.conf

[dead-claudia]

Hello everyone, thanks to your comments, I was able to solve the problem that my Windows executables were not working after the activation of systemd, just by adding the above mentioned entry to the WSLInterop.conf.

Here is a one-liner for you:

sudo sh -c 'echo :WSLInterop:M::MZ::/init:PF > /usr/lib/binfmt.d/WSLInterop.conf'

After wsl --shutdown and logging back in, now next to systemd, explorer.exe and the like are finally working again! ✓ 🥳

(tested on WSL2 @ Ubuntu22.04 and Debian bookworm/sid)

@freckhard Can confirm this worked for me as well.

[rafaspimenta]

The WSLInrerop.conf file solution did not work for me

sanr@GEO-00099386:/~ code .
run-detectors: unable to find an interpreter for /mnt/c/Program Files/Microsoft VS Code/Code.exe

In my case what solved was disabling the cli interpreter using the command:
sudo update-binfmts --disable cli
from here #4567 (comment)

[michaelmhoffman]

Following @tianon's instructions worked for me:

sudo systemctl mask systemd-binfmt.service

I then did wsl --shutdown and restarted. Running cmd.exe from WSL or anything else seems to work fine now.

This seems like the most sensible workaround as it disables the service that is removing the WSLInterop capabilities.

The sudo update-binfmts --disable cli workaround didn't work for me. That might be something that applies to a different root cause.

[g2flyer]

@g2flyer When you say "All that with (unchanged):" do you mean you didn't have to run the workaround?

Yes, windows update alone was what solved the issue ...

[michaelmhoffman]

This was reported on systemd as systemd/systemd#28126. @poettering says the following there:

wsl really should add their entry to /run/binfmt.d/ before invoking systemd, otherwise this will always conflict because we never know whether an entry we find in the kernel should be deleted because we can't find any config for it, or if it should stay because it was inherited down.

I don't think we should change anything in systemd here. On systems where systemd-binfmt is enabled the subsystem is kinda owned by it, and you cannot have multiple owners of it without stepping on each others toes.

Can you file a bug against WSL asking them to fix that? Fix should be simple: just provide whatever they want to provide in a drop-in /run/binfmt.d/ from their agent and things should be good.

Originally posted by @poettering in systemd/systemd#28126 (comment)

[binaryfire]

Will WSLInterop.conf be added by default in future releases? Would like to know if I need to add it to my setup playbook or not. Looks like the systemd guys won't be changing anything on their end.

[massodyssey]

I ran into the same issue on arch linux after install dotnet, notepad.exe cannot run, uninstall dotnet does not work. Then did google search, found page here, tried the one liner fix:
sudo sh -c 'echo :WSLInterop:M::MZ::/init:PF > /usr/lib/binfmt.d/WSLInterop.conf'

now works perfectly, including vscode. Just want to post and hope this would also help others.

[puetzk]

The suggestion to configure systemd-binfmt via /run/binfmt.d seems like something aligns with things that already generate interop units in /run for [boot]systemd=true, like WSLg injecting /run/systemd/generator.early/wslg-mount.service to bind the X11 socket.

Hopefully microsoft sees it the same way...

[LouDnl]

Hello everyone, thanks to your comments, I was able to solve the problem that my Windows executables were not working after the activation of systemd, just by adding the above mentioned entry to the WSLInterop.conf.

Here is a one-liner for you:

sudo sh -c 'echo :WSLInterop:M::MZ::/init:PF > /usr/lib/binfmt.d/WSLInterop.conf'

After wsl --shutdown and logging back in, now next to systemd, explorer.exe and the like are finally working again! ✓ 🥳

(tested on WSL2 @ Ubuntu22.04 and Debian bookworm/sid)

Fixed it for me thanks

[Karocyt]

Sorry, file moved in the 1.0 release. Try here:

https://github.com/arkane-systems/bottle-imp/blob/v0.13.0/othersrc/usr-lib/binfmt.d/WSLInterop.conf

Systemd does that, as those of us who've futzed with alternative systemd solutions in the past have come to know. The solution-slash-workaround is to put this file:

https://github.com/arkane-systems/bottle-imp/blob/master/othersrc/usr-lib/binfmt.d/WSLInterop.conf

in your /usr/lib/binfmt.d .

Worked for me on Win11 Pro Preview
( file now at commit 2742d2 )

[ArianSha]

Issue is still prevalent, hope this gets resolved soon. I had to disable systemd to link WSL ubuntu (22.04.02) to windows git credentials.

[alkuzad]

For me the binfmt went missing every docker run for some reason. I do not know the reason but the fix is obvious - either restart binfmt service or register the format using sudo echo ":WSLInterop:M::MZ::/init:PF" > /proc/sys/fs/binfmt_misc/register

I got tired of manually adding this pre-docker so I fight fire with fire and I made systemd service and path monitor that will do the same if WSLinterop file are missing. Seems to work so far, sharing so maybe some of you will benefit:

/etc/systemd/system/wslinterop-monitor.path

[Unit]
Description=Monitor for WSLInterop
After=systemd-binfmt.service

[Path]
PathChanged=/proc/sys/fs/binfmt_misc/WSLInterop
Unit=wslinterop-force.service
TriggerLimitIntervalSec=0

[Install]
WantedBy=multi-user.target

/etc/systemd/system/wslinterop-force.service

[Unit]
Description=Force WSLInterop
After=systemd-binfmt.service
ConditionPathExists=!/proc/sys/fs/binfmt_misc/WSLInterop

[Service]
Type=oneshot
ExecStart=echo ":WSLInterop:M::MZ::/init:PF"
StandardOutput=file:/proc/sys/fs/binfmt_misc/register

[Install]
WantedBy=multi-user.target

Then:

systemctl daemon-reload
systemctl enable wslinterop-force.service wslinterop-monitor.path
systemctl start wslinterop-force.service wslinterop-monitor.path

[puetzk]

It seems that Microsoft has now implemented a fix for this? at least on my machine (Windows 11 22H2, WSL version: 2.0.9.0) they now made a unit /run/systemd/generator.early/wsl-binfmt.service

# Note: This file is generated by WSL to prevent binfmt.d from overriding WSL's binfmt interpretor.
# To disable this unit, add the following to /etc/wsl.conf:
# [boot]
# protectBinfmt=false

[Unit]
Description=WSL binfmt restore
After=systemd-binfmt.service

[Service]
Type=oneshot
ExecStart=/bin/sh -c '(echo -1 > /proc/sys/fs/binfmt_misc/WSLInterop-late) ; (echo :WSLInterop-late:M::MZ::/init:P > /proc/sys/fs/binfmt_misc/register)'

This is still not the "correct" solution @poettering suggested in systemd/systemd#28126 (using /run/binfmt.d, so that the normal systemd-binfmt.service would know about their registration), but it should at least add their service back after both run (and it schedules itself to run immediately afterward).

https://docs.kernel.org/admin-guide/binfmt-misc.html
However, it has another shortcoming - compared to all the settings proposed here, it's missing the Flags: F (fix binary): https://docs.kernel.org/admin-guide/binfmt-misc.html, which means that this registration doesn't work in a chroot (or other mount namespace that doesn't have wsl's /init mounted).

[earlofhemsley]

#8843 (comment)

Thanks for this. It fixed my problem. I will come back later and read the rest of the solutions since this one is quite old and there may be a newer way to handle this issue now. Cheers!

[syigzaw]

I'm getting a very similar error, but for Linux commands instead. At the very beginning of a new terminal I get:

-bash: /usr/bin/grep: cannot execute binary file: Exec format error

And I also get this:

$ groups
-bash: /usr/bin/groups: cannot execute binary file: Exec format error

However, some other commands that can be found under /usr/bin/, like find and which, work fine for me. I've tried all of the suggestions above and they didn't help. And that isn't a surprise since I didn't actually have any issues running Windows executables, but this error message is the exact same, so I'm wondering if my issue is related.

For background info, this is a fresh WSL2 installation on Windows 10, and after installing WSL2 I installed Docker for Windows, while also setting autoMemoryReclaim to dropcache as suggested here.

[pfeileon]

Since the other thread ( #8952 ) was closed: I've just had this error with the git credential manager and simply restarting WSL (wsl --shutdown) solved the issue, so I'm not sure if systemd is at fault.

[smartexpert]

None of the above worked for me. However, on WSL2 running Ubuntu 20.04.04 LTS this worked:

sudo su -
echo ":WSLInterop:M::MZ::/init:" > /proc/sys/fs/binfmt_misc/register

Exit WSL on Powershell via wsl --shutdown and then restart WSL.

The solution was sourced from here: #10363 (comment) credits to @martin-rueegg

[trallnag]

Just ran into this randomly for the first time in a long time. Simple restart of the distributions seems to have fixed it.

WSL version: 2.2.4.0
Kernel version: 5.15.153.1-2
WSLg version: 1.0.61
MSRDC version: 1.2.5326
Direct3D version: 1.611.1-81528511
DXCore version: 10.0.26091.1-240325-1447.ge-release
Windows version: 10.0.22631.3880

[oxivanisher]

This happens to me sometimes with Ubuntu 24.04, but not always. I currently do not know what the difference between the WSL2 installations of the Ubuntu VMs where. I did the exact same thing, sometimes wslvar worked, sometimes not. 🤷‍♂️

[michaelmhoffman]

systemd/systemd#28126 is now closed. The systemd team regard this as an incorrect configuration by WSL, and suggest putting configuration in /run/binfmt.d before calling systemd as a fix.

My current setup (WSL 2.3.24.0, Windows 11 23H2 22631.4391) has the fix in /run/systemd/generator.earlywsl-binfmt.service and it seems to work for me, despite being less elegant than fixing it in /run/binfmt.d.

I suggest that the issue be closed as fixed.