Unix Domain Sockets on Windows

[JasperVanEsveld]

Proposal

Problem statement

Currently Unix Domain Sockets (UDS) are only supported on Unix systems.
However, unlike the name suggests, they have been available on Windows as well ever since Windows 10 Insider Preview Build 17063 in 2017.
There is a crate that adds Windows UDS functionality, but having the split from the std (linux/mac) UDS and Windows crate causes some issues.

The biggest pain point is tokio/mio which many other crates rely on and
One such crates is Deno, which has UDS support available on Linux under a unstable flag due to this problem.
There are many more, some can be seen referencing this issue.

Motivating examples or use cases

With the current implementation UDS is effectively limited to Linux/Mac only.
While it can be a great tool for Inter Process Communication (IPC).

One example use case I personally wanted to try was high throughput communication between Deno and WebView.
Many projects now rely on things like named pipes, or just forgo the idea of supporting Windows.
Adding Windows support could spread the use of UDS as a method for IPC not only on Windows but also Linux as crates no longer have to choose between cross-platform support or UDS.

Solution sketch

The challenges for a solution can best be described with a couple of questions

• Why has there not been a solution yet?
  From what I gathered it seems like Windows 7/8 have been the blocking factor, both not supporting UDS and being a Tier 1 supported platforms.
  As this will no longer be the case with Rust 1.76, maybe that is a good moment to introduce UDS with Windows support.

• Are there major difference between the platforms UDS implementations?
  This is something I am not knowledgeable in, contacting the contributors behind uds_windows could shed some more light on this.

So a solution could look like an UDS Windows implementation that is released with Rust 1.76.
If there aren't any breaking difference I would propose to make it available under the same API.
The work from uds_windows could be used as a reference, maybe some of the contributors could help upstream the implementation.

Alternatives

Don't implement Windows UDS into the standard library.
Though this will be confusing, especially once Windows 7/8 is dropped as a Tier 1 platform.

Links and related work

• Rust crate with UDS Windows support: uds_windows
• Original issue: Unix domain sockets on Windows rust#56533

What happens now?

This issue is part of the libs-api team API change proposal process. Once this issue is filed the libs-api team will review open proposals as capability becomes available. Current response times do not have a clear estimate, but may be up to several months.

Possible responses

The libs team may respond in various different ways. First, the team will consider the problem (this doesn't require any concrete solution or alternatives to have been proposed):

• We think this problem seems worth solving, and the standard library might be the right place to solve it.
• We think that this probably doesn't belong in the standard library.

Second, if there's a concrete solution:

• We think this specific solution looks roughly right, approved, you or someone else should implement this. (Further review will still happen on the subsequent implementation PR.)
• We're not sure this is the right solution, and the alternatives or other materials don't give us enough information to be sure about that. Here are some questions we have that aren't answered, or rough ideas about alternatives we'd want to see discussed.

[pitaj]

I recommend giving this issue a more meaningful title

[the8472]

The windows version of AF_UNIX only supports a subset of what most (all?) unixes support. So we have to be careful to not reduce ourselves to a lowest common denominator that's lower than what's already supported across unixes.

[ChrisDenton]

I think this needs some design work. Like should there be a common subset std::net or should this be only std::os::windows? If it's in std::net then what happens if the OS doesn't support something (Windows isn't the only non-posix OS around, although admittedly uds is common even then)?

Maybe fully answering those question can be deferred until after an initial implementation.

[JasperVanEsveld]

Oeps, thought I already filled in a title 😅

I agree that it depends on the feature set, and you don't want to limit other platforms if Windows doesn't support something.
Maybe @haraldh could help answer that?

I don't mind to help work on this issue, though I don't have much Rust experience, let alone it's standard library...

As for other non-posix OS that don't support UDS, I don't think there is one on the Tier 1 support list.
So maybe that is not an issue?

[haraldh]

I am no windows expert.. I just stripped https://github.com/Azure/mio-uds-windows to the relevant parts. You should ask there. But maybe it's all coming from https://github.com/yoshuawuyts/miow

[haraldh]

@alexcrichton seems to have been a major contributor to miow

[jmillikin]

Putting these types into std::os::windows::net seems like the best location, using a different external type (not just a typedef) to avoid unintentional extension trait coverage. Maybe some of the internals can be shared, but I'm not sure about that, because Winsock is different enough from BSD sockets in the details.

Some rough thoughts:

• Incoming and SocketAddr should be renamed UnixIncoming and UnixSocketAddr, probably -- the existing names in os::unix::net are bad enough, it'd be double-confusing if os::windows::net::SocketAddr is specific to AF_UNIX.
• UnixListener and UnixStream seem fine as-is.
  • Might have to drop some of the inherent methods that won't work on Windows, such as UnixStream::pair().
• Ancillary data isn't stable in Rust and isn't supported on Windows, so that whole part of the API can just be dropped.
• The Microsoft documentation for this feature is incomplete, for example AF_UNIX isn't listed as a supported address family on https://learn.microsoft.com/en-us/windows/win32/api/winsock2/nf-winsock2-socket -- whoever chooses to implement this might have to spend some time digging through the SDK headers.
• Traits AsRawFd etc need to be replaced with SOCKET-specific equivalents under std::os::windows::io.

[ChrisDenton]

Having a common cross-platform type would be very useful. Yes it would be limited to a common subset but that's fine.

Otherwise the user will need both #[cfg(unix)] and #[cfg(windows)] versions for basically identical code.

[jmillikin]

Having a common cross-platform type would be very useful. Yes it would be limited to a common subset but that's fine.

Otherwise the user will need both #[cfg(unix)] and #[cfg(windows)] versions for basically identical code.

I think this might be required due to the presence of Windows-incompatible inherent methods on the os::unix::net::UnixStream struct. I've seen comments from libs-team members that the standard library rules don't allow #[cfg(target_os)] guards on inherent methods, and it would be a breaking change to move stable API such as UnixStream::pair() to extension traits.

Hopefully code that only uses the common subset could do something like this:

#[cfg(windows)]
use std::os::windows::net::{UnixListener, UnixStream};
#[cfg(unix)]
use std::os::unix::net::{UnixListener, UnixStream};

// can now refer to `UnixListener` and `UnixStream` without `cfg`, so long as only
// common API is used.

[ntninja]

@jmillikin: Correct me if I’m wrong, but looked over this in not too much detail the only stable inherent method on the current UnixListener/UnixStream structs that does not have a direct Windows counterpart is the UnixStream::pair() function you mentioned. If that is indeed the case wouldn’t be feasible to move these structs to std::net (re-exporting them in std::os::unix of course) and emulating that function from the Rust side?

The uds_windows crate has some implementation of this (I’m pretty sure it could be done without spawning a thread just to block on accept() by using a non-blocking connect() after the bind() and before the accept()): https://github.com/haraldh/rust_uds_windows/blob/f45ff960e335a50fb82bef97306858d3e6198355/src/stdnet/net.rs#L178-L200

[jmillikin]

@jmillikin: Correct me if I’m wrong, but looked over this in not too much detail the only stable inherent method on the current UnixListener/UnixStream structs that does not have a direct Windows counterpart is the UnixStream::pair() function you mentioned. [...] and emulating that function from the Rust side?

Do you want to guarantee forever that the API surface of UDS sockets on Unix and UDS sockets on Windows will stay largely identical? I wouldn't feel comfortable making the Rust standard library assert that guarantee.

Using separate structs (even with largely similar API) lets the OS-specific behavior drift over time as necessary to match updates from the OS vendor.

Regarding pair(), my general preference is to avoid implicit emulation of platform-specific behavior -- many deep scars from debugging libraries that went down that path (e.g. gnulib).

If that is indeed the case wouldn’t be feasible to move these structs to std::net (re-exporting them in std::os::unix of course) [...]

std::net is for OS-agnostic networking -- Unix sockets are OS-specific, so a Windows implementation would be best exported from std::os::windows::net.

[Amanieu]

We discussed this in the libs-api meeting and we're happy to add this as a Windows-specific API under std::os::windows::net. We don't think this belongs under std::net because it is still fundamentally an OS-specific API.