AF_UNIX on Windows

[justinmk]

AF_UNIX is supported on Windows now, and has some advantages over named pipes:
https://devblogs.microsoft.com/commandline/af_unix-comes-to-windows/

What are your thoughts on adding support for this to the uv_pipe_* family of functions?

1. Since UV_NAMED_PIPE already means "named pipe" on Windows and "unix socket" on Unix, maybe a new UV_UNIX_SOCKET flag could be used to opt-into AF_UNIX behavior on Windows?
2. Alternative: on Windows versions where AF_UNIX is supported, UV_NAMED_PIPE could use AF_UNIX unconditionally. But this could break applications that assume pipe names have \\.\pipe\... format.

[saghul]

I think something along the lines of 1) would be best. We have uv_pipe_init_ex which. takes a flags argument so there is a place where this could be turned on.

[vtjnash]

I think the main challenge/question would be that the uv_pipe_* code currently expects to operate on a windows HANDLE object, whereas this would be a SOCKET object. So should this fall more under uv_tcp or uv_pipe? I expect it would share most of its code with uv_tcp and little with uv_pipe. On Unix, UV_NAMED_PIPE presently means {"pipe" or "fifo" or "unix socket"}. We could perhaps move the "unix socket" capabilities into the existing libuv stream socket representation (aka uv_tcp) though, which might be more cross-platform stable (due to somewhat better alignment with the underlying system APIs everywhere).

Aside: for anonymous local sockets (e.g. not with connect/bind on filesystem paths), in #1498, I am adding uv_socketpair for making local socket pairs.

[saghul]

That's kinda what Unix does with having most of the stuff in stream.c, right? Makes sense to do the same here IMHO.

[johnwason]

Windows sockets should be able to be used as file handle functions like ReadFile() and WriteFile() (https://docs.microsoft.com/en-us/windows/win32/winsock/socket-handles-2), so the existing pipe.c file can be slightly modified to use socket functions to create the handles, but otherwise be unchanged except in a few places. All named pipe filenames begin with \\.\pipe\ (or potentially another computer name in place of dot), and all unix sockets exist somewhere on the filesystem, so the logic can choose to create a pipe or socket based on the location requested.

[o2genum]

Anyone already working on this?
Or maybe somebody is certain where this fits within libuv?

Really cool thing for IPC on Windows, I'd like to contribute.

[lppedd]

Sorry for simply bumping the issue, but this is a deal breaker for IPC on a single host.

Having to resort to named pipes on Windows only is additional maintenance cost that I'm not sure I'm willing to pursue.

Is there any plan to bring UDS into the library during 2023?

[bnoordhuis]

I think (but am not 100% sure or adamant) that it would be easiest to wait until we drop support for Windows 8 / 2012 at the end of the year (#3889) before adding AF_UNIX support, because then libuv lives in a world where it can simply assume AF_UNIX exists.

Open to be convinced otherwise though, and the above doesn't mean pull requests aren't welcome yet.

[lppedd]

@bnoordhuis AF_UNIX seems to be available starting from the Windows 10 April 2018 update, so I think it's impossible to take for granted it exists, unfortunately. I mean, Windows 10 machines should be up-to-date, tho one cannot be sure about it.

[amoiseev]

I think using AF_UNIX should produce a clear error on older Windows versions.

The error has to be distinguishable enough for the program to be able to fall back to using a TCP socket, for example.

[clemenswasser]

Before anyone waste time into this (as I did yesterday, albeit not related to libuv), it's important to note that the blog post announcing 'AF_UNIX support' contains misleading information and is partially incorrect. The implementation seems to be incomplete and was likely abandoned shortly after the transition from WSL1 to WSL2. For instance, the 'abstract' socket address mode does not function as stated in the blog post. You can find a detailed description of the issue and its observed behavior here. Unfortunately, I encountered this problem and the GitHub Issue only after investing significant time into it 😢. Therefore, the only viable option for using AF_UNIX sockets for IPC on Windows is to utilize 'pathname' sockets that depend on the file system, which is less than ideal.

[mcilloni]

@clemenswasser

the only viable option for using AF_UNIX sockets for IPC on Windows is to utilize 'pathname' sockets that depend on the file system, which is less than ideal.

This is AFAIK already the case with libuv with macOS, which only supports classic UNIX sockets (libuv has no support for Mach Ports AFAIK), and the same goes for *BSD. I don't see a reason why supporting "classic" UDS should be considered acceptable on macOS/Darwin/*BSD and not on Windows, honestly.

[reitowo]

This definitely needs to be landed because on Windows UDS beats named pipe completely in performance.

[reitowo]

About performance, at least under ASP.NET Core gRPC use case, there's a test about it:

https://www.yanxurui.cc/posts/server/2023-11-28-benchmark-tcp-uds-namedpipe/

It seems the UDS outperforms localhost TCP and NamedPipe significantly.

cc @bnoordhuis

[reitowo]

Reusing UV_NAMED_PIPE seems not a problem to me, no new flag needed.

As above said, currently the UDS on Windows needs file system support.

Therefore, the only viable option for using AF_UNIX sockets for IPC on Windows is to utilize 'pathname' sockets that depend on the file system, which is less than ideal.

But considering the performance gain, it still worth supporting.

1. Named pipe has prefix \\.\ or \\?\ while UDS is an absolute local path like C:\Temp.sock (on Windows, obviously), the actual one can be determined by path prefix, reusing existing parameters. So when user use \\.\ as usual just use existing behavior to create a named pipe, while other paths use the new api to create a file based uds, avoiding breaking changes.

2. By doing this, I think UDS can be supported conditionally at Windows 10+, and simply throwing exception at Windows 7/8 if prefix is not named pipe with a dummy implementation. It even doesn't need to wait for sunsetting Windows 8.

[bnoordhuis]

We dropped Windows 8 support (#3889) so this feature request is available for anyone who wants to work on it.

[reitowo]

Anyone working on it yet? I found modifying the existing code challenging. Not sure how to meet the requirements of libuv because the code seems very hacky and tied up with NamedPipe.

Any suggestions? Should I target reusing existing codes (which seems not possible), or is it acceptable to implement new set of uds versions of pipe_connect2, pipe_bind2, etc... and use a flag to determine if the handle is using a unix domain socket? Like

int uv_pipe_connect2(...) {
  if (!uv__has_named_pipe_prefix(name)) {
    return uv__uds_pipe_connect2(...);
  }
}

And on callbacks use a flag like handle->flags & UV_HANDLE_WIN_UDS_PIPE

[bnoordhuis]

That works, as long as it doesn't result in massive code duplication.

[reitowo]

I find most of the IOCP code works like a charm when use socket as handle as Windows supports it natively. So I decide to reuse most of the code.

https://learn.microsoft.com/en-us/windows/win32/winsock/socket-handles-2
https://tinyclouds.org/iocp_links

I started a WIP PR here, currently send and receive data works.

The server part is almost work, remaining some work about AcceptEx.

I didn't touch the pipe pair code, it seems we can just leave it there as usual use NamedPipe and just make UDS supports client and server connections.

var net = require('net');
const ret = net.createConnection({ path: "D:/uds.temp" });

ret.on('connect', () => {
    console.log('connected')
    setInterval(() => {
        ret.write('test')
    }, 10)
})
ret.on('data', (data) => {
    console.log(data.toString())
})

and

const srv = net.createServer((c) => {
    console.log(c)
})
srv.listen("D:/uds2.temp")