Implement Bun.serve({fd}) and Bun.listen({fd}) for socket-activated deployments

[shivak]

Overview. This PR implements Bun.serve({fd}) and Bun.listen({fd}), which runs the HTTP (or TCP) server on a socket that has are already been bound and listened. The overall idea is to have a separate process bind() and listen() to a socket, and pass it to the Bun process. This is used in production deployments for:

• efficiency: servers can be started on demand in, e.g., large multi-tenant edge setups ☺
• security: a privileged process can bind port < 1024 and then pass the socket to an unprivileged server. Also, the socket can be passed into a restricted network namespace wherein the server is run. This allows the server to respond to requests, but blocks all other network activity.
• reliability: socket activation can implement failover.

This feature is available in Go, Node, etc but not Deno (denoland/deno#6529). Here, it's mildly tricky to implement, since it involves changes to two layers of low-level platform bindings - see the companion PRs at the bottom.

Usage. Typical users (on Linux/systemd) would call Bun.serve({fd: process.env.LISTEN_FDS}). On Mac/launchd, the fd is obtained from launch_activate_socket. On Windows, it could ostensibly be transferred by WSADuplicateSocket.

Workaround. socat or systemd-socket-proxyd. These aren't ideal for efficiency, security, or reliability.

Notes.

1. The Zig and C++ code correctly handle the socket type, which is usually just an fd, but isn't on Windows. The Javascript API is simple and oblivious, expecting positive ints to be passed.

2. This introduces an asymmetry between listen() and connect(), since now you can listen() to an fd, but you cannot connect() to one. Hence the somewhat awkward FdOrUnixOrHost type, which I don't think is worth fretting over.

3. Testing requires some Zig code to perform the bind() and listen() typically done out-of-process.

4. I copied the library changes to _libusockets.h and libuwsockets.cpp. These seem to differ in minor (unnecessary?) ways from the underlying libraries, for reasons I don't currently understand.

5. I haven't tried running this on Windows. Also, as @Jarred-Sumner mentioned, some sanitization of fd might be useful. Also, I've never actually written any Zig code, so maybe double-check things.

Building. For politeness, I didn't redirect any submodules. Wait for the following PRs to be merged, or use my feature branches.

• Add support for already-listening sockets Jarred-Sumner/uSockets#3
• Add support for already-listening sockets Jarred-Sumner/uWebSockets#10

[shivak]

Aside: this is probably the safest way to deploy with good multicore performance. The key is SO_REUSEPORT, which Bun uses by default. But there are downsides to making each Bun worker handle its own socket binding:

• security: each worker is unprivileged, and has to keep the port open for additional workers. But then any user process can interfere in the pool.
• usability: No error is thrown when port is taken #1443

With fd, the privileged monitor gets sockets and passes them to only legitimate workers. With multiple workers, a monitor process is anyways necessary, since crashed workers need to be detected and restarted.

[Jarred-Sumner]

Hey, this PR is great and really appreciate the work you put in and I intend to merge it. Trying to ship the bundler so I haven’t had a chance to review in depth yet but @cirospaciari can you take a look?

…

On Fri, May 12, 2023 at 10:08 AM Shiva Kaul ***@***.***> wrote: Aside: this is probably the safest way to deploy with good multicore performance. The key is SO_REUSEPORT, which Bun uses by default. But there are downsides to making each Bun worker handle its own socket binding: - security: each worker is unprivileged, and has to keep the port open for additional workers. But then any user process can interfere in the pool. - usability: #1443 <#1443> With fd, the privileged monitor gets sockets and passes them to only legitimate workers. With multiple workers, a monitor process is anyways necessary, since crashed workers need to be detected and restarted. — Reply to this email directly, view it on GitHub <#2852 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAFNGSYSHLJHMDZF5DY6NEDXFZVAZANCNFSM6AAAAAAX6NLHO4> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[cirospaciari]

1. The Zig and C++ code correctly handle the socket type, which is usually just an fd, but isn't on Windows. The Javascript API is simple and oblivious, expecting positive ints to be passed.

Probably we will need an abstraction here when Windows support for Runtime release.

2. This introduces an asymmetry between listen() and connect(), since now you can listen() to an fd, but you cannot connect() to one. Hence the somewhat awkward FdOrUnixOrHost type, which I don't think is worth fretting over.

The most common use case should be listen() we can work on connect() later if requested.

4. I copied the library changes to _libusockets.h and libuwsockets.cpp. These seem to differ in minor (unnecessary?) ways from the underlying libraries, for reasons I don't currently understand.

This API diverged slightly from the current C API of WebSockets because the C API itself was not done yet when Bun started to use it, and Bun required some changes to achieve some features (like sendfile) and add fixes. So uws.zig bindings actually use libuwsockets.cpp not the capi/libuwebsockets.cpp

5. I haven't tried running this on Windows. Also, as @Jarred-Sumner mentioned, some sanitization of fd might be useful. Also, I've never actually written any Zig code, so maybe double-check things.

I'm working on Windows support, this can be added later.

Building. For politeness, I didn't redirect any submodules. Wait for the following PRs to be merged, or use my feature branches.

• Add support for already-listening sockets Jarred-Sumner/uSockets#3
• Add support for already-listening sockets Jarred-Sumner/uWebSockets#10

I take a look at uSockets and uWebSockets PR's and looks to be great. Probably we will need more testing like listening to the same socket twice on the same loop.

LGTM, thanks @shivak

[shivak]

This PR touches some critical codepaths (creating any kind of server) and the underlying libs, so it is a good candidate to merge before a stable release. In the next few days, I'll resolve the conflicts that have arisen.

[infrahead]

@Jarred-Sumner Is the plan for this to be merged still? I'm hoping to migrate my Node.js web services to Bun, but using systemd sockets is how I architect things for zero-downtime deployments. Alternatively I can look into SO_REUSEPORT which I know works with uWS, which on modern Linux kernels might also be part of a solution for zero-downtime setup, but before I look into it was curious on when might get fd support.

[karimfromjordan]

I would also like to express my interest in this feature. In Node.js this isn't an issue.

[Jarred-Sumner]

I would like for us to do this, but I think it'll need a new PR due to bitrot