Runtime bindings

[ematipico]

At the moment we are able to use rome only via CLI, which is fine for now, but it slows down adoption, because we can't use it in a runtime environment.

My proposal here is start creating official binding for the most famous runtimes: wasm, node and deno.

The idea is to create some APIs to use the tools we have. Here's an example with Node.js runtime:

const rome = require("rome");

let content = "let var";

async function main() {
  let formatted_code = await rome.format(content, {
    lineWidth: 120,
  });
}

Under the hood the format function will communicate with our CLI via child_process, using stdin and stdout, and by defining a protocol of communication between the processes:

function format(content, options) {
  let args = mapOptionsToArgs(options);
  let child =  require("child_process").spawn(PATH_TO_BINARY, "format", ["--runtime", ...args],  ...processOptions );
}

The --runtime is just a way to tell the CLI that a runtime is calling it, so it should do something different with the output and the errors.

We decode the stdout and return the result. Of course this is a tiny example to give an idea.

The core of the bindings will orbit around these concepts:

• a protocol where a runtime can communicate with the CLI and vice-versa
• decoding/encoding of the messages using the rules of the protocol, so both ends can understand the message (for rust we can use serde, for the runtime we will have to do things ad hoc)
• using stdin, stdout and stderr as channels

This would pave the foundation for the integration for any kind of runtime. For example, a Ruby on Rails application can create their own protocol decoder and runtime, so they can use Rome to format/lint their CSS/JSON files.

[ematipico]

This idea is not new, but widely inspired from what esbuild does: https://github.com/evanw/esbuild/tree/master/lib

• shared/ contains the specification of the protocol
• npm/ the APIs for node.js
• deno/ the APIs for the deno runtime

[leops]

The "child_process" part of the proposal overlaps a bit with the idea of having a persistent "daemon" process shared between runs of the CLI and possibly the LSP frontend for editors. In this model the Workspace trait is what would act as an API boundary between process, having been designed from the start so that all procedure calls on the workspace can be serialized and sent between remote contexts.

Now I don't think we would want to commit to making the Workspace interface stable, but I could envision the various "Rome SDKs" being built on top of this interface: the client code would be responsible for connecting to an existing daemon process or spawning it if necessary (at first this process could be hosted as a special mode of the CLI and the SDK logic would be simplified to "always spawn"), then would expose high-level commands implemented in JS and translating these to low-level workspace remote procedure calls.

WASM is a slightly different story because it can be used as a build target for the Rust code of the workspace server, letting it be run in "hosted mode" directly in the client process. Since the Workspace is mostly isolated from the underlying OS and platform it wouldn't need any tweaks to run in WASM mode, however the WASM platform currently lacks multi-threading capabilities so this may impact performance

[IWANABETHATGUY]

If you are looking for node binding, https://github.com/napi-rs/napi-rs should be a good option, which is widely used in swc, parcel

[ematipico]

Yeah I was aware of it, but I am not sure this the best option for what we want to do. That's why I didn't consider it.

Still, it's worth evaluate it

[MichaReiser]

I like the idea.

What are your thoughts around using wasm for the node and deno bindings? Is this less desirable because of performance reasons (use of native file system APIs rather than using trampoline calls into the host environment)

[ematipico]

I believe we should give both as alternatives. A user can use the wasm bindings or the node ones too. The wasm bindings, as Léo said, they are like a "self hosted" version of Rome, so they can be used pretty much anywhere as long as the host supports WASM, but as you said there might be some performance penalties so giving the runtime bindings is also a good alternative.

[MichaReiser]

I think it could be worth to start with the wasm bindings and do some profiling to understand the slowdown compared to the CLI version before working on a node and deno binding so that we can make an informed decision why we want to support 3 bindings instead of just one (and 4 if you want to support bun too).

[ematipico]

Sounds like a good plan. I can already say that using wasm is not very practical, because we rely on wasm-pack. wasm-pack allows to build for nodejs, but if we desire to build for ESM, importing the module will result in hell 😆 And if we decided to wrap wasm for deno, the loading of the wasm code will have to be done manually.

[MichaReiser]

I wasn't aware that wasm-pack has no deno support. My idea was to create a manual wasm API that may or may not use the generated one internally. This will allow us to tweak the ergonomics of the API). This may, hopefully, also help us to expose an ESM API?

[ematipico]

It's essentially what esbuild does :) but they have to work around the WASM generated by go