deno support through more generic FFI

[konsumer]

Deno is in a transitional period for native-libraries, right now. Native plugins are getting removed (#8490) in 1.13 and replaced with FFI (#11152). An upside of this is, in my opinion, is that FFI will be the generic interface, mostly useful in any language that supports it (c, deno, python, node, rust, etc.)

Recently, I wanted to make a visual canvas that emulates the browser API, but in a streamlined canvas-only graphical app (that can run on a pizero, as an example.) I set out to find something that would make the canvas. It seems like all the canvas-solutions I found for deno are either using the deprecated native plugin API, or use WASM for offscreen-rendering, mostly. Many of the solutions that are already made in JS (for node) are NAPI or neon. Particularly skia-canvas caught my eye, which uses neon. It's also off-screen, so I'd need to pass those buffers to a windowing library, but that seems like a small change to add, if everything else was bound and working. Looking at their elegant binding-code, I became very impressed with neon. I am a pretty weak rust programmer, but it was easy to follow and understand, and seems like a real joy to work with.

I'd love to be able to write similar code (or even use existing neon-bound projects) with plain FFI. In deno we have a few numerical types, and in recent PR, that will probably be in soon, byte-buffers. Does this seem possible to work with neon, directly or over some sort of adapter-layer? Am I thinking about this all wrong?

[kjvalencik]

@konsumer This is interesting! I need to understand the FFI design in Deno a bit better. Is that documented anywhere or does it require reading the source right now?

I'm skeptical that Neon could support both Node and Deno because many APIs in Neon are designed specifically with the internal mechanics of Node in mind and I'm not sure if those would hold up for Deno. However, I wouldn't be sure until I understood more.

Neon is going through a bit of a transition at the moment while it gets close to v1.0 release. After that release, the original ("legacy") backend written in C++ will be removed. I would want to wait until then to consider adding another backend because the code already has a lot of #[cfg(..)].

I would expect Deno to have an official FFI SDK written in Rust (sort of how Node has one in C++) since it is written in Rust. Do you know if there are plans for that?

[konsumer]

I'm skeptical that Neon could support both Node and Deno because many APIs in Neon are designed specifically with the internal mechanics of Node in mind and I'm not sure if those would hold up for Deno. However, I wouldn't be sure until I understood more.

Me too! I know that NAPI is so much more rich than these basic types, and as I understand it, deno wants it that way for security, so I could see it being a totally dead-path. I could also see making a kind of neon-deno that implements something sort of similar, totally separately, and maybe limited, over FFI.

Neon is going through a bit of a transition at the moment while it gets close to v1.0 release. After that release, the original ("legacy") backend written in C++ will be removed. I would want to wait until then to consider adding another backend because the code already has a lot of #[cfg(..)].

That seems very sensible. I am working on building up my rust skills, so maybe by then I will be able to actually help more.

Is that documented anywhere or does it require reading the source right now?

I would expect Deno to have an official FFI SDK written in Rust (sort of how Node has one in C++) since it is written in Rust. Do you know if there are plans for that?

It's all very tentative, right now. here is the PR that implements the first pass (no string/buffer.) And here is a non-merged PR for basic buffers/strings.

I am working on this lib as a simple "learn how deno FFI works" project, and to help deno-canvas (offline-only, wasm-compiled canvas api) make windows.

On deno-side, it looks like this and on rust-side, it looks like this (using no mangle and pub extern "C")

As I understand it, it's meant to not have any native-side API requirements, so you can wrap anything that can expose FFI (C DLL interface.) So it's roughly more like node-ffi or luajit ffi, etc, than NAPI. I imagine neon would be an adapter layer between those. like if on the deno side it had utils to unwrap buffers into nicer neon-types, and on the rust side it did similar. I really have no idea how it would actually work, though.

[konsumer]

Personally, my eventual goal is to have both a native windowing library, and a canvas API (either in wasm or directly in same FFI lib, to keep memory-transfers fast, and skip all the memory-allocation with ffi<->deno<->wasm.) If on top of that, if we also had neon-like binding (to ease writing deno-libs, but also ease porting some rust-made node-libs, and pass more advanced types between deno and rust) I think it would open up a lot of possibilities, for node & deno, but also any language with plain FFI bindings (I could imagine using the same DLL, with neon in lua, or python, with a little code on the other-language-side.) This would mean (if I am imagining it right) that node/lua/python/c people could all use the same rust canvas/window library I am working on, by loading a little neon-adapter in their language, and not much more.