Consider NIF-able engines such as Duktape after SpiderMonkey update is complete

[wohali]

Summary

#1875 is a good step for us in CouchDB 3.0. However, we don't want to lose the good work Paul did on ChakraCore completely. Perhaps we can re-use a bunch of his NIF work for Duktape.

Desired Behaviour

#1875 gives us a new couchjs.

In this ticket, couchjs would go away and all JavaScript handling would be done inside of a NIF.

Possible Solution

Additional context

Could improve JS performance significantly.

[garrensmith]

What would be the advantage of using Duktape over v8 or Node?

…

________________________________ From: Joan Touzet <[email protected]> Sent: Tuesday, January 29, 2019 7:19 PM To: apache/couchdb Cc: Subscribed Subject: [apache/couchdb] Consider Duktape after SpiderMonkey update is complete (#1884) Summary #1875<#1875> is a good step for us in CouchDB 3.0. However, we don't want to lose the good work Paul did on ChakraCore completely. Perhaps we can re-use a bunch of his NIF work for Duktape<https://www.duktape.org/>. Desired Behaviour #1875<#1875> gives us a new couchjs. In this ticket, couchjs would go away and all JavaScript handling would be done inside of a NIF. Possible Solution Additional context Could improve JS performance significantly. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub<#1884>, or mute the thread<https://github.com/notifications/unsubscribe-auth/AAK9Am93iu3e2V4k_lvSHk20v7LaFdDcks5vIIK2gaJpZM4aYiQj>.

[wohali]

@garrensmith Per @davisp V8 is single thread per OS process, so we can't NIF it efficiently. This would be NIF-able.

[akircher]

I don't know about how important being NIF-able is. But I would like https://github.com/denoland/deno to also be considered. It's like node.js but better sandboxed and is more modern and Web API compatible

[wohali]

@akircher This ticket is only about NIF-able engines. While deno has some sandboxing, we're content to stay with SpiderMonkey for an external process.

[davisp]

I looked at Duktape once but it seemed to not support a lot of JavaScript features according to [1]. Though I haven't got a clue on how much of that is stuff that would actually be used in the context of CouchDB.

[1] https://kangax.github.io/compat-table/es6/

[akircher]

I don't know erlang and don't know what makes something NIF-able. However Deno has released Web Workers which (1) run in their own separate thread, (2) has an API based on web-standards, (3) has all of the sandboxing that Web Workers provide, and (4) supports all modern Javascript.

Maybe this could work in Couch DB?

[popojargo]

^Would like to know what NIF-able means

[wohali]

Means that it has to be embeddable directly into the Erlang VM in such a way as to be compatible with its environment:

http://erlang.org/doc/man/erl_nif.html

Separate thread is actually a detraction in this situation.

[davisp]

Deno is based on V8 which I believe is still single interpreter per OS process so wouldn't be useful in this context.

[mibes]

I don't want to side-track this discussion, but have you considered Lua as an alternative (or additional) scripting language?

When you are looking to integrate the engine into the core CouchDB code you could benefit from this Erlang implementation: https://github.com/rvirding/luerl

It comes with a built-in sandbox that allows you to control resources and timeouts.

I understand Lua isn't as popular as JavaScript, but it is used by a number of other data stores, for example Redis, so it may be fit for purpose.

[wohali]

Our forward path in general is the already-present Mango which covers a large majority of cases.

JavaScript is necessary for backward compatibility, so we can't drop it entirely.

We know about Luerl.

[wohali]

@dch mentions QuickJS Javascript Engine (https://bellard.org/quickjs/) - from bellard, supports es2019.

[chintan-mishra]

I would like to mention Rustler for safe NIFs to reduce chances of crashing BEAM VM.

[wohali]

@chintan-mishra good to know, but that doesn't help specifically for JavaScript embedding. Erlang -> Rust -> JS just means twice the symbol marshalling, which is already our performance bottleneck. And I'm not about to reimplement a JS interpreter in Rust... ;)

[chintan-mishra]

@wohali that mention was in regard to Deno for JS engine.

[wohali]

@chintan-mishra got it, but see Paul's comment: #1884 (comment)

[davisp]

Also I found out that v8 itself isn't single threaded. Just Node.js's use of v8 so I've been wrong on that for quite some time. On the other hand I've also discovered that apparently most projects that use v8 end up vendoring a copy and building it as part of the build chain rather than re-use standard OS packages. So while v8 might be a usable engine its got a pretty terrible dependency story given the ASF process.

I'm fairly intrigued by that QuickJS project but given its only about two weeks old I'm a bit hesitant to pursue it very far. Seems almost too magically awesome given the relative size of other implementations both in team size and code size.

[janl]

I think modern spidermonkey is equally fine, it should be relatively straightforward to port our bindings to the new API for someone more fluent in C/++ than I am, but given enough time even I think I could get it done which means it can't be that hard. It's just a rather thankless job.

[wohali]

I'm grabbing the SM modernisation effort, working off of @janl 's work to date. It's a "would be nice" for 3.0, but it comes after a bunch of other work on my pile right now (like improvements and bugfixes to docker and the packaging.)

Let's keep this ticket for non-SM stuff. The SM update ticket is #1875.

[wohali]

We seem to be very happy with SM60, and in a good place to update to newer versions of SM.

Closing until we need a non-SM replacement.