Loading binaries in ES Modules

[guybedford]

Given that we removed support for .node files in ECMAScript modules, the assumption is that we would use process.dlopen to load binaries.

process.dlopen will create a new binary instance each time it is used or throw an error. Will this still be sufficient for binary loading?

[guybedford]

//cc @jasnell

[devsnek]

we can just pass .node through to the cjs loader. simple change.

[guybedford]

@devsnek that means that any module that imports a .node file is now no longer browser-compatibile unless it does so in a conditional with dynamic import.

[devsnek]

huh

[ljharb]

Any module using process.binding wouldn’t be either, though?

[addaleax]

I’m confused – is this about process.dlopen() (official API for loading native addons manually) or process.binding() (deprecated, evil access to Node’s internals)?

[addaleax]

Also note that process.dlopen() doesn’t have nice effects of require() like ensuring that the returned value is a singleton for each file. When calling it twice on an addon, it will either fail or give you a new instance of the module.

[guybedford]

Ah, I mixed this up with process.dlopen yes :) Thanks @addaleax for correcting me here.

[addaleax]

@guybedford I do feel like ESM should have a better way than raw process.dlopen() for loading addons.

[guybedford]

Also note that process.dlopen() doesn’t have nice effects of require() like ensuring that the returned value is a singleton for each file. When calling it twice on an addon, it will either fail or give you a new instance of the module.

Interesting - could it be made into a singleton do you think?

[guybedford]

@addaleax sure, will reopen for discussion.

[devsnek]

they should absolutely be a supported thing to import. they have their own file extension that isn't contentious. all we need to do is pass the request to cjs, like with json

[guybedford]

@addaleax I'm not sure process.dlopen is so bad, since most projects bundle their binaries and are the only consumers of it.

[addaleax]

Also note that process.dlopen() doesn’t have nice effects of require() like ensuring that the returned value is a singleton for each file. When calling it twice on an addon, it will either fail or give you a new instance of the module.

Interesting - could it be made into a singleton do you think?

It would be a breaking change, and I think it depends on whether we ever want to allow loading addons from different contexts/realms.

I don’t think process.dlopen() is terrible, but yeah, it’s also not the most convenient thing to use.

[guybedford]

In addition if you look at usage on npm, most libraries don't do require('./x.node') and rather have a custom wrapper using require('bindings')('./x.node') or similar. These wrappers all effectively would want the process.dlopen-style usage.

[MylesBorins]

I think it is worth considering that wasm modules will hopefully replace the 90% case of native modules. AFAIK many packages that expose native modules expose a CJS interface that in turn brings in the native module itself. I'm going to dig into some of the top native modules and see what interface they expose and post findings in here

[weswigham]

I'm less sure about that - a lot of native module exist to provide bindings for a dynamically linked system lib - even if you can take those libs and compile them to wasm, you'd lose the platform behaviors and dynamic linking (if they even still work without binding to platform APIs). Although it's probably preferable for things like decoders and encoders and other things that were just rewritten as a native module for perf. Upsides and downsides, I suppose.

[MylesBorins]

Can this be closed?

[guybedford]

Feel free to close, my original questions were answered on this.

…

On Wed, 10 Apr 2019 at 09:40, Myles Borins ***@***.***> wrote: Can this be closed? — You are receiving this because you modified the open/close state. Reply to this email directly, view it on GitHub <#279 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAkiypifuOPkkSubXP_02fcTlw6qO1Ucks5vfZVYgaJpZM4bUj0t> .