XS Compartment constructor with module loader hooks

[dckc]

@kriskowal my current solution in agoric-sdk is endo-load.js, called from start-xs.js

It consumes the output of findmods.js, which runs under node.js and uses filing-cabinet to walk the tree of module dependencies.

As I think I mentioned, in contrast to xsnap, where all .js scripts and modules are loaded / parsed at runtime, start-xs.js and endo-load.js start in a world where all the StaticModuleRecords are in compiled bytecodes. See Makefile for details.

originally raised as: Agoric/agoric-sdk#57

[dckc]

oh... the endo-types branch might be easier to read, @kriskowal :
https://github.com/dckc/agoric-sdk/blob/endo-types/packages/xs-vat-worker/src/endo-load.js
https://github.com/dckc/agoric-sdk/blob/endo-types/packages/xs-vat-worker/src/endo-types.d.ts

[dckc]

Now that findmods.js is obsolete in favor of xsnap and rollup, let's use this to represent dynamicModuleLoading and such as discussed in Agoric/agoric-sdk#2107 .

[dckc]

@kriskowal here's hoping you can get to this soonish...

[kriskowal]

This might be ready-to-try. It needs compartment-mapper and ses-shim (full stack, including babel) running in XS.

[dckc]

Including babel? I thought the purpose of this was to avoid babel. I wonder where I got confused.

[kriskowal]

Oh, no. I am confused because you put it on my plate. If it’s on my plate, that implies we’d be doing compartments with the ses-shim. If we’re doing “XS Compartment constructor with module loader hooks”, we’re writing C.

[dckc]

yes, I'm kinda muddled in my thinking. Hm.

[erights]

My understanding is that the next step is to use the builtin XS Compartment only as an evaluator that can replace the 8 magic lines of code. We still need to build @kriskowal 's Compartment on top of that in order to handle modules the way we need.

[erights]

When removing the 8 magic lines of code, we need to carefully examine which if any safety checks we're removing. The builtin XS Compartment will still do something not to our liking with dynamic import(...) expression, so we'll still need to prohibit it. We should also continue to prohibit html comments, but this is arguable. We can almost drop the prohibition on direct eval, because the builtin XS Compartment implements direct eval correctly. However, there's no way to do so while still enforcing these very prohibitions.

[kriskowal]

I think this issue largely falls under #400, surfacing native XS Compartment when running SES on XSnap. That in turn also covers work Moddable has already done to support dynamic module loading. All that remains is some hypothetical work to make compartment mapper useful for generating XS native manifests, but I don’t intend to track that since it’s not well-motivated. I’m closing this in favor of ongoing tracking in #400. Please reöpen if this represents some other work.