Compartment maps and host modules

[kriskowal]

Every bundle contains a graph of interlinked modules. The bundler collects the transitive dependencies of an “entry” module. However, just as we enter the bundle’s self-contained module graph, certain edges of the import graph may also exit the bundle. These “exit modules” do not get captured by the bundle, so the host must provide them when they import the bundle. To that end, “exit modules” are by definition “host built-in modules”.

To date, there exist no host modules in Node.js that are safe to share with a confined program. Most host modules expose powerful I/O. None of them are hardened.

The Endo bundlers do not allow a program to import host modules. Endo’s bundlers interpret all absolute module specifiers as linkage to other packages, and the surrounding package.json must express all these dependencies. The bundlers treat any other absolute module specifier as an import error. Endo’s bundlers could instead treat these as implicit dependencies on the host environment and defer the failure to link to the time of importBundle if the host did not expressly permit access to the host module.

Node.js recently introduced the form import fs from "node:fs" instead of merely import fs from "fs" to mitigate a namespace collision between host modules and external modules. CommonJS specified only two kinds of module specifier: “absolute” and “relative”, but “absolute” gets used in practice both for dependency and host modules.

So, Endo bundlers could alternately adopt a safer compromise: bundleSource could fail if it cannot find a package that exports the absolute module specifier, and implicitly treat all colon-tagged module specifiers, like node:fs or endo:far, as host module specifiers.

We could conceivably build a platform of Endo host modules that are powerless and hardened. These could mitigate multiple concerns of eval twins including instance bloat, identity discontinuity, and allow shared caches (e.g., the pass-style memo, the harden memo, and perhaps someday, the three-party-handoff connection hints WeakMap). We currently solve this problem by exposing hardened powers on the global environment.

So, for example, @endo/pass-style, @endo/far, or @endo/eventual-send might be elevated to builtin host modules like endo:pass-style, endo:far, or endo:eventual-send`, such that they can be safely shared with multiple guest programs and excluded from bundles.

Every program assumes that its host has prepared a suitable environment with certain globals and modules. These assumptions can be expressed in a package.json in the "engines" section, like {"engines": {"node": "^20 || ^21"}}. This would indicate that the program can expect to import fs from "node:fs", for example. We should capture this information in compartment-map.json so that importBundle can fail early if it can’t satisfy an engine constraint with the current version of the Endo environment.

importBundle will need to be able to selectively grant access to host modules. It will also need to import those lazily. @naugtur has already done some work to defer access to host modules using dynamic import and automatically wrapping the resulting namespace objects. We may also want to confine Endo host modules in another compartment.

We can also rely on the Node.js "imports" and "exports" directives in package.json to allow us to package these modules like @endo/far such that dependencies can reach for them using import "@endo/far" and benefit from TypeScript analysis of the real sources, but redirect bundlers to the host builtins. Consider this package.json for @endo/far that would preserve debugging information during development, runtime behavior in bare Node.js, but allow "endo" to omit the sources from a bundle and rely on importBundle to provide the host power.

{
  "name": "@endo/far",
  "exports": {
    "endo": {
      ".": "endo:far"
    },
    "default": {
      ".": "./index.js"
    }
  }
}