Proposal to overhaul swingset vat configuration

[FUDCo]

What is the Problem Being Solved?

The current swingset configuration object (the thing that gets passed to the swingset controller's buildVatController function to tell it what to do) is a bit ad hoc:

config = {
  hostStorage: Storage
  verbose: BOOL?
  devices: [
    [
      name: STRING
      srcpath: PATHSTRING
      endowments: OBJ
    ]*
  ]?
  bootstrapIndexJS: PATHSTRING?
  vats: Map<vatName: STRING, vat: {
    sourcepath: PATHSTRING
    options: {
      enablePipelining: BOOL?
    }
  }>
}

The controller module provides a loadBasedir function that will scan a directory for a sterotypical pattern of source files and generate one of the above config structures based on what it finds there. It looks for a bootstrap vat in bootstrap.js and any number of regular vats in files with names of the form vat-foo.js. It does extra work to ignore other files in the directory.

Among the problems with the status quo configuration object are:

• it mixes definitional information (such as source paths) with run-time setup information (like the storage object).

• there's no graceful way to configure different vats that are based on a common source file

• there's no graceful way to deal with vats that are already bundled

• the way it deals with the bootstrap is hacky and non-orthogonal

• since it uses a Map it's limited to being a run-time data structure, rather than something that can be stored in a configuration file and edited, subjected to source control, and so on.

• a similar issue applies to endowments, which are object references that only exist at run time.

• it's lexically and structurally inconsistent: e.g., choice of a map vs. an object vs. an array for a group of things; in one place it uses sourcepath and in another it uses srcpath; etc.

• we'd like a way to statically describe a vat or a set of vats to be deployed by the dynamic vat mechanism, but this is too entangled with the specifics of how the vat controller works for that.

More immediately, we'd like to be able to launch the vats of a swingset definition as either a statically configured set of vats or as a dynamically configured set of metered or unmetered vats, so we can benchmark the performance impact of turning on metering.

Description of the Design

Here is a proposed alternative:

config = {
  vats: {
    VATNAME: {
      sourceSpec: PATHSTRING?
      bundleSpec: PATHSTRING?
      bundleHash: HASHSTRING?
      bundleName: NAMESTRING?
      bundle: CODESTRING?
      parameters: DATAOBJECT?
      endowments: DATAOBJECT?
      creationOptions: {
        enablePipelining: BOOL?
        metered: BOOL?
      }?
    }+
  }
  bootstrap: VATNAME?
  bundles: {
    BUNDLENAME: {
      sourceSpec: PATHSTRING?
      bundleSpec: PATHSTRING?
      bundle: CODESTRING?
    }*
  }?
  devices: {
    DEVICENAME: {
      sourceSpec: PATHSTRING?
      bundleSpec: PATHSTRING?
      bundleHash: HASHSTRING?
      bundleName: NAMESTRING?
      bundle: CODESTRING?
      parameters: DATAOBJECT?
      endowments: DATAOBJECT?
    }*
  }?
}

This configuration object describes what we are calling a "vat group": one or more vats organized together in some purposeful way. The configuration object is a pure data object that can be serialized as JSON, and so stored in a file, loaded from a file, passed in a message over the network, etc., in addition to being something that can be passed around as an object internally.

The vats object describes the vats in the vat group, with one entry per vat keyed to the vat's name (the name is scoped to the vat group). A vat group must contain at least on vat.

The bundles object describes vat bundles independent of which vat they are associated with. In particular, this makes those bundles available for dynamic vat creation without requiring them to be instantiated as vats unless and until they are needed.

The devices object similarly describes devices that will be part of the swingset. Currently these are passed as initialization parameters to the bootstrap vat, though in theory they could be configured as endowments to other vats if we felt so inclined (I presume that at the moment we do not).

The bootstrap property, if present, names the vat that is to be the bootstrap vat. If this property is missing, there will be no bootstrap vat, which may be disallowed in certain use cases.

The code that realizes a vat or device can be specified in one of five ways: sourceSpec, the path to the module or file containing the root object definition; bundleSpec, the path to a file or module containing a pre-generated bundle such as is currently generated by the vat controller from paths of the first kind; bundleHash, a hashstring that can be used as a retrieval key to fetch a bundle from a content-addressed repository of some kind (without, at this point, saying anything about the nature, location, or API of such a repository); bundleName, a simple name string that can be used to designate a bundle by referring to the a vat, device, or bundle described by some other vats, devices, or bundles entry; or bundle, a prebundled source string. These five alternatives are mutually exclusive, but one of them must be provided. (Note also that the final version of this design my well get rid of some of these options.)

parameters is a data object (defaults to the empty object if omitted) that is passed to the vat or device as one of its creation parameters. I'm proposing to include this as an additional parameter alongside the current vatPowers object. Note that this is in contrast to the one existing options property in the legacy configuration object, which is a directive to the vat controller itself. When launching a statically defined vat group via a command line tool such as swingset-runner or ag-solo, an array of strings containing the command line parameters is added to the bootstrap vat's parameters object as the property argv.

endowments is a data object that is interpreted by the vat creation mechanism to describe the endowments of the vat or device (yes, more description is needed here), which are passed to the vat in the vatPowers parameters to its buildRootObject method.

creationOptions takes the place of the options property in the legacy configuration object. It is a directive to the vat controller rather than to the vat itself:

• enablePipelining specifies whether the vat should be launched with promise pipelining enabled (defaults to false if omitted).

• metered specifies whether the vat should be have metering turned on or not (defaults to false if omitted).

In the new configuration scheme described here, the vat controller object's buildVatController method retains its role in the startup flow, but its behavior is slightly changed due to the change in the way things are configured. The differences are:

• The argv parameter is replaced with a runtimeOptions object, which captures the genesis options, the verbose flag, and the storage object (the argv parameter per se goes away here, having been replaced by an parameter passed to the buildRootObject function as described above).

• Command line parameters are given to the bootstrap vat in the parameters parameter to buildRootObject rather than being passed as an argument of the bootstrap message. This means that the bootstrap vat creation can be parameterized by command line args at vat creation time.

• Devices are created in the kernel by the device manager according to the configured parameters, rather than being created externally and passed in. This is probably the detail that requires the most additional design consideration, though from my inspection of the existing code I think the change will actually be modest.

Currently, the dynamic vat creation mechanism creates a single vat at a time, taking the source bundle (contents) and the metered flag as parameters. I propose changing it (or, alternatively, adding a new method) to create a vat group (and return a collection of vatIDs rather than a single one), taking the same configuration object as described here, with the only difference being that it is not required to have a bootstrap vat (though if there is, it will be sent a bootstrap message just as it would be if created by buildVatController).

Security Considerations

• We need to make sure that the device configuration scheme doesn't open up a hole for any old vat creator to wire up arbitrary devices of their own specification.

• Additional degrees of freedom for configuration always deserve to be looked at to ensure they don't introduce new capabilities.

Test Plan

[FUDCo]

@warner @michaelfig

[michaelfig]

Well thought-out. I look forward to seeing the first steps toward a clearer vat/device configuration.

[warner]

@FUDCo can we close this now? Could we copy this specification into swingset/docs/ somehow?

[FUDCo]

Yes. Putting this in docs makes sense. It's a better place to find it anyway. Let's make doing that the gating criterion for closing out this issue.

[FUDCo]

Closed by #1573