Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

allow bundlecaps in vatParameters #4381

Closed
warner opened this issue Jan 25, 2022 · 3 comments · Fixed by #4872
Closed

allow bundlecaps in vatParameters #4381

warner opened this issue Jan 25, 2022 · 3 comments · Fixed by #4872
Assignees
Labels
enhancement New feature or request SwingSet package: SwingSet
Milestone

Comments

@warner
Copy link
Member

warner commented Jan 25, 2022

What is the Problem Being Solved?

To enable contract upgrade, specifically to allow the version-2 vat code to perform #3062 Kind-behavior replacements during the very first crank of the new version's worker lifetime (before any inbound deliveries might reference those virtual objects), we need to change ZCF to:

  • realize when it's running in version >= 2, rather than version 1
  • read vatParameters to learn the new contract bundlecap that is to be loaded
  • use the bundlecap to retrieve the actual bundle
  • importBundle() to load the contract code
  • inform the contract code that an upgrade is in progress, and let it complete
    • the contract code can then reattach the virtual-object Kinds created by its predecessor

Currently, vatParameters is stored as a string in the kvStore, delivered to vat workers as they are started, and unpacked with a simple JSON.parse by liveslots.

To make this work, we need vatParameters to be able to hold caps, so the vat-upgrade code can pass a bundlecap for the new contract version to the upgraded ZCF vat. It isn't sufficient to use a bundleID, because one needs access to devices.bundle to synchronously convert that into a real bundle, and then we'd need to store devices.bundle in vatParameters. The need to fetch the bundle in the initial startup crank precludes a scheme which sends a message to vatAdmin to retrieve the bundle, since the response (dispatch.notify) could not be delivered until some future crank, which is too late.

Description of the Design

Here's what I've sketched out so far:

  • the kvStore holds vNN.vatParameters as JSON.stringify(capdata), using krefs instead of vrefs
  • vatKeeper.setVatParameters(capdata) is responsible for updating refcounts
    • so if version-1 references bundlecapA and version-2 references bundlecapB instead, setVatParameters(ver2capdata) will increfB and decref A (except that bundlecaps, like all device nodes, are currently immortal, and don't have refcounts; but all other objects should be tracked properly)
  • the vat upgrade process uses vatKeeper.setVatParameters before launching the new vat

Past that, things get murky. As part of #2910 we're looking at changing the early steps of a vat worker into more messages that can be delivered (at the very least, buildRootObject must be called as part of a delivery, so there is a transcript writer waiting to record the syscalls it makes, and more importantly the responses that come back). If we go with dispatch.buildRootObject, then we can include vatParameters in the KernelDeliveryObject/VatDeliveryObject (using the vat translators to convert the kvStore's krefs into vrefs, populating the clist at the same time, remembering to make sure we get GC right). The transcript for this dispatch.buildRootObject delivery will include the D(bundlecap).getBundle() and it's gigantic response, which is ok for now but eventually we want to get that response out of the transcript, if/when we get tighter integration between liveslots, bundlecaps, and the Endo module loader.

Security Considerations

Not a lot; vatParameters is a communication pathway between the caller who asks for the vat to be created, and the root object of the new vat, and this just enables that pathway to include arbitrary objects.

In our #1848 vat/contract upgrade plan, vatParameters can be different for each version of a given vat, so it's provided by the vat-upgrading caller, in addition to the vat-creating caller. But vatParameters cannot be read back out from a vat, so it doesn't provide a direct communication pathway between creator and upgrader (unless facilitated by the child vat itself).

Test Plan

unit tests

@warner warner added enhancement New feature or request SwingSet package: SwingSet MN-1 labels Jan 25, 2022
@warner warner self-assigned this Jan 25, 2022
@Tartuffo Tartuffo removed the MN-1 label Feb 7, 2022
@warner
Copy link
Member Author

warner commented Feb 10, 2022

This depends on #2910 because that will turn vat-startup into a distinct delivery, going through a VatTranslator, making it a great place to deliver vatParameters that contain vrefs.

The create-vat run-queue event should change to move vatParameters out of options and into a separate place that is known to hold a kref-based capdata object. This capdata needs to be treated like a run-queue deliver event for GC refcounting purposes. The start-vat event should have a similar place for the vatParameters that are delivered to static vats (although there's no likely way for them to get slots, since their vatParameters only come from the config object, which is defined before any vats even exist).

@warner
Copy link
Member Author

warner commented Mar 3, 2022

#4726 is a sub-task of this one, so I'm moving 2 of it's 3 story points to #4726

@warner
Copy link
Member Author

warner commented Mar 4, 2022

#4588 will probably get closed as a side-effect of this one, when I change vatAdmin to use the new "device hooks" feature and we get to translate drefs to krefs.

warner added a commit that referenced this issue Mar 9, 2022
For consistency, `kernel.createTestVat()` now sends a `dispatch.startVat` to
these `setup`-based test vats, just like all normal vats.

Some tests were modified to ignore the extra delivery they now observe.

refs #4381
warner added a commit that referenced this issue Mar 9, 2022
Previously, `vatParameters` arrived at a vat worker in the `setBundle`
message to the supervisor (the same one that provided the vat's source
bundle). This changes the code to deliver them inside the
`dispatch.startVat()` invocation instead.

Liveslots-based vats see no change: both cases hand vatParameters in via the
`buildRootObject()` call. `setup()`-based vats must change: the `setup()`
call no longer contains a `vatParameters` argument, and vatParameters are
available temporally later than before.

As a result, the comms vat must wait for `startVat()` to learn its two
configuration settings: `identifierBase` and `sendExplicitSeqNums`. Comms now
uses both to initialize its state DB, and reads `sendExplicitSeqNums` from
the DB for each `transit()` call.

This paves the way for `vatParameters` to contain slots, not just inert data.
The VatTranslator that converts `startVat` KernelDeliveryObjects into
VatDeliveryObjects does not yet perform slot translation, nor does liveslots
treat `vatParameters` as capdata, but they are now in the right place to
perform those tasks.

refs #4381
closes #4766
warner added a commit that referenced this issue Mar 12, 2022
For consistency, `kernel.createTestVat()` now sends a `dispatch.startVat` to
these `setup`-based test vats, just like all normal vats.

Some tests were modified to ignore the extra delivery they now observe.

refs #4381
warner added a commit that referenced this issue Mar 12, 2022
Previously, `vatParameters` arrived at a vat worker in the `setBundle`
message to the supervisor (the same one that provided the vat's source
bundle). This changes the code to deliver them inside the
`dispatch.startVat()` invocation instead.

Liveslots-based vats see no change: both cases hand vatParameters in via the
`buildRootObject()` call. `setup()`-based vats must change: the `setup()`
call no longer contains a `vatParameters` argument, and vatParameters are
available temporally later than before.

As a result, the comms vat must wait for `startVat()` to learn its two
configuration settings: `identifierBase` and `sendExplicitSeqNums`. Comms now
uses both to initialize its state DB, and reads `sendExplicitSeqNums` from
the DB for each `transit()` call.

This paves the way for `vatParameters` to contain slots, not just inert data.
The VatTranslator that converts `startVat` KernelDeliveryObjects into
VatDeliveryObjects does not yet perform slot translation, nor does liveslots
treat `vatParameters` as capdata, but they are now in the right place to
perform those tasks.

refs #4381
closes #4766
warner added a commit that referenced this issue Mar 14, 2022
The `startVat()` message accepts "vat parameters", but previously these were
arbitrary (inert) data. This changes the argument to contain capdata.
Liveslots will `unserialize` this data into the `vatParameters` argument
given to `buildRootObject()`.

The kernel was updated to populate this field with capdata in all pathways
that create create/start/upgrade-vat events. Parameters passed in through
config (which must be inert data) are serialized into capdata.

Parameters passed through `E(vatAdminService).createVat(bundle, {
vatParameters })` are also treated as inert data and serialized into capdata,
however when we move to "device hooks", this will change, and dynamic vat
creators will be able to pass object references in those parameters.

refs #4381
warner added a commit that referenced this issue Mar 14, 2022
The `startVat()` message accepts "vat parameters", but previously these were
arbitrary (inert) data. This changes the argument to contain capdata.
Liveslots will `unserialize` this data into the `vatParameters` argument
given to `buildRootObject()`.

The kernel was updated to populate this field with capdata in all pathways
that create create/start/upgrade-vat events. Parameters passed in through
config (which must be inert data) are serialized into capdata.

Parameters passed through `E(vatAdminService).createVat(bundle, {
vatParameters })` are also treated as inert data and serialized into capdata,
however when we move to "device hooks", this will change, and dynamic vat
creators will be able to pass object references in those parameters.

refs #4381
warner added a commit that referenced this issue Mar 18, 2022
The `startVat()` message accepts "vat parameters", but previously these were
arbitrary (inert) data. This changes the argument to contain capdata.
Liveslots will `unserialize` this data into the `vatParameters` argument
given to `buildRootObject()`.

The kernel was updated to populate this field with capdata in all pathways
that create create/start/upgrade-vat events. Parameters passed in through
config (which must be inert data) are serialized into capdata.

Parameters passed through `E(vatAdminService).createVat(bundle, {
vatParameters })` are also treated as inert data and serialized into capdata,
however when we move to "device hooks", this will change, and dynamic vat
creators will be able to pass object references in those parameters.

refs #4381
warner added a commit that referenced this issue Mar 18, 2022
The `startVat()` message accepts "vat parameters", but previously these were
arbitrary (inert) data. This changes the argument to contain capdata.
Liveslots will `unserialize` this data into the `vatParameters` argument
given to `buildRootObject()`.

The kernel was updated to populate this field with capdata in all pathways
that create create/start/upgrade-vat events. Parameters passed in through
config (which must be inert data) are serialized into capdata.

Parameters passed through `E(vatAdminService).createVat(bundle, {
vatParameters })` are also treated as inert data and serialized into capdata,
however when we move to "device hooks", this will change, and dynamic vat
creators will be able to pass object references in those parameters.

refs #4381
warner added a commit that referenced this issue Mar 18, 2022
The `startVat()` message accepts "vat parameters", but previously these were
arbitrary (inert) data. This changes the argument to contain capdata.
Liveslots will `unserialize` this data into the `vatParameters` argument
given to `buildRootObject()`.

The kernel was updated to populate this field with capdata in all pathways
that create create/start/upgrade-vat events. Parameters passed in through
config (which must be inert data) are serialized into capdata.

Parameters passed through `E(vatAdminService).createVat(bundle, {
vatParameters })` are also treated as inert data and serialized into capdata,
however when we move to "device hooks", this will change, and dynamic vat
creators will be able to pass object references in those parameters.

refs #4381
warner added a commit that referenced this issue Mar 19, 2022
This allows `E(vatAdminService).createVat(bundleID, { vatParameters })` to
include object references in `vatParameters`, which are then delivered
through the `dispatch.startVat()` delivery and made available to
`buildRootObject(vatPowers, vatParameters)`.

From within a vat, `vatPowers.exitVat(completion)` and
`.exitVatWithFailure(reason)` can include object references, and they will be
delivered to the parent caller's `done` promise.

From outside the vat, `E(adminNode).terminateWithFailure(reason)` can take
object refs in `reason` and they will be delivered to the parent caller's
`done` promise. `E(adminNode).upgrade(bundleID, vatParameters)` can take
object refs in `vatParameters` just like `createVat` (although `upgrade`
itself is still non-functional).

The kernel will maintain a refcount on each object passed through this
mechanism, to keep it from being collected while in transit.

This is implemented with the new "kernel device hooks" feature, which allows
a device to call into the kernel and have its drefs translated into krefs.

This will help with ZCF/contract vat upgrade, to pass the new contract
bundlecap into the new ZCF vat via vatParameters.

closes #4588
closes #4381
refs #1848
warner added a commit that referenced this issue Mar 19, 2022
This allows `E(vatAdminService).createVat(bundleID, { vatParameters })` to
include object references in `vatParameters`, which are then delivered
through the `dispatch.startVat()` delivery and made available to
`buildRootObject(vatPowers, vatParameters)`.

In vat-vat-admin, the CreateVatOptions validation was rewritten to
ensure that any existing slots are only in vatParameters.

From within a vat, `vatPowers.exitVat(completion)` and
`.exitVatWithFailure(reason)` can include object references, and they will be
delivered to the parent caller's `done` promise.

From outside the vat, `E(adminNode).terminateWithFailure(reason)` can take
object refs in `reason` and they will be delivered to the parent caller's
`done` promise. `E(adminNode).upgrade(bundleID, vatParameters)` can take
object refs in `vatParameters` just like `createVat` (although `upgrade`
itself is still non-functional).

The kernel will maintain a refcount on each object passed through this
mechanism, to keep it from being collected while in transit.

This is implemented with the new "kernel device hooks" feature, which allows
a device to call into the kernel and have its drefs translated into
krefs.

This will help with ZCF/contract vat upgrade, to pass the new contract
bundlecap into the new ZCF vat via vatParameters.

closes #4588
closes #4381
refs #1848
warner added a commit that referenced this issue Mar 22, 2022
This allows `E(vatAdminService).createVat(bundleID, { vatParameters })` to
include object references in `vatParameters`, which are then delivered
through the `dispatch.startVat()` delivery and made available to
`buildRootObject(vatPowers, vatParameters)`.

In vat-vat-admin, the CreateVatOptions validation was rewritten to
ensure that any existing slots are only in vatParameters.

From within a vat, `vatPowers.exitVat(completion)` and
`.exitVatWithFailure(reason)` can include object references, and they will be
delivered to the parent caller's `done` promise.

From outside the vat, `E(adminNode).terminateWithFailure(reason)` can take
object refs in `reason` and they will be delivered to the parent caller's
`done` promise. `E(adminNode).upgrade(bundleID, vatParameters)` can take
object refs in `vatParameters` just like `createVat` (although `upgrade`
itself is still non-functional).

The kernel will maintain a refcount on each object passed through this
mechanism, to keep it from being collected while in transit.

This is implemented with the new "kernel device hooks" feature, which allows
a device to call into the kernel and have its drefs translated into
krefs.

This will help with ZCF/contract vat upgrade, to pass the new contract
bundlecap into the new ZCF vat via vatParameters.

closes #4588
closes #4381
refs #1848
@mergify mergify bot closed this as completed in #4872 Mar 22, 2022
@Tartuffo Tartuffo added this to the Mainnet 1 milestone Mar 23, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request SwingSet package: SwingSet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants