sufficiently-deterministic GC

[warner]

What is the Problem Being Solved?

#1872 was about how swingset should approach GC given the lack of engine-level determinism. In #1872 (comment) we figured that we can get enough deterministic behavior out of the engine we're using to use a somewhat simpler design. This ticket captures that subset and replaces #1872.

The checklist of subtasks is:

• add a no-op syscall.dropImports to the vat-to-kernel pathway, so liveslots can charge ahead without waiting for the kernel add no-op syscall.dropImports to vat-kernel protocol #2635
• add a no-op dispatch.dropExports to the kernel-to-vat pathway, so the kernel can charge ahead without waiting for liveslots. This needs to be added to the comms vat too. add a no-op dispatch.dropExports to the kernel-to-vat pathway #2653
• give liveslots a way to sense vat quiescence so it can drive GC at the right time change liveslots to use WeakRefs to track objects and report their disuse #2660
• change vat managers and liveslots to drive GC at the right time change liveslots to use WeakRefs to track objects and report their disuse #2660
• change liveslots to use WeakRefs to track exports change liveslots to use WeakRefs to track objects and report their disuse #2660
• change liveslots to honor dispatch.dropExport make liveSlots honor dropExports #2664
• have liveslots emit syscall.dropImports at the right time, based on finalizers running change liveslots to use WeakRefs to track objects and report their disuse #2660
  • stop recording syscall.dropImports in the transcript, or expecting a replay to match those calls
• flesh out the kernel-side refcounting tables
• fully implement kernel-side reference counting, maybe deferring cycle collection for now, invoking dispatch.dropExports as soon as the refcount drops to zero GC: implement full kernel-side refcount management #2646
• implement both sides of GC in the comms vat and the comms protocol, using the ack/lastMentioned mechanism described in Cope with asynchrony in remote retirement of promise IDs #2509 (comment)

Description of the Design

On the export side, liveslots will change slotToVal to hold WeakRef values instead of strong references to Remotables. A separate Set named exports will hold strong references until a dispatch.dropExports relieves the vat of that duty.

On the import side, the slotToVal weakref allows liveslots to sense when an import (Presence) becomes unreferenced. Liveslots will use a FinalizationRegistry to detect this. Acting upon the callbacks (calling syscall.dropImports) is deferred to the end of the crank, which liveslots needs to be able to sense internally. Liveslots is also responsible for calling a platform-provided "force GC" function, and giving it time to fire the callbacks. A flag should tell liveslots to enable all this behavior.

We'll have one mode we use on-chain, which requires determinism. It will run vats under XS, trigger a single worker's GC at the end of each crank, and call dropImport right away.

There will be a different mode for solo machines, which need hangover-consistency but not determinism. This can run vats under Node.js, and doesn't need to trigger GC deliberately. If/when finalizers fire, vats will notice it at the end of crank and invoke syscall.dropImports at the right time (we must still make sure dropImports only happens at the end of crank, and only for the vat being invoked). We might need some extra mechanism to make sure infrequently-called vats have a chance to shed their imports.

[warner]

This was sufficiently implemented in #3106 . We still have remaining tasks to handle virtual objects and virtualized data completely, but normal Remotables and Presences ought to be collected now, both in the kernel and over comms.