feat(SwingSet): evictAfterSnapshot option for periodic eviction of all vats

[dckc]

Add a switch in case "turn it off and turn it on again" helps manage vat worker RAM usage.

refs: #5910

Documentation Considerations

$EVICT_AFTER_SNAPSHOT environment variable is documented.

Testing Considerations

unit tests updated

passes a smoke test:

agoric-sdk/packages/cosmic-swingset$ EVICT_AFTER_SNAPSHOT=1 make scenario2-run-chain
...
2022-08-17T02:15:44.347Z SwingSet: vat: v1: agoricNamesAdmin settled; remaining: []
XS snapshot written to /home/connolly/projects/agoric-sdk/packages/cosmic-swingset/t1/n0/data/ag-cosmos-chain-state/xs-snapshots/ba667a6f910f04ec834edfb9f71b480f94b8f2fad3fe40bd0a27a4292a9d6f5d.gz : 2664962 bytes compressed, 14694111 raw
2022-08-17T02:15:45.181Z SwingSet: kernel: periodic eviction v1

[dckc]

@arirubinstein @warner maybe we should land this after all, until we have a better idea?

[warner]

@arirubinstein @warner maybe we should land this after all, until we have a better idea?

Let's look at the slogfile timing and get a sense for how long it takes to evict the worker (and how long it takes to load it back in again later).. that should help us make a decision.

[warner]

Let's add runtimeOptions.warehousePolicy.evictAfterSnapshot, which launch-chain.js will add to the third argument of its makeSwingsetController() call, to enable this eviction only on the chain. Swingset will have it disabled by default. Let's add a copy of test-controller.js 's static vats are unmetered on XS to exercise setting this to true, which should cause it to observe t.deepEqual(c.dump().log, ['buildRootObject called', 'bootstrap called', 'buildRootObject called', 'bootstrap called']). We could probably strip out some other parts of the test but if it's sensitive to the thing that we're testing, maybe we don't bother. It's probably worth a note in the new test that we expect the normal actions of the test (whatever basedir-controller-2 's config is provoking) to result in enough deliveries to trigger exactly one snapshot writing, followed by at least one delivery, so that we start a worker twice.

Although .. hang on, the second worker startup should be happening from the snapshot, so why is it invoking vatPowers.testLog('buildRootObject called') more than once? We might need to investigate that, to make sure the test is doing what we expect.

[dckc]

... hang on, ... so why ...

I don't have a good mental model; I hope you can figure that out, @warner .
@FUDCo maybe you know?

[dckc]

overtaken by #5987