Price feed benchmarks

[FUDCo]

Implement benchmarks for price feeds with and without liquidation.

This includes adjustments to the boot tooling to accommodate its use for benchmarking, as well as improvements and enhancements to the bechmarkerator API that were required to make these benchmarks possible.

Closes #8496

[turadg]

I think these comments won't appear in IDE or auto-gen output.

The type of the Benchmark object is better expressed in .ts. We could have a .d.ts for it (with a sibling .js dummy) or just move this file to .ts syntax. Are you okay with the latter? I could do it

[FUDCo]

I've been frustrated (in a way I think we've discussed previously) by the need to, in essence, write the declarations twice: once for the type checker, using one syntax, and once for the human reader, in a different syntax to provide documentation text that the JS type syntax did not permit. Since the benchmarks already have to be executed using tsx anyway, we're already in the TS world here, so I think converting this to TS may be the best course. If you're game for that I'm happy to have you do it, but I'd prefer to do it in a later PR as I've already been burned a couple times by TS conversions happening in the middle of trying to do something else unrelated.

[turadg]

Good call. A full conversion could wait for another PR. I just pushed a commit to solve this specific duplicate issue by extracting the types to a types.ts.

[turadg]

these are known. Please type them. You could have a makeTools() function and make this have type ReturnType<typeof makeTools>

[turadg]

please maintain the t. since it's using the Ava API (even if benchmarkerator doesn't have full Ava).

Suggested change

	like,
	t,

It's type can be t: Pick<ExecutionContext, 'like'>

[FUDCo]

I think I need some additional TS syntax support on this one. tsc complains:
error TS2693: 'Pick' only refers to a type, but is being used as a value here.
It looks to me like a type annotation, but apparently I'm not holding it right.

[turadg]

I couldn't repro that but I did run into another kink, the API has a .skip property: https://github.com/avajs/ava/blob/af5684dff58a79da862fa57fad0e946985f730a2/types/assertions.d.cts#L186-L194

For now I just pushed a commit that suppresses the problem. (Along with some annotations for drivers)

[FUDCo]

Weirdly, tsc is happy if I leave the type of the partial implementation of t undeclared.

[turadg]

it's only silent because there's no type annotation for t in the called function. Your force push removed a0dce27 and 9b5062f

I'm pushing another fix on top of your change.

[warner]

Just to confirm my understanding:

This price-feed benchmark code is taking advantage of unit-test code tooling, by calling into a priceFeedDriver, defined in packages/boot/tools/drivers.ts, which is basically defined as:

    async setPrice(price: number) {
      await Promise.all(
        oracleWallets.map(w =>
          w.executeOfferMaker(
            Offers.fluxAggregator.PushPrice,
            {
              offerId: `push-${price}-${Date.now()}`,
              roundId,
              unitPrice: BigInt(price * 1_000_000),
            },
            adminOfferId,
          ),
        ),
      );
      // prepare for next round
      oracleWallets.push(NonNullish(oracleWallets.shift()));
      roundId += 1n;
      // TODO confirm the new price is written to storage
    },

and w.executeOfferMaker boils down to

      const offerCapData = marshaller.toCapData(
        harden({
          method: 'executeOffer',
          offer,
        }),
      );
      return EV(walletPresence).handleBridgeAction(offerCapData, true);

and EV boils down to something like controller.queueToVatObject and some stuff to let it run.

So the round starts with the marshalling of the same message that vat-bridge would send to vat-walletFactory, and then injecting that message into the kernel.

Relative to the real chain, the benchmark is doing slightly more work (marshalling the wallet message) and slightly less work (not involving device-bridge or vat-bridge). I think that's fine: we can measure how much time we're spending in vat-bridge from our mainnet logs (I don't think it's much), and it would be too much hassle to involve device-bridge. It might be nice to exclude the extra marshalling call, but I don't think it will will significant.

One thing to note for the future.. having the benchmark share code with unit tests is convenient, but we need remain aware of diverging interests. For example, the TODO in setPrice suggests that at some point the unit test will acquire code to look at chain storage and assert that the new price has been written. That's not something we'd want the benchmark doing (it'd be work that the real chain operation does not do, and thus make our benchmark results artifically worse).

I'm hoping we can get to a point where benchmarks are less entwined with the unit test approach. Something wheree packages/boot/ exports some constructor function that builds the wallet spend message (ideally in the same form we drop into vat-bridge), and some other functions/configs that help us build a small chain (complete with zoe, vaults, wallets, etc), and then the benchmark's setup can build the chain, and the benchmark's round() can deliver the spend message to vat-board and wait for controller.run() to finish.

[warner]

Looks good

[warner]

This readLatest/assertion is overhead in the benchmarked code, right? Like, the benchmark results will be slightly worse than the real chain, because the real chain doesn't do these additional assertions?

Probably trivial, but I wanted to make sure I understand what is/is-not in the benchmarked path.

[turadg]

True. Since the vstorage is cached the difference will be small. Perhaps worth verifying.

Regardless, there can be a flag for whether to verify any values. Perhaps whether the final argument "t" is provided. That might be better than mocking it.

[FUDCo]

Yes, though readLatest is reading from the simulated vstorage, which amounts to a Map lookup, so the overhead is modest.