Cache for expensive-to-instantiate values from the KV

[eddyashton]

The idea is that we currently have values that we need to store in the KV, and access transactionally (ie - correct-at-this-read-version), but which are very expensive to deserialise from bytes in the KV to their useable in-memory form. An example of this is certificates. We often want to store these certificates as JSON in the KV (for easy audit), but we then need to do expensive conversions from bytes -> JSON -> raw string -> crypto verifier, every time we actually use the certificate. Instead we'd like to (especially for certificates which rarely change!) just keep a crypto::VerifierPtr in-memory, and still trust that its correct within a transaction (ie - is using the cert at key K at our current read version). We have a custom system for caching the Verifiers used to verify HTTP signatures, but I think we can build a more generic one.

Basically we should be able to extend the custom caching we do for HTTP signature verifiers to a more generic caching mechanism, "if you've already deserialised and instantiated K from M at version v, give me that instance again, or build a fresh one if you need to". This may be useful to simplify the LedgerSecrets, which simulates this behaviour by explicitly taking a read-dependency it doesn't use (though in that case the cache is also used non-transactionally in other places).

As a code sketch, we currently do:

auto widgets_view = tx.ro(my_widgets_map);

// Widget has a neat JSON object serialisation, but this call both reads that from bytes
// AND construct an in-memory Widget, and the latter is expensive! It allocates 1TB
// and computes a few billion primes and recites the alphabet backwards. Even if my app
// only wrote this value once, in the very first call, I then need to reconstruct this to use the
// definition in every call from then on.
const Widget w = widgets_view->get("my_widget");

Instead we should be able to do:

class WidgetCache
{
  // NB: WidgetPtr points to a const Widget - you can't modify it!
  WidgetPtr get(ReadOnlyTx& tx, const K& k)
  {
    auto widgets_view = tx.ro(my_widgets_map);
    const auto last_written = widgets_view->get_version_of_previous_write(k);
    auto cached = cache.find(k);
    if (cached != cache.end())
    {
      auto [version, widget] = *cached;
      if (version == last_written)
      {
        return widget;
      }
    }

   // Widget at k has changed or was never in cache. Build from current value, place into cache, then return
  }
};
...
WidgetPtr w = widget_cache.get(tx, "my_widget");

The key is that this still executes transactionally; we still have a dependency on k so we are re-executed if it changed while we were executing. This caching is completely node-local, it can be further backed by an LRU or culled by the app, it just allows us to skip the repeated bytes-to-instantiation conversion.

We should at least build a sample of this to show how to do it safely, and explore building a generic solution for ourselves and app developers. As with #274 it could be built entirely in app-space, but if its useful for our tables we can build it anyway and decide if its something we support in the public API.