asim: allocation simulator

[kvoli]

This is a tracking issue for the allocation simulator.

Why do we need a simulator?

There is an agreement that unit tests and roachtests are useful but not enough to verify allocation changes, and that a fast (not using real time) and repeatable simulation can be useful to reproduce rare bugs in the allocator. In addition to verifying correctness, optimizing the allocation and rebalancing algorithm is an area that is costly to verify and necessitates slow feedback loops (see QPS Accounting - Experiments internal); this may be addressed by a deterministic simulator.

design doc internal

Tracking

• set up skeleton kv: introducing a simulator for the allocator #78272
• random load generator asim: random load generator #79773
• static config loader asim: load cluster configuration to be simulated #79961
• apply workload to state asim: apply workload to simulator state #80127
  • load adapter asim: replica load adapter #81004
• simulate store scanner asim: add replica pacer #81770
• simulate gossip asim: add state exchange for stores #81710
• simulate state changes asim,kvserver: simulate rq rebalancing #82089
• simulate replicate queue consider rebalance asim,kvserver: simulate rq rebalancing #82089
• generate a score, cost and statistics of a simulation run asim: generate a score, cost and statistics for a simulation run #82628
• simulate load based splitting asim: support load based range splitting #82630
• simulate size based splitting asim: simulate size based range splitting #82660
• simulate store rebalancer asim: simulate store rebalancer #82631
• validate the simulator against the a crdb cluster with equivalent configuration
  • validate range count rebalancing asim: validate range count rebalancing #83989
  • validate rebalancing asim: validate unified rebalancing #83990

Jira issue: CRDB-16618