asim: enable event series generation

[wenyihu6]

asim: enable event series generation

This patch enables interesting event series to be generated and scheduled as
part of the simulation.

The following command is now supported:
"rand_events" [type={cycle_via_hardcoded_survival_goals}]
[duration_to_assert_on_event=<time.Duration>]
e.g. rand_events type=cycle_via_hardcoded_survival_goals duration=5m

• rand_events: generates interesting event series to be scheduled in the
  simulation.
• type: type of event series to be scheduled.
• duration_to_assert_on_event: delay to add the assertion events post
  mutation events for mutation-assertion event series.

---

asim: better output for executed events

Previously, the randomized testing framework only reports number of scheduled
mutation and assertion events, offering no insights on their execution or the
results of the assertion events. This patch adds more output info to include
executed mutation events, conducted assertion checks, assertion results, and
their execution timing.

New verbosity flags for eval is now supported to print these info.

"eval" [verbose=events]

Part of: #106192
Release Note: none

[cockroach-teamcity]

This change is 

[wenyihu6]

Please only review the last two commits. The preceding commits are from #109316.

[kvoli]

Looks good, lets discuss how we want to handle the CI timeout offline.

Reviewed 15 of 25 files at r1, 9 of 9 files at r2, 13 of 13 files at r3, all commit messages.
Reviewable status: complete! 0 of 0 LGTMs obtained (waiting on @wenyihu6)

---

-- commits line 49 at r2:
nit: the wording of this line seems off.

---

pkg/kv/kvserver/asim/event/mutation_event.go line 64 at r3 (raw file):

func (se SetSpanConfigEvent) String() string {
	return fmt.Sprintf("set span config event with span=%s, config=%s", se.Span.String(), se.Config.String())

Is the .String() call necessary on these? My understanding was that these were inferred by the format function if they exist.

---

pkg/kv/kvserver/asim/event/mutation_event.go line 99 at r3 (raw file):

func (sne SetNodeLivenessEvent) String() string {
	return fmt.Sprintf("set node liveness event with nodeID=%d, liveness_status=%s", sne.NodeId, sne.LivenessStatus.String())

Same as above.

---

pkg/kv/kvserver/asim/event/mutation_event.go line 110 at r3 (raw file):

func (sce SetCapacityOverrideEvent) String() string {
	return fmt.Sprintf("set capacity override event with storeID=%d, capacity_override=%s", sce.StoreID, sce.CapacityOverride.String())

Same as above.

[wenyihu6]

pkg/kv/kvserver/asim/event/mutation_event.go line 64 at r3 (raw file):

Previously, kvoli (Austen) wrote…

Is the .String() call necessary on these? My understanding was that these were inferred by the format function if they exist.

opps forgot about it

[wenyihu6]

pkg/kv/kvserver/asim/event/mutation_event.go line 99 at r3 (raw file):

Previously, kvoli (Austen) wrote…

Same as above.

Done.

[wenyihu6]

pkg/kv/kvserver/asim/event/mutation_event.go line 110 at r3 (raw file):

Previously, kvoli (Austen) wrote…

Same as above.

Done.

[kvoli]

Discussed offline, we will resolve the timing out tests before merging.

Reviewed 29 of 29 files at r4, 12 of 28 files at r5, 1 of 1 files at r6, all commit messages.
Reviewable status: complete! 1 of 0 LGTMs obtained (waiting on @wenyihu6)

[wenyihu6]

pkg/kv/kvserver/asim/tests/testdata/rand/events line 4 at r7 (raw file):

----

change_static_option ranges=1

I forgot to shorten the duration when changing the number of ranges generated to 1. After making this change, the time it took to stabilize decreased significantly. The test only takes 2.5 seconds now. I have another PR to remove some of the log events that were time-consuming. But this is no longer blocking this PR. Happy to close this other PR if we prefer to keep the log events for tracing.

[wenyihu6]

pkg/kv/kvserver/asim/tests/testdata/rand/events line 4 at r7 (raw file):

Previously, wenyihu6 (Wenyi Hu) wrote…

I forgot to shorten the duration when changing the number of ranges generated to 1. After making this change, the time it took to stabilize decreased significantly. The test only takes 2.5 seconds now. I have another PR to remove some of the log events that were time-consuming. But this is no longer blocking this PR. Happy to close this other PR if we prefer to keep the log events for tracing.

link to this other PR - #109804

[wenyihu6]

TFTR!

bors r=kvoli

[craig]

Build succeeded:

• Bazel Essential CI (Cockroach)