internal/metamorphic: TestMeta failed [spurious key]

[jbowens]

Encountered on a PR's CI run but it reproduces on current master SHA (01dcf57): https://github.com/cockroachdb/pebble/actions/runs/11040241548/job/30667767078?pr=3962

2024-09-25T20:08:06.3357754Z ===== SEED =====
2024-09-25T20:08:06.3358259Z 1727294659951576337
2024-09-25T20:08:06.3358854Z ===== DIFF =====
2024-09-25T20:08:06.3359960Z _meta/240925-200419.9512717887793/{standard-000,random-000}
2024-09-25T20:08:06.3360860Z @@ -3306,11 +3306,11 @@
2024-09-25T20:08:06.3361644Z  iter33.SeekGE("cpsifwvfka@3", "") // [false] <nil> #3305
2024-09-25T20:08:06.3362692Z  iter30.First() // [true,"cqqkguannt@6","",<no range>] <nil> #3306
2024-09-25T20:08:06.3364186Z  iter31.Last() // [true,"tvtwgzgsf",<no point>,["tvtwgzgsf","uxnufhrjlnpd")=>{"@6"="zbucyczbfyiknnhnzpwg"}] <nil> #3307
2024-09-25T20:08:06.3365662Z  db1.Set("ysaqsz@33", "vdxzpzjejpqq") // <nil> #3308
2024-09-25T20:08:06.3367390Z  iter27.SeekPrefixGE("ysaqsz@22") // [false] pebble: SeekPrefixGE supplied with key outside of upper bound #3309
2024-09-25T20:08:06.3368948Z -snap25.Get("mnwk@32") // [""] pebble: not found #3310
2024-09-25T20:08:06.3369909Z +snap25.Get("mnwk@32") // ["mnbtyjhmaaseexaqx"] <nil> #3310
2024-09-25T20:08:06.3374791Z  iter29.First() // [true,"cpsifwvfka@10",<no point>,["cpsifwvfka@10","dhvyv")=>{"@30"=""}*] <nil> #3311
2024-09-25T20:08:06.3376295Z  iter30.SeekPrefixGE("cqqkguannt@6") // [true,"cqqkguannt@6","",<no range>] <nil> #3312
2024-09-25T20:08:06.3377944Z  iter27.SetBounds("uxnufhrjlnpd@19", "ymogpw@8") // <nil> #3313
2024-09-25T20:08:06.3378741Z  iter27.SeekGE("uxnufhrjlnpd@19", "") // [true,"uxnufhrjlnpd@19","adhjmaaguwxjmijya",<no range>] <nil> #3314
2024-09-25T20:08:06.3379545Z  iter27.Prev("") // [false] <nil> #3315

Jira issue: PEBBLE-263

[RaduBerinde]

The link is broken.. I'll see if I can repro locally.

[jbowens]

I reproduced it on d73ab80 (#3897). Trying on b34a393 now.

[jbowens]

No dice on b34a393 (cc @itsbilal)

[RaduBerinde]

Same here.

[itsbilal]

Does it readily reproduce, or require a good amount of stressing? Trying a stress run here

[RaduBerinde]

Around 4 minutes or so with go test -tags invariants -exec "stress -p 3 -timeout 30m" -timeout 0 -test.v ./internal/metamorphic

[jbowens]

Reproduces very readily with

go test -run TestMeta$ -seed 1727294659951576337 -tags invariants -exec 'stress -p 1'