Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

storage/metamorphic: TestPebbleEquivalence failed #85515

Closed
cockroach-teamcity opened this issue Aug 3, 2022 · 2 comments · Fixed by #85555
Closed

storage/metamorphic: TestPebbleEquivalence failed #85515

cockroach-teamcity opened this issue Aug 3, 2022 · 2 comments · Fixed by #85555
Assignees
@jbowens
Labels
branch-master Failures and bugs on the master branch. C-test-failure Broken test (automatically or manually discovered). O-robot Originated from a bot. T-storage Storage Team
Milestone
22.2

Comments

@cockroach-teamcity
Copy link
Member

cockroach-teamcity commented Aug 3, 2022 •
edited by exalate-issue-sync bot
Loading

storage/metamorphic.TestPebbleEquivalence failed with artifacts on master @ 47d34c9dd0c63dfd462e4380a65470f4cf18e3d6:

    --- FAIL: TestPebbleEquivalence/engine/random=3 (0.12s)
=== RUN   TestPebbleEquivalence/engine/random=5
check file = /artifacts/tmp/_tmp/b2a21e13e3bb63e34c8b8052805e4125/TestPebbleEquivalence2826645106/output.meta
output file = /artifacts/tmp/_tmp/b2a21e13e3bb63e34c8b8052805e4125/TestPebbleEquivalence_engine_random=53798614003/output.meta
store path = /artifacts/tmp/_tmp/b2a21e13e3bb63e34c8b8052805e4125/TestPebbleEquivalence_engine_random=53363859730/store
    generator.go:416: mismatching output at line 288, operation index 187: expected valid = false
        , got key = "nytaxpdxyq"/0.000000026,0
    --- FAIL: TestPebbleEquivalence/engine/random=5 (0.11s)
=== RUN   TestPebbleEquivalence/engine/random=9
check file = /artifacts/tmp/_tmp/b2a21e13e3bb63e34c8b8052805e4125/TestPebbleEquivalence2826645106/output.meta
output file = /artifacts/tmp/_tmp/b2a21e13e3bb63e34c8b8052805e4125/TestPebbleEquivalence_engine_random=91722665158/output.meta
store path = /artifacts/tmp/_tmp/b2a21e13e3bb63e34c8b8052805e4125/TestPebbleEquivalence_engine_random=93174139385/store
    generator.go:416: mismatching output at line 288, operation index 187: expected valid = false
        , got key = "nytaxpdxyq"/0.000000026,0
    --- FAIL: TestPebbleEquivalence/engine/random=9 (0.13s)
=== RUN   TestPebbleEquivalence/engine/standard=10
check file = /artifacts/tmp/_tmp/b2a21e13e3bb63e34c8b8052805e4125/TestPebbleEquivalence2826645106/output.meta
output file = /artifacts/tmp/_tmp/b2a21e13e3bb63e34c8b8052805e4125/TestPebbleEquivalence_engine_standard=101507238009/output.meta
store path = /artifacts/tmp/_tmp/b2a21e13e3bb63e34c8b8052805e4125/TestPebbleEquivalence_engine_standard=102922820382/store
    generator.go:416: mismatching output at line 288, operation index 187: expected valid = false
        , got key = "nytaxpdxyq"/0.000000026,0
    --- FAIL: TestPebbleEquivalence/engine/standard=10 (0.11s)
=== RUN   TestPebbleEquivalence/engine/standard=17
check file = /artifacts/tmp/_tmp/b2a21e13e3bb63e34c8b8052805e4125/TestPebbleEquivalence2826645106/output.meta
output file = /artifacts/tmp/_tmp/b2a21e13e3bb63e34c8b8052805e4125/TestPebbleEquivalence_engine_standard=171604700334/output.meta
store path = /artifacts/tmp/_tmp/b2a21e13e3bb63e34c8b8052805e4125/TestPebbleEquivalence_engine_standard=171943483487/store
    generator.go:416: mismatching output at line 288, operation index 187: expected valid = false
        , got key = "nytaxpdxyq"/0.000000026,0
    --- FAIL: TestPebbleEquivalence/engine/standard=17 (0.11s)
=== RUN   TestPebbleEquivalence/engine/random=0
check file = /artifacts/tmp/_tmp/b2a21e13e3bb63e34c8b8052805e4125/TestPebbleEquivalence2826645106/output.meta
output file = /artifacts/tmp/_tmp/b2a21e13e3bb63e34c8b8052805e4125/TestPebbleEquivalence_engine_random=01869950469/output.meta
store path = /artifacts/tmp/_tmp/b2a21e13e3bb63e34c8b8052805e4125/TestPebbleEquivalence_engine_random=01639268860/store
    generator.go:416: mismatching output at line 288, operation index 187: expected valid = false
        , got key = "nytaxpdxyq"/0.000000026,0
    --- FAIL: TestPebbleEquivalence/engine/random=0 (0.12s)
=== RUN   TestPebbleEquivalence/engine/random=2
check file = /artifacts/tmp/_tmp/b2a21e13e3bb63e34c8b8052805e4125/TestPebbleEquivalence2826645106/output.meta
output file = /artifacts/tmp/_tmp/b2a21e13e3bb63e34c8b8052805e4125/TestPebbleEquivalence_engine_random=21437216804/output.meta
store path = /artifacts/tmp/_tmp/b2a21e13e3bb63e34c8b8052805e4125/TestPebbleEquivalence_engine_random=21303649320/store
    generator.go:416: mismatching output at line 288, operation index 187: expected valid = false
        , got key = "nytaxpdxyq"/0.000000026,0
    --- FAIL: TestPebbleEquivalence/engine/random=2 (0.11s)
=== RUN   TestPebbleEquivalence/engine/random=4
check file = /artifacts/tmp/_tmp/b2a21e13e3bb63e34c8b8052805e4125/TestPebbleEquivalence2826645106/output.meta
output file = /artifacts/tmp/_tmp/b2a21e13e3bb63e34c8b8052805e4125/TestPebbleEquivalence_engine_random=42446750057/output.meta
store path = /artifacts/tmp/_tmp/b2a21e13e3bb63e34c8b8052805e4125/TestPebbleEquivalence_engine_random=44041893817/store
    generator.go:416: mismatching output at line 288, operation index 187: expected valid = false
        , got key = "nytaxpdxyq"/0.000000026,0
    --- FAIL: TestPebbleEquivalence/engine/random=4 (0.11s)

Parameters: TAGS=bazel,gss

Help

See also: How To Investigate a Go Test Failure (internal)

/cc @cockroachdb/storage

This test on roachdash | Improve this report!

Jira issue: CRDB-18283

Epic CRDB-16238
@cockroach-teamcity cockroach-teamcity added branch-master Failures and bugs on the master branch. C-test-failure Broken test (automatically or manually discovered). O-robot Originated from a bot. labels Aug 3, 2022
@cockroach-teamcity cockroach-teamcity added this to the 22.2 milestone Aug 3, 2022
@cockroach-teamcity
Copy link
Member Author

storage/metamorphic.TestPebbleEquivalence failed with artifacts on master @ 47d34c9dd0c63dfd462e4380a65470f4cf18e3d6:

=== RUN   TestPebbleEquivalence
    test_log_scope.go:162: test logs captured to: /artifacts/tmp/_tmp/b2a21e13e3bb63e34c8b8052805e4125/logTestPebbleEquivalence754224738
    test_log_scope.go:80: use -show-logs to present logs inline
first run output file: /artifacts/tmp/_tmp/b2a21e13e3bb63e34c8b8052805e4125/TestPebbleEquivalence248149461/output.meta
=== CONT  TestPebbleEquivalence
    meta_test.go:174: -- test log scope end --
--- FAIL: TestPebbleEquivalence (152.87s)
=== RUN   TestPebbleEquivalence/engine/random=2
check file = /artifacts/tmp/_tmp/b2a21e13e3bb63e34c8b8052805e4125/TestPebbleEquivalence248149461/output.meta
output file = /artifacts/tmp/_tmp/b2a21e13e3bb63e34c8b8052805e4125/TestPebbleEquivalence_engine_random=22308291408/output.meta
store path = /artifacts/tmp/_tmp/b2a21e13e3bb63e34c8b8052805e4125/TestPebbleEquivalence_engine_random=2192258702/store
    generator.go:416: mismatching output at line 258, operation index 157: expected kvs = [{"yslccqsshe" {[0 0 0 0 3 103 118 104 116 100 120 108 116 108 106 110 111 115 120] 0.000000011,1}} {"xwmtfctsm" {[0 0 0 0 3 116 103 105 103 102 108 105 121 119 116 101 119 117] 0.000000014,0}} {"xityfkbbakk" {[0 0 0 0 3 118 108 97 117 97 120 110 109] 0.000000014,0}} {"wrvjslhlt" {[0 0 0 0 3 116 105 120 106 111 104 101 97 110 102 101 106 106 109 110] 0.000000014,0}} {"webbmjvtne" {[] 0.000000011,1}} {"vglrddyoxwa" {[0 0 0 0 3 110 120 109 107 114 110 119] 0.000000014,0}} {"vcnaqgrcy" {[0 0 0 0 3 105 109 97 100 97 100 120 119 112] 0.000000011,2}} {"unhjkkngvmg" {[0 0 0 0 3 120 100 112 120 103] 0.000000011,1}} {"uhdxduawihaea" {[0 0 0 0 3 114 114 99 102 118 110 109 107 121 109] 0.000000011,2}} {"trgkqigssti" {[0 0 0 0 3 109 106 104 102 104 104 111 111 103 103 118 111] 0.000000011,1}} {"rrteudoi" {[] 0.000000011,0}} {"rojcallsplcjqiu" {[] 0.000000011,0}} {"qgulmswllo" {[] 0.000000011,0}} {"qcpokdsi" {[0 0 0 0 3 114 111 97 116 120 116 100 101] 0.000000011,1}} {"psbifwmaqycnin" {[] 0.000000011,0}} {"pgababyofk" {[0 0 0 0 3 121 108 114 98 103 103 107 97] 0.000000014,0}} {"ofttqirfw" {[0 0 0 0 3 110 113 121 101 110 103 113 100] 0.000000014,0}}], intents = [], resumeSpan = <nil>, numBytes = 689, numKeys = 17
        , got kvs = [{"yslccqsshe" {[0 0 0 0 3 103 118 104 116 100 120 108 116 108 106 110 111 115 120] 0.000000011,1}} {"xwmtfctsm" {[0 0 0 0 3 116 103 105 103 102 108 105 121 119 116 101 119 117] 0.000000014,0}} {"xityfkbbakk" {[0 0 0 0 3 118 108 97 117 97 120 110 109] 0.000000014,0}} {"wrvjslhlt" {[0 0 0 0 3 116 105 120 106 111 104 101 97 110 102 101 106 106 109 110] 0.000000014,0}} {"webbmjvtne" {[] 0.000000011,1}} {"vglrddyoxwa" {[10 26 10 16 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 7 42 2 8 14 56 11 74 0 18 4 8 14 16 0 24 0 32 12 40 19 72 1] 0,0}} {"vcnaqgrcy" {[0 0 0 0 3 105 109 97 100 97 100 120 119 112] 0.000000011,2}} {"unhjkkngvmg" {[0 0 0 0 3 120 100 112 120 103] 0.000000011,1}} {"uhdxduawihaea" {[0 0 0 0 3 114 114 99 102 118 110 109 107 121 109] 0.000000011,2}} {"trgkqigssti" {[0 0 0 0 3 109 106 104 102 104 104 111 111 103 103 118 111] 0.000000011,1}} {"rrteudoi" {[] 0.000000011,0}} {"rojcallsplcjqiu" {[] 0.000000011,0}} {"qgulmswllo" {[] 0.000000011,0}} {"qcpokdsi" {[0 0 0 0 3 114 111 97 116 120 116 100 101] 0.000000011,1}} {"psbifwmaqycnin" {[] 0.000000011,0}} {"pgababyofk" {[0 0 0 0 3 121 108 114 98 103 103 107 97] 0.000000014,0}} {"ofttqirfw" {[0 0 0 0 3 110 113 121 101 110 103 113 100] 0.000000014,0}}], intents = [], resumeSpan = <nil>, numBytes = 710, numKeys = 17
    --- FAIL: TestPebbleEquivalence/engine/random=2 (0.13s)

Parameters: TAGS=bazel,gss,deadlock

Help

See also: How To Investigate a Go Test Failure (internal)

This test on roachdash | Improve this report!

@jbowens jbowens self-assigned this Aug 3, 2022
@jbowens
Copy link
Collaborator

jbowens commented Aug 3, 2022

I expect this is related to cockroachdb/pebble#1840, hopefully due to another instance of inappropriately setting the same key twice within an external sstable.

@jbowens jbowens mentioned this issue Aug 3, 2022
Merged
@nicktrav nicktrav added the T-storage Storage Team label Aug 3, 2022
@craig craig bot closed this as completed in 9650b0d Aug 3, 2022
@nicktrav nicktrav mentioned this issue Aug 5, 2022
Closed
@jbowens jbowens mentioned this issue Aug 8, 2022
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@jbowens jbowens

Labels
branch-master Failures and bugs on the master branch. C-test-failure Broken test (automatically or manually discovered). O-robot Originated from a bot. T-storage Storage Team
Projects
None yet
Milestone
22.2
Development

Successfully merging a pull request may close this issue.

storage: deduplicate ingested keys in MVCC metamorphic tests jbowens/cockroach
3 participants
@nicktrav @jbowens @cockroach-teamcity