kv/kvserver: TestClosedTimestampWorksWhenRequestsAreSentToNonLeaseHolders failed [old closedts] #60682

cockroach-teamcity · 2021-02-17T16:44:07Z

(kv/kvserver).TestClosedTimestampWorksWhenRequestsAreSentToNonLeaseHolders failed on master@e7bee86121642e26a233c01913c19ef366a9235a:

=== RUN   TestClosedTimestampWorksWhenRequestsAreSentToNonLeaseHolders
    test_log_scope.go:73: test logs captured to: /go/src/github.com/cockroachdb/cockroach/artifacts/logTestClosedTimestampWorksWhenRequestsAreSentToNonLeaseHolders637603134
    test_log_scope.go:74: use -show-logs to present logs inline
    client_closed_timestamp_test.go:85: condition failed to evaluate within 45s: "/Table/Max": transfer lease unexpected error: cannot replace lease repl=(n4,s4):3 seq=75 start=1613579070.393241544,0 epo=1 pro=1613579071.395114327,0 with repl=(n1,s1):1 seq=0 start=1613579071.397144482,0 exp=<nil> pro=1613579071.397014503,0: illegal lease: epoch=0, interval=[1613579071.397144482,0, <nil>)
        goroutine 66825 [running]:
        runtime/debug.Stack(0xc004675018, 0x52af860, 0xc00343e7a0)
        	/usr/local/go/src/runtime/debug/stack.go:24 +0x9f
        github.com/cockroachdb/cockroach/pkg/testutils.SucceedsSoon(0x548b380, 0xc006664300, 0xc004675018)
        	/go/src/github.com/cockroachdb/cockroach/pkg/testutils/soon.go:36 +0x6d
        github.com/cockroachdb/cockroach/pkg/kv/kvserver_test.TestClosedTimestampWorksWhenRequestsAreSentToNonLeaseHolders.func1(0xc0005e6310, 0x100000001)
        	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvserver/client_closed_timestamp_test.go:85 +0x7c
        github.com/cockroachdb/cockroach/pkg/kv/kvserver_test.TestClosedTimestampWorksWhenRequestsAreSentToNonLeaseHolders.func2(0x0, 0x0)
        	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvserver/client_closed_timestamp_test.go:97 +0x151
        github.com/cockroachdb/cockroach/pkg/kv/kvserver_test.TestClosedTimestampWorksWhenRequestsAreSentToNonLeaseHolders(0xc006664300)
        	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvserver/client_closed_timestamp_test.go:140 +0x9bb
        testing.tRunner(0xc006664300, 0x4cb1f10)
        	/usr/local/go/src/testing/testing.go:1123 +0xef
        created by testing.(*T).Run
        	/usr/local/go/src/testing/testing.go:1168 +0x2b3
    panic.go:617: -- test log scope end --
test logs left over in: /go/src/github.com/cockroachdb/cockroach/artifacts/logTestClosedTimestampWorksWhenRequestsAreSentToNonLeaseHolders637603134
--- FAIL: TestClosedTimestampWorksWhenRequestsAreSentToNonLeaseHolders (47.57s)

More

Parameters:

GOFLAGS=-json

make stressrace TESTS=TestClosedTimestampWorksWhenRequestsAreSentToNonLeaseHolders PKG=./pkg/kv/kvserver TESTTIMEOUT=5m STRESSFLAGS='-timeout 5m' 2>&1

See this test on roachdash
_{powered by pkg/cmd/internal/issues}

Epic: CRDB-2557

The text was updated successfully, but these errors were encountered:

cockroach-teamcity · 2021-03-17T15:43:58Z

(kv/kvserver).TestClosedTimestampWorksWhenRequestsAreSentToNonLeaseHolders failed on master@3267fc56b51ed7757858032d9cd0634ab401efaa:

=== RUN   TestClosedTimestampWorksWhenRequestsAreSentToNonLeaseHolders
    test_log_scope.go:73: test logs captured to: /go/src/github.com/cockroachdb/cockroach/artifacts/logTestClosedTimestampWorksWhenRequestsAreSentToNonLeaseHolders586246913
    test_log_scope.go:74: use -show-logs to present logs inline
    client_closed_timestamp_test.go:87: condition failed to evaluate within 45s: "/Table/Max": transfer lease unexpected error: cannot replace lease repl=(n4,s4):3 seq=2 start=1615994713.537229456,0 epo=1 pro=1615994713.537114180,0 with repl=(n1,s1):1 seq=0 start=1615994757.817792885,0 exp=<nil> pro=1615994757.817792885,0: couldn't request lease for (n1,s1):1: liveness record not found in cache
        goroutine 69311 [running]:
        runtime/debug.Stack(0xc00351cfe8, 0x54a0ae0, 0xc003617040)
        	/usr/local/go/src/runtime/debug/stack.go:24 +0x9f
        github.com/cockroachdb/cockroach/pkg/testutils.SucceedsSoon(0x5680da0, 0xc007158000, 0xc00351cfe8)
        	/go/src/github.com/cockroachdb/cockroach/pkg/testutils/soon.go:36 +0x6d
        github.com/cockroachdb/cockroach/pkg/kv/kvserver_test.TestClosedTimestampWorksWhenRequestsAreSentToNonLeaseHolders.func1(0xc001004700, 0x100000001)
        	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvserver/client_closed_timestamp_test.go:87 +0x7c
        github.com/cockroachdb/cockroach/pkg/kv/kvserver_test.TestClosedTimestampWorksWhenRequestsAreSentToNonLeaseHolders.func2(0x0, 0x0)
        	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvserver/client_closed_timestamp_test.go:99 +0x151
        github.com/cockroachdb/cockroach/pkg/kv/kvserver_test.TestClosedTimestampWorksWhenRequestsAreSentToNonLeaseHolders(0xc007158000)
        	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvserver/client_closed_timestamp_test.go:142 +0xa3b
        testing.tRunner(0xc007158000, 0x4e89e68)
        	/usr/local/go/src/testing/testing.go:1123 +0xef
        created by testing.(*T).Run
        	/usr/local/go/src/testing/testing.go:1168 +0x2b3
    panic.go:617: -- test log scope end --
test logs left over in: /go/src/github.com/cockroachdb/cockroach/artifacts/logTestClosedTimestampWorksWhenRequestsAreSentToNonLeaseHolders586246913
--- FAIL: TestClosedTimestampWorksWhenRequestsAreSentToNonLeaseHolders (47.20s)

More

Parameters:

GOFLAGS=-json

make stressrace TESTS=TestClosedTimestampWorksWhenRequestsAreSentToNonLeaseHolders PKG=./pkg/kv/kvserver TESTTIMEOUT=5m STRESSFLAGS='-timeout 5m' 2>&1

See this test on roachdash
_{powered by pkg/cmd/internal/issues}

cockroach-teamcity · 2021-04-14T18:05:14Z

kv/kvserver.TestClosedTimestampWorksWhenRequestsAreSentToNonLeaseHolders failed with artifacts on master @ f60daea1297e4ab84be428e045a4d985fbe29718:

=== RUN   TestClosedTimestampWorksWhenRequestsAreSentToNonLeaseHolders
    test_log_scope.go:73: test logs captured to: /go/src/github.com/cockroachdb/cockroach/artifacts/logTestClosedTimestampWorksWhenRequestsAreSentToNonLeaseHolders356413558
    test_log_scope.go:74: use -show-logs to present logs inline
    client_closed_timestamp_test.go:87: condition failed to evaluate within 45s: "/Table/Max": transfer lease unexpected error: cannot replace lease repl=(n4,s4):3 seq=2 start=1618422280.006895891,0 epo=1 pro=1618422280.006802583,0 with repl=(n1,s1):1 seq=0 start=1618422324.308568654,0 exp=<nil> pro=1618422324.308568654,0: couldn't request lease for (n1,s1):1: liveness record not found in cache
        goroutine 72467 [running]:
        runtime/debug.Stack(0xc00243ef28, 0x55bae00, 0xc004fce440)
        	/usr/local/go/src/runtime/debug/stack.go:24 +0x9f
        github.com/cockroachdb/cockroach/pkg/testutils.SucceedsSoon(0x579dd80, 0xc004a1a780, 0xc00243ef28)
        	/go/src/github.com/cockroachdb/cockroach/pkg/testutils/soon.go:36 +0x6d
        github.com/cockroachdb/cockroach/pkg/kv/kvserver_test.TestClosedTimestampWorksWhenRequestsAreSentToNonLeaseHolders.func1(0xc002c74070, 0x100000001)
        	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvserver/client_closed_timestamp_test.go:87 +0x7c
        github.com/cockroachdb/cockroach/pkg/kv/kvserver_test.TestClosedTimestampWorksWhenRequestsAreSentToNonLeaseHolders.func2(0x0, 0x0)
        	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvserver/client_closed_timestamp_test.go:99 +0x151
        github.com/cockroachdb/cockroach/pkg/kv/kvserver_test.TestClosedTimestampWorksWhenRequestsAreSentToNonLeaseHolders(0xc004a1a780)
        	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvserver/client_closed_timestamp_test.go:142 +0xa7b
        testing.tRunner(0xc004a1a780, 0x4f7d8e8)
        	/usr/local/go/src/testing/testing.go:1123 +0xef
        created by testing.(*T).Run
        	/usr/local/go/src/testing/testing.go:1168 +0x2b3
    panic.go:617: -- test log scope end --
test logs left over in: /go/src/github.com/cockroachdb/cockroach/artifacts/logTestClosedTimestampWorksWhenRequestsAreSentToNonLeaseHolders356413558
--- FAIL: TestClosedTimestampWorksWhenRequestsAreSentToNonLeaseHolders (47.34s)

Reproduce

To reproduce, try:

make stressrace TESTS=TestClosedTimestampWorksWhenRequestsAreSentToNonLeaseHolders PKG=./pkg/kv/kvserver TESTTIMEOUT=5m STRESSFLAGS='-timeout 5m' 2>&1

Parameters in this failure:

GOFLAGS=-json

/cc @ajwerner _{This test on roachdash | Improve this report!}

…seHolders Refs: cockroachdb#60682 Reason: flaky test Generated by bin/skip-test. Release justification: non-production code changes Release note: None

63411: opt: make check constraint and partial index columns anonymous r=mgartner a=mgartner #### opt: make check constraint and partial index columns anonymous This commit updates optbuilder so that synthesized columns used for maintaining check constraints and partial indexes are anonymized in the scope resulting from their projection. This prevents these columns from being found in the scope when resolving columns while building expressions. This simplifies some code and eliminates a class of bugs that can cause "ambiguous column" errors when a user's table has column names that match the synthesized column names. Now that these synthesized columns are anonymous, there cannot be ambiguity between them and table columns. Release note: None #### opt: simplify partial index PUT/DEL column building Previously, the scopes to build synthesized partial index PUT and DEL columns were passed as input the building functions. They were always called with `mb.outScope` or `mb.fetchScope`, so there was no need to provide them as arguments. The building functions no longer accept any arguments. Release note: None 63638: gcjob: share ClearRange logic for tables and indexes r=pbardea a=ajwerner No obvious reason to keep these separate. Release note: None 63645: scripts: allow GCEWORKER_USER env to override ssh user r=RaduBerinde a=cucaroach fixes #63641 63652: kv/kvserver: skip TestClosedTimestampWorksWhenRequestsAreSentToNonLeaseHolders r=RaduBerinde a=RaduBerinde Refs: #60682 Reason: flaky test Generated by bin/skip-test. Release justification: non-production code changes Release note: None Co-authored-by: Marcus Gartner <[email protected]> Co-authored-by: Andrew Werner <[email protected]> Co-authored-by: Tommy Reilly <[email protected]> Co-authored-by: Radu Berinde <[email protected]>

Closes cockroachdb#61989 Closes cockroachdb#60682 Closes cockroachdb#59448

cockroach-teamcity added branch-master Failures and bugs on the master branch. C-test-failure Broken test (automatically or manually discovered). O-robot Originated from a bot. labels Feb 17, 2021

cockroach-teamcity assigned ajwerner Feb 17, 2021

ajwerner assigned andreimatei and unassigned ajwerner Feb 18, 2021

nvanbenschoten mentioned this issue Apr 9, 2021

roachtest: wait for rebalancing to each region in follower-reads test #63353

Merged

RaduBerinde mentioned this issue Apr 14, 2021

kv/kvserver: skip TestClosedTimestampWorksWhenRequestsAreSentToNonLeaseHolders #63652

Merged

RaduBerinde added the skipped-test label Apr 14, 2021

tbg changed the title ~~kv/kvserver: TestClosedTimestampWorksWhenRequestsAreSentToNonLeaseHolders failed~~ kv/kvserver: TestClosedTimestampWorksWhenRequestsAreSentToNonLeaseHolders failed [old closedts] Jun 1, 2021

jlinder added the T-kv KV Team label Jun 16, 2021

andreimatei added a commit to andreimatei/cockroach that referenced this issue Aug 19, 2021

closedts: delete old closedts mechanism

15636bc

Closes cockroachdb#61989 Closes cockroachdb#60682 Closes cockroachdb#59448

andreimatei added a commit to andreimatei/cockroach that referenced this issue Aug 26, 2021

closedts: delete old closedts mechanism

c4724db

Closes cockroachdb#61989 Closes cockroachdb#60682 Closes cockroachdb#59448

andreimatei added a commit to andreimatei/cockroach that referenced this issue Aug 30, 2021

closedts: delete old closedts mechanism

71054f6

Closes cockroachdb#61989 Closes cockroachdb#60682 Closes cockroachdb#59448

craig bot closed this as completed in 793b4c8 Aug 31, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

kv/kvserver: TestClosedTimestampWorksWhenRequestsAreSentToNonLeaseHolders failed [old closedts] #60682

kv/kvserver: TestClosedTimestampWorksWhenRequestsAreSentToNonLeaseHolders failed [old closedts] #60682

cockroach-teamcity commented Feb 17, 2021 •

edited by lunevalex

Loading

cockroach-teamcity commented Mar 17, 2021

cockroach-teamcity commented Apr 14, 2021

kv/kvserver: TestClosedTimestampWorksWhenRequestsAreSentToNonLeaseHolders failed [old closedts] #60682

kv/kvserver: TestClosedTimestampWorksWhenRequestsAreSentToNonLeaseHolders failed [old closedts] #60682

Comments

cockroach-teamcity commented Feb 17, 2021 • edited by lunevalex Loading

cockroach-teamcity commented Mar 17, 2021

cockroach-teamcity commented Apr 14, 2021

cockroach-teamcity commented Feb 17, 2021 •

edited by lunevalex

Loading