Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

roachtest: follower-reads/nodes=3 failed #50262

Closed
cockroach-teamcity opened this issue Jun 16, 2020 · 27 comments
Closed

roachtest: follower-reads/nodes=3 failed #50262

cockroach-teamcity opened this issue Jun 16, 2020 · 27 comments
Assignees
@andreimatei
Labels
branch-master Failures and bugs on the master branch. C-test-failure Broken test (automatically or manually discovered). O-roachtest O-robot Originated from a bot. release-blocker Indicates a release-blocker. Use with branch-release-2x.x label to denote which branch is blocked.
Milestone
20.2

Comments

@cockroach-teamcity
Copy link
Member

(roachtest).follower-reads/nodes=3 failed on master@9eea0be192499fa7d29b5eca5ed14173982b25f0:

The test failed on branch=master, cloud=gce:
test artifacts and logs in: /home/agent/work/.go/src/github.com/cockroachdb/cockroach/artifacts/follower-reads/nodes=3/run_1
	follower_reads.go:210,test_runner.go:753: fewer than 2 follower reads occurred: saw [0 2 1] before and [0 3 1] after
More

Artifacts: /follower-reads/nodes=3
Related:

See this test on roachdash
powered by pkg/cmd/internal/issues
@cockroach-teamcity cockroach-teamcity added branch-master Failures and bugs on the master branch. C-test-failure Broken test (automatically or manually discovered). O-roachtest O-robot Originated from a bot. release-blocker Indicates a release-blocker. Use with branch-release-2x.x label to denote which branch is blocked. labels Jun 16, 2020
@cockroach-teamcity cockroach-teamcity added this to the 20.2 milestone Jun 16, 2020
@tbg
Copy link
Member

tbg commented Jun 16, 2020

@andreimatei this looks like you got assigned correctly for once! :-)

@cockroach-teamcity
Copy link
Member Author

(roachtest).follower-reads/nodes=3 failed on master@a8e821d13b2b623c38ba82c577536ecde47bd685:

The test failed on branch=master, cloud=gce:
test artifacts and logs in: /home/agent/work/.go/src/github.com/cockroachdb/cockroach/artifacts/follower-reads/nodes=3/run_1
	follower_reads.go:210,test_runner.go:753: fewer than 2 follower reads occurred: saw [0 2 1] before and [0 3 1] after
More

Artifacts: /follower-reads/nodes=3
Related:

See this test on roachdash
powered by pkg/cmd/internal/issues

@cockroach-teamcity
Copy link
Member Author

(roachtest).follower-reads/nodes=3 failed on master@d1f2dc29d4a07ea462845d63c9c2a331205ebcaa:

The test failed on branch=master, cloud=gce:
test artifacts and logs in: /home/agent/work/.go/src/github.com/cockroachdb/cockroach/artifacts/follower-reads/nodes=3/run_1
	follower_reads.go:210,test_runner.go:753: fewer than 2 follower reads occurred: saw [0 1 1] before and [2 1 1] after
More

Artifacts: /follower-reads/nodes=3
Related:

See this test on roachdash
powered by pkg/cmd/internal/issues

@cockroach-teamcity
Copy link
Member Author

(roachtest).follower-reads/nodes=3 failed on master@2dd15cd26f95801c678b1034e98a15560dc10baa:

The test failed on branch=master, cloud=gce:
test artifacts and logs in: /home/agent/work/.go/src/github.com/cockroachdb/cockroach/artifacts/follower-reads/nodes=3/run_1
	follower_reads.go:210,test_runner.go:753: fewer than 2 follower reads occurred: saw [0 2 1] before and [0 3 1] after
More

Artifacts: /follower-reads/nodes=3
Related:

See this test on roachdash
powered by pkg/cmd/internal/issues

@cockroach-teamcity
Copy link
Member Author

(roachtest).follower-reads/nodes=3 failed on master@eb69237e5ee14728f32bbb667695ebb7472d9535:

The test failed on branch=master, cloud=gce:
test artifacts and logs in: /home/agent/work/.go/src/github.com/cockroachdb/cockroach/artifacts/follower-reads/nodes=3/run_1
	follower_reads.go:210,test_runner.go:753: fewer than 2 follower reads occurred: saw [0 2 1] before and [0 3 1] after
More

Artifacts: /follower-reads/nodes=3
Related:

See this test on roachdash
powered by pkg/cmd/internal/issues

@cockroach-teamcity
Copy link
Member Author

(roachtest).follower-reads/nodes=3 failed on master@6b9c32bf8d8553be6b0540f82ce18327b88262b5:

The test failed on branch=master, cloud=gce:
test artifacts and logs in: /home/agent/work/.go/src/github.com/cockroachdb/cockroach/artifacts/follower-reads/nodes=3/run_1
	follower_reads.go:210,test_runner.go:753: fewer than 2 follower reads occurred: saw [0 2 1] before and [0 3 1] after
More

Artifacts: /follower-reads/nodes=3
Related:

See this test on roachdash
powered by pkg/cmd/internal/issues

@andreimatei
Copy link
Contributor

Seems like a real failure. Will look.

@cockroach-teamcity
Copy link
Member Author

(roachtest).follower-reads/nodes=3 failed on master@e85ac5f2d70542523a61f430f1fc354023b6c7e4:

The test failed on branch=master, cloud=gce:
test artifacts and logs in: /home/agent/work/.go/src/github.com/cockroachdb/cockroach/artifacts/follower-reads/nodes=3/run_1
	follower_reads.go:210,test_runner.go:753: fewer than 2 follower reads occurred: saw [0 2 1] before and [0 3 1] after
More

Artifacts: /follower-reads/nodes=3
Related:

See this test on roachdash
powered by pkg/cmd/internal/issues

@cockroach-teamcity
Copy link
Member Author

(roachtest).follower-reads/nodes=3 failed on master@4acd262796288cc84cb2b104bd77c1fc8cac73db:

The test failed on branch=master, cloud=gce:
test artifacts and logs in: /home/agent/work/.go/src/github.com/cockroachdb/cockroach/artifacts/follower-reads/nodes=3/run_1
	follower_reads.go:210,test_runner.go:753: fewer than 2 follower reads occurred: saw [0 2 1] before and [0 3 1] after
More

Artifacts: /follower-reads/nodes=3
Related:

See this test on roachdash
powered by pkg/cmd/internal/issues

@cockroach-teamcity
Copy link
Member Author

(roachtest).follower-reads/nodes=3 failed on master@d3791a81c0716478de08d44459d3fcf5b4f3ea1e:

The test failed on branch=master, cloud=gce:
test artifacts and logs in: /home/agent/work/.go/src/github.com/cockroachdb/cockroach/artifacts/follower-reads/nodes=3/run_1
	follower_reads.go:210,test_runner.go:753: fewer than 2 follower reads occurred: saw [0 2 1] before and [0 3 1] after
More

Artifacts: /follower-reads/nodes=3
Related:

See this test on roachdash
powered by pkg/cmd/internal/issues

@cockroach-teamcity
Copy link
Member Author

(roachtest).follower-reads/nodes=3 failed on master@8f768ad14cfb3f514db6d40465b2dd60ee1f2890:

The test failed on branch=master, cloud=gce:
test artifacts and logs in: /home/agent/work/.go/src/github.com/cockroachdb/cockroach/artifacts/follower-reads/nodes=3/run_1
	follower_reads.go:210,test_runner.go:757: fewer than 2 follower reads occurred: saw [0 1 1] before and [0 2 1] after
More

Artifacts: /follower-reads/nodes=3
Related:

See this test on roachdash
powered by pkg/cmd/internal/issues

@cockroach-teamcity
Copy link
Member Author

(roachtest).follower-reads/nodes=3 failed on master@c627e3490d30e8ba88f6c7136717a392a054da4e:

The test failed on branch=master, cloud=gce:
test artifacts and logs in: /home/agent/work/.go/src/github.com/cockroachdb/cockroach/artifacts/follower-reads/nodes=3/run_1
	follower_reads.go:210,test_runner.go:757: fewer than 2 follower reads occurred: saw [0 2 1] before and [0 3 1] after
More

Artifacts: /follower-reads/nodes=3
Related:

See this test on roachdash
powered by pkg/cmd/internal/issues

@cockroach-teamcity
Copy link
Member Author

(roachtest).follower-reads/nodes=3 failed on master@17c8048e80935f8a01477416980d18bf39cba1bb:

The test failed on branch=master, cloud=gce:
test artifacts and logs in: /home/agent/work/.go/src/github.com/cockroachdb/cockroach/artifacts/follower-reads/nodes=3/run_1
	follower_reads.go:210,test_runner.go:757: fewer than 2 follower reads occurred: saw [0 2 1] before and [0 3 1] after
More

Artifacts: /follower-reads/nodes=3
Related:

See this test on roachdash
powered by pkg/cmd/internal/issues

@cockroach-teamcity
Copy link
Member Author

(roachtest).follower-reads/nodes=3 failed on master@3a03f3843a8cdf04f82c52753c61cf01b0d2ddcd:

The test failed on branch=master, cloud=gce:
test artifacts and logs in: /home/agent/work/.go/src/github.com/cockroachdb/cockroach/artifacts/follower-reads/nodes=3/run_1
	follower_reads.go:210,test_runner.go:757: fewer than 2 follower reads occurred: saw [0 2 1] before and [2 2 1] after
More

Artifacts: /follower-reads/nodes=3
Related:

See this test on roachdash
powered by pkg/cmd/internal/issues

@cockroach-teamcity
Copy link
Member Author

(roachtest).follower-reads/nodes=3 failed on master@456a07cfc1e53b87abc7709052e54efb1450e758:

The test failed on branch=master, cloud=gce:
test artifacts and logs in: /home/agent/work/.go/src/github.com/cockroachdb/cockroach/artifacts/follower-reads/nodes=3/run_1
	follower_reads.go:210,test_runner.go:757: fewer than 2 follower reads occurred: saw [0 2 1] before and [0 3 1] after
More

Artifacts: /follower-reads/nodes=3
Related:

See this test on roachdash
powered by pkg/cmd/internal/issues

@cockroach-teamcity
Copy link
Member Author

(roachtest).follower-reads/nodes=3 failed on master@3e0de239121813ea4d47873388a2828a66d9edf7:

The test failed on branch=master, cloud=gce:
test artifacts and logs in: /home/agent/work/.go/src/github.com/cockroachdb/cockroach/artifacts/follower-reads/nodes=3/run_1
	follower_reads.go:210,test_runner.go:757: fewer than 2 follower reads occurred: saw [0 2 1] before and [0 3 1] after
More

Artifacts: /follower-reads/nodes=3
Related:

See this test on roachdash
powered by pkg/cmd/internal/issues

@cockroach-teamcity
Copy link
Member Author

(roachtest).follower-reads/nodes=3 failed on master@9304ecd70e9f3ba4cb16b5443a10b4e17d7baee0:

The test failed on branch=master, cloud=gce:
test artifacts and logs in: /home/agent/work/.go/src/github.com/cockroachdb/cockroach/artifacts/follower-reads/nodes=3/run_1
	follower_reads.go:210,test_runner.go:757: fewer than 2 follower reads occurred: saw [0 2 1] before and [0 3 1] after
More

Artifacts: /follower-reads/nodes=3
Related:

See this test on roachdash
powered by pkg/cmd/internal/issues

@cockroach-teamcity cockroach-teamcity mentioned this issue Jul 8, 2020
Closed
@cockroach-teamcity
Copy link
Member Author

(roachtest).follower-reads/nodes=3 failed on master@e3fb5aa18d0f5064f7ba5d4df3864e94b3abb96d:

The test failed on branch=master, cloud=gce:
test artifacts and logs in: /home/agent/work/.go/src/github.com/cockroachdb/cockroach/artifacts/follower-reads/nodes=3/run_1
	follower_reads.go:210,test_runner.go:757: fewer than 2 follower reads occurred: saw [0 2 1] before and [0 3 1] after
More

Artifacts: /follower-reads/nodes=3
Related:

See this test on roachdash
powered by pkg/cmd/internal/issues

@cockroach-teamcity cockroach-teamcity mentioned this issue Jul 9, 2020
Closed
@cockroach-teamcity
Copy link
Member Author

(roachtest).follower-reads/nodes=3 failed on master@1b5d070c93375d3e14c146241e8bafde349529bd:

The test failed on branch=master, cloud=gce:
test artifacts and logs in: /home/agent/work/.go/src/github.com/cockroachdb/cockroach/artifacts/follower-reads/nodes=3/run_1
	follower_reads.go:210,test_runner.go:757: fewer than 2 follower reads occurred: saw [0 1 1] before and [0 2 1] after
More

Artifacts: /follower-reads/nodes=3
Related:

See this test on roachdash
powered by pkg/cmd/internal/issues

@cockroach-teamcity
Copy link
Member Author

(roachtest).follower-reads/nodes=3 failed on master@542c83e7db1694208e09c7fea7d350db4993b897:

The test failed on branch=master, cloud=gce:
test artifacts and logs in: /home/agent/work/.go/src/github.com/cockroachdb/cockroach/artifacts/follower-reads/nodes=3/run_1
	follower_reads.go:210,test_runner.go:757: fewer than 2 follower reads occurred: saw [0 2 1] before and [0 3 1] after
More

Artifacts: /follower-reads/nodes=3
Related:

See this test on roachdash
powered by pkg/cmd/internal/issues

@cockroach-teamcity
Copy link
Member Author

(roachtest).follower-reads/nodes=3 failed on master@12b58af06d9104bc585bc0773bbe29d68ff2d826:

The test failed on branch=master, cloud=gce:
test artifacts and logs in: /home/agent/work/.go/src/github.com/cockroachdb/cockroach/artifacts/follower-reads/nodes=3/run_1
	follower_reads.go:210,test_runner.go:757: fewer than 2 follower reads occurred: saw [0 2 1] before and [0 3 1] after
More

Artifacts: /follower-reads/nodes=3
Related:

See this test on roachdash
powered by pkg/cmd/internal/issues

@cockroach-teamcity
Copy link
Member Author

(roachtest).follower-reads/nodes=3 failed on master@161d68e8c2149d401738d2c52cf70071a1c3b610:

The test failed on branch=master, cloud=gce:
test artifacts and logs in: /home/agent/work/.go/src/github.com/cockroachdb/cockroach/artifacts/follower-reads/nodes=3/run_1
	follower_reads.go:210,test_runner.go:757: fewer than 2 follower reads occurred: saw [0 2 1] before and [0 3 1] after
More

Artifacts: /follower-reads/nodes=3
Related:

See this test on roachdash
powered by pkg/cmd/internal/issues

@irfansharif
Copy link
Contributor

This is part of the release check list. #51035 (comment). #51195, #51111 are the dups. Checking it off given we're actively working on it.

@irfansharif irfansharif mentioned this issue Jul 15, 2020
Closed
25 tasks
@cockroach-teamcity cockroach-teamcity mentioned this issue Jul 22, 2020
Closed
@cockroach-teamcity
Copy link
Member Author

(roachtest).follower-reads/nodes=3 failed on master@e9a4f83e3eee59510f97db2c6e0df9b57cf6b944:

The test failed on branch=master, cloud=gce:
test artifacts and logs in: /home/agent/work/.go/src/github.com/cockroachdb/cockroach/artifacts/follower-reads/nodes=3/run_1
	follower_reads.go:210,test_runner.go:757: fewer than 2 follower reads occurred: saw [0 2 1] before and [0 3 1] after
More

Artifacts: /follower-reads/nodes=3
Related:

See this test on roachdash
powered by pkg/cmd/internal/issues

@knz
Copy link
Contributor

knz commented Jul 22, 2020

The error is still current.

@cockroach-teamcity
Copy link
Member Author

(roachtest).follower-reads/nodes=3 failed on master@b8a50cc4d062293915969cdc83e3ec4d057cede5:

The test failed on branch=master, cloud=gce:
test artifacts and logs in: /home/agent/work/.go/src/github.com/cockroachdb/cockroach/artifacts/follower-reads/nodes=3/run_1
	follower_reads.go:210,test_runner.go:757: fewer than 2 follower reads occurred: saw [0 2 1] before and [0 3 1] after
More

Artifacts: /follower-reads/nodes=3
Related:

See this test on roachdash
powered by pkg/cmd/internal/issues

@cockroach-teamcity
Copy link
Member Author

(roachtest).follower-reads/nodes=3 failed on master@bfa6307c292ef4dfed4a53cb99f506e6dab26533:

The test failed on branch=master, cloud=gce:
test artifacts and logs in: /home/agent/work/.go/src/github.com/cockroachdb/cockroach/artifacts/follower-reads/nodes=3/run_1
	follower_reads.go:294,follower_reads.go:228,test_runner.go:757: 16 latency values ([134.725629ms 134.709245ms 134.676477ms 134.660093ms 134.676477ms 134.709245ms 134.709245ms 134.709245ms 134.725629ms 134.742013ms 134.742013ms 134.774781ms 134.774781ms 134.742013ms 134.709245ms 134.709245ms]) are above target latency 20ms, 3 permitted
More

Artifacts: /follower-reads/nodes=3
Related:

See this test on roachdash
powered by pkg/cmd/internal/issues

This was referenced Aug 11, 2020
Closed
Closed
Closed
Closed
@tbg tbg closed this as completed Aug 27, 2020
@nvanbenschoten nvanbenschoten mentioned this issue Oct 12, 2020
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@andreimatei andreimatei

Labels
branch-master Failures and bugs on the master branch. C-test-failure Broken test (automatically or manually discovered). O-roachtest O-robot Originated from a bot. release-blocker Indicates a release-blocker. Use with branch-release-2x.x label to denote which branch is blocked.
Projects
None yet
Milestone
20.2
Development

No branches or pull requests
5 participants
@andreimatei @knz @tbg @irfansharif @cockroach-teamcity