Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

teamcity: failed tests on master: acceptance/TestBuildInfo #10368

Closed
cockroach-teamcity opened this issue Nov 1, 2016 · 1 comment
Closed

teamcity: failed tests on master: acceptance/TestBuildInfo #10368

cockroach-teamcity opened this issue Nov 1, 2016 · 1 comment
Labels
C-test-failure Broken test (automatically or manually discovered). O-robot Originated from a bot.
Milestone
Q2

Comments

@cockroach-teamcity
Copy link
Member

The following tests appear to have failed:

#40044:

--- FAIL: acceptance/TestBuildInfo (593.904s)
Test ended in panic.
I161101 16:56:56.296947 77 acceptance/cluster/localcluster.go:290  creating docker network with name: cockroachdb_acceptance-63bb6baa
I161101 16:56:56.403360 77 acceptance/cluster/localcluster.go:322  Initializing Cluster AdHoc 3x1:
{"name":"AdHoc 3x1","nodes":[{"count":3,"stores":[{"count":1,"max_ranges":0}]}],"duration":5000000000}
I161101 16:56:56.407328 77 acceptance/cluster/docker.go:107  ImagePull cockroachdb/builder:20161024-143446 already exists
I161101 16:56:57.556302 77 acceptance/cluster/localcluster.go:641  creating certs (1024bit) in: /home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/acceptance/.localcluster.certs.353351060
I161101 16:56:57.662441 77 acceptance/cluster/localcluster.go:425  creating docker container with name: roach-63bb6baa-0
I161101 16:56:58.697976 77 acceptance/cluster/localcluster.go:536  *** started roach-63bb6baa-0 ***
  ui:        https://127.0.0.1:33197
  trace:     https://127.0.0.1:33197/debug/requests
  logs:      /home/agent/work/.go/src/github.com/cockroachdb/cockroach/artifacts/acceptance/TestBuildInfo-63bb6baa/roach-63bb6baa-0/cockroach.INFO
  pprof:     docker exec -it 63db2 /bin/bash -c 'go tool pprof /cockroach <(wget --no-check-certificate -qO- https://$(hostname):8080/debug/pprof/heap)'
  cockroach: [start --ca-cert=/certs/ca.crt --cert=/certs/node.crt --key=/certs/node.key --host=roach-63bb6baa-0 --alsologtostderr=INFO --verbosity=1 --store=path=/data1.0 --log-dir=/logs/roach-63bb6baa-0 --logtostderr=false]

  cli-env:   COCKROACH_INSECURE=false COCKROACH_CA_CERT=/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/acceptance/.localcluster.certs.353351060/ca.crt COCKROACH_CERT=/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/acceptance/.localcluster.certs.353351060/node.crt COCKROACH_KEY=/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/acceptance/.localcluster.certs.353351060/node.key COCKROACH_HOST=127.0.0.1 COCKROACH_PORT=33197
I161101 16:56:58.698048 77 acceptance/cluster/localcluster.go:425  creating docker container with name: roach-63bb6baa-1
I161101 16:56:59.293509 77 acceptance/cluster/localcluster.go:536  *** started roach-63bb6baa-1 ***
  ui:        https://127.0.0.1:33199
  trace:     https://127.0.0.1:33199/debug/requests
  logs:      /home/agent/work/.go/src/github.com/cockroachdb/cockroach/artifacts/acceptance/TestBuildInfo-63bb6baa/roach-63bb6baa-1/cockroach.INFO
  pprof:     docker exec -it 820d1 /bin/bash -c 'go tool pprof /cockroach <(wget --no-check-certificate -qO- https://$(hostname):8080/debug/pprof/heap)'
  cockroach: [start --ca-cert=/certs/ca.crt --cert=/certs/node.crt --key=/certs/node.key --host=roach-63bb6baa-1 --alsologtostderr=INFO --verbosity=1 --store=path=/data2.0 --join=roach-63bb6baa-0:26257 --log-dir=/logs/roach-63bb6baa-1 --logtostderr=false]

  cli-env:   COCKROACH_INSECURE=false COCKROACH_CA_CERT=/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/acceptance/.localcluster.certs.353351060/ca.crt COCKROACH_CERT=/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/acceptance/.localcluster.certs.353351060/node.crt COCKROACH_KEY=/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/acceptance/.localcluster.certs.353351060/node.key COCKROACH_HOST=127.0.0.1 COCKROACH_PORT=33199
I161101 16:56:59.293596 77 acceptance/cluster/localcluster.go:425  creating docker container with name: roach-63bb6baa-2
I161101 16:57:00.788182 77 acceptance/cluster/localcluster.go:536  *** started roach-63bb6baa-2 ***
  ui:        https://127.0.0.1:33201
  trace:     https://127.0.0.1:33201/debug/requests
  logs:      /home/agent/work/.go/src/github.com/cockroachdb/cockroach/artifacts/acceptance/TestBuildInfo-63bb6baa/roach-63bb6baa-2/cockroach.INFO
  pprof:     docker exec -it 46d5b /bin/bash -c 'go tool pprof /cockroach <(wget --no-check-certificate -qO- https://$(hostname):8080/debug/pprof/heap)'
  cockroach: [start --ca-cert=/certs/ca.crt --cert=/certs/node.crt --key=/certs/node.key --host=roach-63bb6baa-2 --alsologtostderr=INFO --verbosity=1 --store=path=/data3.0 --join=roach-63bb6baa-0:26257 --log-dir=/logs/roach-63bb6baa-2 --logtostderr=false]

  cli-env:   COCKROACH_INSECURE=false COCKROACH_CA_CERT=/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/acceptance/.localcluster.certs.353351060/ca.crt COCKROACH_CERT=/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/acceptance/.localcluster.certs.353351060/node.crt COCKROACH_KEY=/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/acceptance/.localcluster.certs.353351060/node.key COCKROACH_HOST=127.0.0.1 COCKROACH_PORT=33201
I161101 16:57:00.788198 77 acceptance/replication_test.go:55  waiting for first range to have 3 replicas
I161101 16:57:01.790342 77 util/stop/stopper.go:396  stop has been called, stopping or quiescing all running tasks
I161101 16:57:02.792860 77 util/stop/stopper.go:396  stop has been called, stopping or quiescing all running tasks
I161101 16:57:03.795511 77 util/stop/stopper.go:396  stop has been called, stopping or quiescing all running tasks
I161101 16:57:04.798264 77 util/stop/stopper.go:396  stop has been called, stopping or quiescing all running tasks
I161101 16:57:05.800556 77 util/stop/stopper.go:396  stop has been called, stopping or quiescing all running tasks
I161101 16:57:06.803266 77 util/stop/stopper.go:396  stop has been called, stopping or quiescing all running tasks
I161101 16:57:07.807039 77 util/stop/stopper.go:396  stop has been called, stopping or quiescing all running tasks
panic: test timed out after 10m0s

goroutine 145 [running]:
panic(0x13cd7a0, 0xc4205258e0)
	/usr/local/go/src/runtime/panic.go:500 +0x1a1
testing.startAlarm.func1()
	/usr/local/go/src/testing/testing.go:918 +0x10b
created by time.goFunc
	/usr/local/go/src/time/sleep.go:154 +0x44

goroutine 1 [chan receive, 9 minutes]:
testing.(*T).Run(0xc420094180, 0x155c807, 0xd, 0x16a4c40, 0xc4202afc01)
	/usr/local/go/src/testing/testing.go:647 +0x316
testing.RunTests.func1(0xc420094180)
	/usr/local/go/src/testing/testing.go:793 +0x6d
testing.tRunner(0xc420094180, 0xc42044cda8)
	/usr/local/go/src/testing/testing.go:610 +0x81
testing.RunTests(0x16a75e8, 0x1e83e20, 0x24, 0x24, 0xc4202afdf0)
	/usr/local/go/src/testing/testing.go:799 +0x2f5
testing.(*M).Run(0xc42044cef8, 0x16a4d80)
	/usr/local/go/src/testing/testing.go:743 +0x85
github.com/cockroachdb/cockroach/pkg/acceptance.runTests(0xc42044cef8)
	/go/src/github.com/cockroachdb/cockroach/pkg/acceptance/util_test.go:138 +0x48
github.com/cockroachdb/cockroach/pkg/acceptance.TestMain(0xc42044cef8)
	/go/src/github.com/cockroachdb/cockroach/pkg/acceptance/main_test.go:38 +0x45
main.main()
	github.com/cockroachdb/cockroach/pkg/acceptance/_test/_testmain.go:122 +0xc6

goroutine 17 [syscall, 10 minutes, locked to thread]:
runtime.goexit()
	/usr/local/go/src/runtime/asm_amd64.s:2086 +0x1

goroutine 35 [syscall, 10 minutes]:
os/signal.signal_recv(0xc42002ffa8)
	/usr/local/go/src/runtime/sigqueue.go:116 +0x157
os/signal.loop()
	/usr/local/go/src/os/signal/signal_unix.go:22 +0x22
created by os/signal.init.1
	/usr/local/go/src/os/signal/signal_unix.go:28 +0x41

goroutine 34 [chan receive]:
github.com/cockroachdb/cockroach/pkg/util/log.(*loggingT).flushDaemon(0x21ebe20)
	/go/src/github.com/cockroachdb/cockroach/pkg/util/log/clog.go:1001 +0x77
created by github.com/cockroachdb/cockroach/pkg/util/log.init.1
	/go/src/github.com/cockroachdb/cockroach/pkg/util/log/clog.go:579 +0x95

goroutine 8 [chan receive, 10 minutes]:
github.com/cockroachdb/cockroach/pkg/acceptance.runTests.func1()
	/go/src/github.com/cockroachdb/cockroach/pkg/acceptance/util_test.go:129 +0xb6
created by github.com/cockroachdb/cockroach/pkg/acceptance.runTests
	/go/src/github.com/cockroachdb/cockroach/pkg/acceptance/util_test.go:137 +0x3a

goroutine 77 [select, 9 minutes]:
google.golang.org/grpc/transport.(*Stream).Header(0xc4200da870, 0x16a7420, 0xc4200eb1e8, 0x1ea5e80)
	/go/src/google.golang.org/grpc/transport/transport.go:239 +0x299
google.golang.org/grpc.recvResponse(0x1e9fe00, 0x2224ee8, 0x0, 0x0, 0x0, 0x0, 0x1e93940, 0xc4202c0180, 0x0, 0x0, ...)
	/go/src/google.golang.org/grpc/call.go:62 +0xad
google.golang.org/grpc.Invoke(0x7fc161bf89a8, 0xc420014558, 0x156f8c8, 0x21, 0x153a5a0, 0xc4203f5f10, 0x14dac80, 0xc4202f1740, 0xc420514200, 0x0, ...)
	/go/src/google.golang.org/grpc/call.go:202 +0x8a1
github.com/cockroachdb/cockroach/pkg/roachpb.(*externalClient).Batch(0xc420526058, 0x7fc161bf89a8, 0xc420014558, 0xc4203f5f10, 0x0, 0x0, 0x0, 0xc4200eb500, 0x72b3f8, 0xc420018480)
	/go/src/github.com/cockroachdb/cockroach/pkg/roachpb/api.pb.go:1553 +0xd2
github.com/cockroachdb/cockroach/pkg/internal/client.sender.Send(0x1e8d7c0, 0xc420526058, 0x7fc161bf89a8, 0xc420014558, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, ...)
	/go/src/github.com/cockroachdb/cockroach/pkg/internal/client/rpc_sender.go:48 +0xbf
github.com/cockroachdb/cockroach/pkg/internal/client.(*sender).Send(0xc420555260, 0x7fc161bf89a8, 0xc420014558, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, ...)
	<autogenerated>:4 +0xb0
github.com/cockroachdb/cockroach/pkg/internal/client.(*DB).send(0xc420561090, 0x7fc161bf89a8, 0xc420014558, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, ...)
	/go/src/github.com/cockroachdb/cockroach/pkg/internal/client/db.go:511 +0xbf
github.com/cockroachdb/cockroach/pkg/internal/client.(*DB).Run.func1(0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, ...)
	/go/src/github.com/cockroachdb/cockroach/pkg/internal/client/db.go:445 +0x6e
github.com/cockroachdb/cockroach/pkg/internal/client.sendAndFill(0xc42055a700, 0xc42037cc00, 0x0, 0x1ea6900)
	/go/src/github.com/cockroachdb/cockroach/pkg/internal/client/db.go:416 +0x168
github.com/cockroachdb/cockroach/pkg/internal/client.(*DB).Run(0xc420561090, 0x7fc161bf89a8, 0xc420014558, 0xc42037cc00, 0x8, 0x18)
	/go/src/github.com/cockroachdb/cockroach/pkg/internal/client/db.go:447 +0xac
github.com/cockroachdb/cockroach/pkg/internal/client.(*DB).Get(0xc420561090, 0x7fc161bf89a8, 0xc420014558, 0x14b2b20, 0xc42055a6e0, 0xc42055a6e0, 0x18, 0x18, 0x1, 0xc42055a6e0, ...)
	/go/src/github.com/cockroachdb/cockroach/pkg/internal/client/db.go:214 +0x9a
github.com/cockroachdb/cockroach/pkg/internal/client.(*DB).GetProto(0xc420561090, 0x7fc161bf89a8, 0xc420014558, 0x14b2b20, 0xc42055a6e0, 0x1e99e80, 0xc4202f16e0, 0xc4200eb9e0, 0x60033c)
	/go/src/github.com/cockroachdb/cockroach/pkg/internal/client/db.go:222 +0x69
github.com/cockroachdb/cockroach/pkg/acceptance.countRangeReplicas(0xc420561090, 0xc420069200, 0x0, 0xc420561001)
	/go/src/github.com/cockroachdb/cockroach/pkg/acceptance/replication_test.go:36 +0x157
github.com/cockroachdb/cockroach/pkg/acceptance.checkRangeReplication.func1(0x0, 0x0)
	/go/src/github.com/cockroachdb/cockroach/pkg/acceptance/replication_test.go:71 +0x1f4
github.com/cockroachdb/cockroach/pkg/util.RetryForDuration(0xa7a358200, 0xc4200ebce0, 0x2, 0x157c408)
	/go/src/github.com/cockroachdb/cockroach/pkg/util/testing.go:129 +0xcd
github.com/cockroachdb/cockroach/pkg/util.SucceedsSoonDepth(0x1, 0x1ea3240, 0xc4205343c0, 0xc4200ebce0)
	/go/src/github.com/cockroachdb/cockroach/pkg/util/testing.go:115 +0x41
github.com/cockroachdb/cockroach/pkg/util.SucceedsSoon(0x1ea3240, 0xc4205343c0, 0xc4200ebce0)
	/go/src/github.com/cockroachdb/cockroach/pkg/util/testing.go:109 +0x48
github.com/cockroachdb/cockroach/pkg/acceptance.checkRangeReplication(0xc4205343c0, 0x1ea79a0, 0xc4204fc100, 0x4a817c800)
	/go/src/github.com/cockroachdb/cockroach/pkg/acceptance/replication_test.go:83 +0x2d3
github.com/cockroachdb/cockroach/pkg/acceptance.StartCluster(0xc4205343c0, 0xc42040edf0, 0x9, 0xc42041c940, 0x1, 0x1, 0x12a05f200, 0x1ea79a0, 0xc4204fc100)
	/go/src/github.com/cockroachdb/cockroach/pkg/acceptance/util_test.go:345 +0x188
github.com/cockroachdb/cockroach/pkg/acceptance.runTestOnConfigs.func1(0xc4205343c0, 0xc4200ebec8, 0x16a4da0)
	/go/src/github.com/cockroachdb/cockroach/pkg/acceptance/util_test.go:312 +0x4e
github.com/cockroachdb/cockroach/pkg/acceptance.runTestOnConfigs(0xc4205343c0, 0x16a4da0)
	/go/src/github.com/cockroachdb/cockroach/pkg/acceptance/util_test.go:315 +0xe2
github.com/cockroachdb/cockroach/pkg/acceptance.TestBuildInfo(0xc4205343c0)
	/go/src/github.com/cockroachdb/cockroach/pkg/acceptance/build_info_test.go:30 +0x37
testing.tRunner(0xc4205343c0, 0x16a4c40)
	/usr/local/go/src/testing/testing.go:610 +0x81
created by testing.(*T).Run
	/usr/local/go/src/testing/testing.go:646 +0x2ec

goroutine 37 [select, 10 minutes, locked to thread]:
runtime.gopark(0x16a7ea0, 0x0, 0x1554bb0, 0x6, 0x18, 0x2)
	/usr/local/go/src/runtime/proc.go:259 +0x13a
runtime.selectgoImpl(0xc42002af30, 0x0, 0x18)
	/usr/local/go/src/runtime/select.go:423 +0x11d9
runtime.selectgo(0xc42002af30)
	/usr/local/go/src/runtime/select.go:238 +0x1c
runtime.ensureSigM.func1()
	/usr/local/go/src/runtime/signal1_unix.go:304 +0x2f3
runtime.goexit()
	/usr/local/go/src/runtime/asm_amd64.s:2086 +0x1

goroutine 11 [IO wait, 9 minutes]:
net.runtime_pollWait(0x7fc161c712b8, 0x72, 0x4)
	/usr/local/go/src/runtime/netpoll.go:160 +0x59
net.(*pollDesc).wait(0xc420477790, 0x72, 0xc4200439d0, 0xc420014068)
	/usr/local/go/src/net/fd_poll_runtime.go:73 +0x38
net.(*pollDesc).waitRead(0xc420477790, 0x1e94740, 0xc420014068)
	/usr/local/go/src/net/fd_poll_runtime.go:78 +0x34
net.(*netFD).Read(0xc420477730, 0xc420490000, 0x1000, 0x1000, 0x0, 0x1e94740, 0xc420014068)
	/usr/local/go/src/net/fd_unix.go:243 +0x1a1
net.(*conn).Read(0xc420032240, 0xc420490000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/net.go:173 +0x70
net/http.(*persistConn).Read(0xc42000b000, 0xc420490000, 0x1000, 0x1000, 0x30, 0xc420043b58, 0x60033c)
	/usr/local/go/src/net/http/transport.go:1261 +0x154
bufio.(*Reader).fill(0xc420068de0)
	/usr/local/go/src/bufio/bufio.go:97 +0x10c
bufio.(*Reader).Peek(0xc420068de0, 0x1, 0x0, 0x1, 0x0, 0xc420156a80, 0x0)
	/usr/local/go/src/bufio/bufio.go:129 +0x62
net/http.(*persistConn).readLoop(0xc42000b000)
	/usr/local/go/src/net/http/transport.go:1418 +0x1a1
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:1062 +0x4e9

goroutine 12 [select, 9 minutes]:
net/http.(*persistConn).writeLoop(0xc42000b000)
	/usr/local/go/src/net/http/transport.go:1646 +0x3bd
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:1063 +0x50e

goroutine 160 [IO wait]:
net.runtime_pollWait(0x7fc161c70fb8, 0x72, 0x8)
	/usr/local/go/src/runtime/netpoll.go:160 +0x59
net.(*pollDesc).wait(0xc420476d80, 0x72, 0xc42027a828, 0xc420014068)
	/usr/local/go/src/net/fd_poll_runtime.go:73 +0x38
net.(*pollDesc).waitRead(0xc420476d80, 0x1e94740, 0xc420014068)
	/usr/local/go/src/net/fd_poll_runtime.go:78 +0x34
net.(*netFD).Read(0xc420476d20, 0xc420506c00, 0x400, 0x400, 0x0, 0x1e94740, 0xc420014068)
	/usr/local/go/src/net/fd_unix.go:243 +0x1a1
net.(*conn).Read(0xc42012e140, 0xc420506c00, 0x400, 0x400, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/net.go:173 +0x70
crypto/tls.(*block).readFromUntil(0xc42052b470, 0x7fc161c73de0, 0xc42012e140, 0x5, 0xc42012e140, 0xc42040e580)
	/usr/local/go/src/crypto/tls/conn.go:476 +0x91
crypto/tls.(*Conn).readRecord(0xc42046e700, 0x16a7f17, 0xc42046e808, 0x619fc0)
	/usr/local/go/src/crypto/tls/conn.go:578 +0xc4
crypto/tls.(*Conn).Read(0xc42046e700, 0xc4204ca000, 0x8000, 0x8000, 0x0, 0x0, 0x0)
	/usr/local/go/src/crypto/tls/conn.go:1113 +0x116
bufio.(*Reader).fill(0xc4202f12c0)
	/usr/local/go/src/bufio/bufio.go:97 +0x10c
bufio.(*Reader).Read(0xc4202f12c0, 0xc4202cc338, 0x9, 0x9, 0xc42027aca8, 0x619eb0, 0xc420362100)
	/usr/local/go/src/bufio/bufio.go:209 +0x1bc
io.ReadAtLeast(0x1e8ce40, 0xc4202f12c0, 0xc4202cc338, 0x9, 0x9, 0x9, 0x30, 0xc4201efdd8, 0x1e90b80)
	/usr/local/go/src/io/io.go:307 +0xa4
io.ReadFull(0x1e8ce40, 0xc4202f12c0, 0xc4202cc338, 0x9, 0x9, 0xc42027aeb0, 0xedcc00, 0xc4201cf2c0)
	/usr/local/go/src/io/io.go:325 +0x58
golang.org/x/net/http2.readFrameHeader(0xc4202cc338, 0x9, 0x9, 0x1e8ce40, 0xc4202f12c0, 0x0, 0xc400000000, 0xc42040e580, 0xc)
	/go/src/golang.org/x/net/http2/frame.go:237 +0x7b
golang.org/x/net/http2.(*Framer).ReadFrame(0xc4202cc300, 0x0, 0x0, 0x0, 0x0)
	/go/src/golang.org/x/net/http2/frame.go:464 +0xa4
google.golang.org/grpc/transport.(*framer).readFrame(0xc4202bf770, 0xc4201efe90, 0xc4201efe90, 0x0, 0x0)
	/go/src/google.golang.org/grpc/transport/http_util.go:505 +0x2f
google.golang.org/grpc/transport.(*http2Client).reader(0xc420514300)
	/go/src/google.golang.org/grpc/transport/http2_client.go:921 +0xb6
created by google.golang.org/grpc/transport.newHTTP2Client
	/go/src/google.golang.org/grpc/transport/http2_client.go:203 +0x8b1

goroutine 82 [IO wait, 9 minutes]:
net.runtime_pollWait(0x7fc161c71138, 0x72, 0x6)
	/usr/local/go/src/runtime/netpoll.go:160 +0x59
net.(*pollDesc).wait(0xc4203f5090, 0x72, 0xc420141818, 0xc420014068)
	/usr/local/go/src/net/fd_poll_runtime.go:73 +0x38
net.(*pollDesc).waitRead(0xc4203f5090, 0x1e94740, 0xc420014068)
	/usr/local/go/src/net/fd_poll_runtime.go:78 +0x34
net.(*netFD).Read(0xc4203f5030, 0xc42026e000, 0x1000, 0x1000, 0x0, 0x1e94740, 0xc420014068)
	/usr/local/go/src/net/fd_unix.go:243 +0x1a1
net.(*conn).Read(0xc42012e128, 0xc42026e000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/net.go:173 +0x70
net/http.(*persistConn).Read(0xc4204fc200, 0xc42026e000, 0x1000, 0x1000, 0x9cc4a8, 0xc420532388, 0x7d)
	/usr/local/go/src/net/http/transport.go:1261 +0x154
bufio.(*Reader).fill(0xc4202f0e40)
	/usr/local/go/src/bufio/bufio.go:97 +0x10c
bufio.(*Reader).ReadSlice(0xc4202f0e40, 0x10a, 0x2, 0x0, 0x199, 0x186, 0x186)
	/usr/local/go/src/bufio/bufio.go:330 +0xb5
net/http/internal.readChunkLine(0xc4202f0e40, 0x5d3659, 0xc4202f0118, 0x0, 0x1, 0xc420141a88)
	/usr/local/go/src/net/http/internal/chunked.go:110 +0x34
net/http/internal.(*chunkedReader).beginChunk(0xc4201f00c0)
	/usr/local/go/src/net/http/internal/chunked.go:47 +0x32
net/http/internal.(*chunkedReader).Read(0xc4201f00c0, 0xc42020e001, 0x5ff, 0x5ff, 0x619db0, 0xc420141b70, 0xc420141b78)
	/usr/local/go/src/net/http/internal/chunked.go:77 +0x7b
net/http.(*body).readLocked(0xc420362040, 0xc42020e001, 0x5ff, 0x5ff, 0x2, 0x2, 0x0)
	/usr/local/go/src/net/http/transfer.go:651 +0x61
net/http.(*body).Read(0xc420362040, 0xc42020e001, 0x5ff, 0x5ff, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/http/transfer.go:643 +0xf6
net/http.(*bodyEOFSignal).Read(0xc420362080, 0xc42020e001, 0x5ff, 0x5ff, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/http/transport.go:1974 +0x116
encoding/json.(*Decoder).refill(0xc420532340, 0xa, 0x9)
	/usr/local/go/src/encoding/json/stream.go:152 +0xfa
encoding/json.(*Decoder).readValue(0xc420532340, 0x0, 0x0, 0x80)
	/usr/local/go/src/encoding/json/stream.go:128 +0x198
encoding/json.(*Decoder).Decode(0xc420532340, 0x139c720, 0xc420138700, 0x0, 0x0)
	/usr/local/go/src/encoding/json/stream.go:57 +0x8e
github.com/docker/docker/client.(*Client).Events.func1(0xc4202f0120, 0xc4203f4e70, 0x0, 0x0, 0x0, 0x0, 0xc4203b82d0, 0x7fc161c73d28, 0xc420474580, 0xc4202f00c0)
	/go/src/github.com/docker/docker/client/events.go:50 +0x2a7
created by github.com/docker/docker/client.(*Client).Events
	/go/src/github.com/docker/docker/client/events.go:63 +0xe2

goroutine 81 [select, 9 minutes]:
github.com/cockroachdb/cockroach/pkg/acceptance/cluster.(*LocalCluster).monitor.func1(0x1)
	/go/src/github.com/cockroachdb/cockroach/pkg/acceptance/cluster/localcluster.go:608 +0x4aa
github.com/cockroachdb/cockroach/pkg/acceptance/cluster.(*LocalCluster).monitor(0xc4204fc100)
	/go/src/github.com/cockroachdb/cockroach/pkg/acceptance/cluster/localcluster.go:628 +0x61
created by github.com/cockroachdb/cockroach/pkg/acceptance/cluster.(*LocalCluster).Start
	/go/src/github.com/cockroachdb/cockroach/pkg/acceptance/cluster/localcluster.go:652 +0x57e

goroutine 79 [select, 9 minutes]:
net/http.(*persistConn).readLoop(0xc4204fc200)
	/usr/local/go/src/net/http/transport.go:1541 +0x9c9
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:1062 +0x4e9

goroutine 80 [select, 9 minutes]:
net/http.(*persistConn).writeLoop(0xc4204fc200)
	/usr/local/go/src/net/http/transport.go:1646 +0x3bd
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:1063 +0x50e

goroutine 99 [IO wait, 9 minutes]:
net.runtime_pollWait(0x7fc161c711f8, 0x72, 0x7)
	/usr/local/go/src/runtime/netpoll.go:160 +0x59
net.(*pollDesc).wait(0xc4203f5100, 0x72, 0xc4202369d0, 0xc420014068)
	/usr/local/go/src/net/fd_poll_runtime.go:73 +0x38
net.(*pollDesc).waitRead(0xc4203f5100, 0x1e94740, 0xc420014068)
	/usr/local/go/src/net/fd_poll_runtime.go:78 +0x34
net.(*netFD).Read(0xc4203f50a0, 0xc42056a000, 0x1000, 0x1000, 0x0, 0x1e94740, 0xc420014068)
	/usr/local/go/src/net/fd_unix.go:243 +0x1a1
net.(*conn).Read(0xc4204fe040, 0xc42056a000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/net.go:173 +0x70
net/http.(*persistConn).Read(0xc4204fc400, 0xc42056a000, 0x1000, 0x1000, 0x8a0d30, 0xc420236b58, 0x5c90ad)
	/usr/local/go/src/net/http/transport.go:1261 +0x154
bufio.(*Reader).fill(0xc4204d34a0)
	/usr/local/go/src/bufio/bufio.go:97 +0x10c
bufio.(*Reader).Peek(0xc4204d34a0, 0x1, 0xc420236bbd, 0x1, 0x0, 0xc420381da0, 0x0)
	/usr/local/go/src/bufio/bufio.go:129 +0x62
net/http.(*persistConn).readLoop(0xc4204fc400)
	/usr/local/go/src/net/http/transport.go:1418 +0x1a1
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:1062 +0x4e9

goroutine 100 [select, 9 minutes]:
net/http.(*persistConn).writeLoop(0xc4204fc400)
	/usr/local/go/src/net/http/transport.go:1646 +0x3bd
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:1063 +0x50e

goroutine 159 [select]:
github.com/cockroachdb/cockroach/pkg/rpc.(*Context).runHeartbeat(0xc4202e4d80, 0xc420514200, 0xc4205543b0, 0xf, 0x0, 0x0)
	/go/src/github.com/cockroachdb/cockroach/pkg/rpc/context.go:292 +0x650
github.com/cockroachdb/cockroach/pkg/rpc.(*Context).GRPCDial.func1.1.1()
	/go/src/github.com/cockroachdb/cockroach/pkg/rpc/context.go:227 +0x69
github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker.func1(0xc420018480, 0xc4202becf0)
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:196 +0x7d
created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:197 +0x66

goroutine 142 [chan receive, 9 minutes]:
github.com/cockroachdb/cockroach/pkg/rpc.NewContext.func1()
	/go/src/github.com/cockroachdb/cockroach/pkg/rpc/context.go:138 +0x76
github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker.func1(0xc420018480, 0xc42041dc60)
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:196 +0x7d
created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:197 +0x66

goroutine 158 [select, 9 minutes]:
google.golang.org/grpc.(*addrConn).transportMonitor(0xc42009e3c0)
	/go/src/google.golang.org/grpc/clientconn.go:720 +0x63e
google.golang.org/grpc.(*ClientConn).resetAddrConn.func1(0xc42009e3c0)
	/go/src/google.golang.org/grpc/clientconn.go:479 +0x1de
created by google.golang.org/grpc.(*ClientConn).resetAddrConn
	/go/src/google.golang.org/grpc/clientconn.go:480 +0x3c7

goroutine 161 [select, 9 minutes]:
google.golang.org/grpc/transport.(*http2Client).controller(0xc420514300)
	/go/src/google.golang.org/grpc/transport/http2_client.go:999 +0x59d
created by google.golang.org/grpc/transport.newHTTP2Client
	/go/src/google.golang.org/grpc/transport/http2_client.go:233 +0xe78

Please assign, take a look and update the issue accordingly.
@cockroach-teamcity cockroach-teamcity added this to the Q2 milestone Nov 1, 2016
@cockroach-teamcity cockroach-teamcity added O-robot Originated from a bot. C-test-failure Broken test (automatically or manually discovered). labels Nov 1, 2016
@tamird
Copy link
Contributor

tamird commented Nov 3, 2016

#10409.

@tamird tamird closed this as completed Nov 3, 2016
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
C-test-failure Broken test (automatically or manually discovered). O-robot Originated from a bot.
Projects
None yet
Milestone
Q2
Development

No branches or pull requests
2 participants
@tamird @cockroach-teamcity