Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

teamcity: failed tests on master: acceptance/TestSingleKey #10371

Closed
cockroach-teamcity opened this issue Nov 1, 2016 · 1 comment
Closed

teamcity: failed tests on master: acceptance/TestSingleKey #10371

cockroach-teamcity opened this issue Nov 1, 2016 · 1 comment
Labels
C-test-failure Broken test (automatically or manually discovered). O-robot Originated from a bot.
Milestone
Q2

Comments

@cockroach-teamcity
Copy link
Member

The following tests appear to have failed:

#40119:

--- FAIL: acceptance/TestSingleKey (288.028s)
Test ended in panic.
I161101 18:21:48.375840 1546 acceptance/cluster/localcluster.go:290  creating docker network with name: cockroachdb_acceptance-c5b92605
I161101 18:21:48.463707 1546 acceptance/cluster/localcluster.go:322  Initializing Cluster AdHoc 3x1:
{"name":"AdHoc 3x1","nodes":[{"count":3,"stores":[{"count":1,"max_ranges":0}]}],"duration":5000000000}
I161101 18:21:48.468816 1546 acceptance/cluster/docker.go:107  ImagePull cockroachdb/builder:20161024-143446 already exists
I161101 18:21:49.457818 1546 acceptance/cluster/localcluster.go:641  creating certs (1024bit) in: /home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/acceptance/.localcluster.certs.114596107
I161101 18:21:49.577517 1546 acceptance/cluster/localcluster.go:425  creating docker container with name: roach-c5b92605-0
I161101 18:21:50.109016 1546 acceptance/cluster/localcluster.go:536  *** started roach-c5b92605-0 ***
  ui:        https://127.0.0.1:33555
  trace:     https://127.0.0.1:33555/debug/requests
  logs:      /home/agent/work/.go/src/github.com/cockroachdb/cockroach/artifacts/acceptance/TestSingleKey-c5b92605/roach-c5b92605-0/cockroach.INFO
  pprof:     docker exec -it cf0c2 /bin/bash -c 'go tool pprof /cockroach <(wget --no-check-certificate -qO- https://$(hostname):8080/debug/pprof/heap)'
  cockroach: [start --ca-cert=/certs/ca.crt --cert=/certs/node.crt --key=/certs/node.key --host=roach-c5b92605-0 --alsologtostderr=INFO --verbosity=1 --store=path=/data1.0 --log-dir=/logs/roach-c5b92605-0 --logtostderr=false]

  cli-env:   COCKROACH_INSECURE=false COCKROACH_CA_CERT=/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/acceptance/.localcluster.certs.114596107/ca.crt COCKROACH_CERT=/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/acceptance/.localcluster.certs.114596107/node.crt COCKROACH_KEY=/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/acceptance/.localcluster.certs.114596107/node.key COCKROACH_HOST=127.0.0.1 COCKROACH_PORT=33555
I161101 18:21:50.109085 1546 acceptance/cluster/localcluster.go:425  creating docker container with name: roach-c5b92605-1
I161101 18:21:50.743232 1546 acceptance/cluster/localcluster.go:536  *** started roach-c5b92605-1 ***
  ui:        https://127.0.0.1:33557
  trace:     https://127.0.0.1:33557/debug/requests
  logs:      /home/agent/work/.go/src/github.com/cockroachdb/cockroach/artifacts/acceptance/TestSingleKey-c5b92605/roach-c5b92605-1/cockroach.INFO
  pprof:     docker exec -it 2c6b9 /bin/bash -c 'go tool pprof /cockroach <(wget --no-check-certificate -qO- https://$(hostname):8080/debug/pprof/heap)'
  cockroach: [start --ca-cert=/certs/ca.crt --cert=/certs/node.crt --key=/certs/node.key --host=roach-c5b92605-1 --alsologtostderr=INFO --verbosity=1 --store=path=/data2.0 --join=roach-c5b92605-0:26257 --log-dir=/logs/roach-c5b92605-1 --logtostderr=false]

  cli-env:   COCKROACH_INSECURE=false COCKROACH_CA_CERT=/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/acceptance/.localcluster.certs.114596107/ca.crt COCKROACH_CERT=/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/acceptance/.localcluster.certs.114596107/node.crt COCKROACH_KEY=/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/acceptance/.localcluster.certs.114596107/node.key COCKROACH_HOST=127.0.0.1 COCKROACH_PORT=33557
I161101 18:21:50.743328 1546 acceptance/cluster/localcluster.go:425  creating docker container with name: roach-c5b92605-2
I161101 18:21:51.639416 1546 acceptance/cluster/localcluster.go:536  *** started roach-c5b92605-2 ***
  ui:        https://127.0.0.1:33559
  trace:     https://127.0.0.1:33559/debug/requests
  logs:      /home/agent/work/.go/src/github.com/cockroachdb/cockroach/artifacts/acceptance/TestSingleKey-c5b92605/roach-c5b92605-2/cockroach.INFO
  pprof:     docker exec -it 4ae32 /bin/bash -c 'go tool pprof /cockroach <(wget --no-check-certificate -qO- https://$(hostname):8080/debug/pprof/heap)'
  cockroach: [start --ca-cert=/certs/ca.crt --cert=/certs/node.crt --key=/certs/node.key --host=roach-c5b92605-2 --alsologtostderr=INFO --verbosity=1 --store=path=/data3.0 --join=roach-c5b92605-0:26257 --log-dir=/logs/roach-c5b92605-2 --logtostderr=false]

  cli-env:   COCKROACH_INSECURE=false COCKROACH_CA_CERT=/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/acceptance/.localcluster.certs.114596107/ca.crt COCKROACH_CERT=/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/acceptance/.localcluster.certs.114596107/node.crt COCKROACH_KEY=/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/acceptance/.localcluster.certs.114596107/node.key COCKROACH_HOST=127.0.0.1 COCKROACH_PORT=33559
I161101 18:21:51.639493 1546 acceptance/replication_test.go:55  waiting for first range to have 3 replicas
I161101 18:21:52.642356 1546 util/stop/stopper.go:396  stop has been called, stopping or quiescing all running tasks
I161101 18:21:53.644830 1546 util/stop/stopper.go:396  stop has been called, stopping or quiescing all running tasks
I161101 18:21:54.647175 1546 util/stop/stopper.go:396  stop has been called, stopping or quiescing all running tasks
I161101 18:21:55.649466 1546 util/stop/stopper.go:396  stop has been called, stopping or quiescing all running tasks
I161101 18:21:56.651996 1546 util/stop/stopper.go:396  stop has been called, stopping or quiescing all running tasks
I161101 18:21:57.654657 1546 util/stop/stopper.go:396  stop has been called, stopping or quiescing all running tasks
I161101 18:21:58.658487 1546 util/stop/stopper.go:396  stop has been called, stopping or quiescing all running tasks
panic: test timed out after 10m0s

goroutine 1649 [running]:
panic(0x13cd6e0, 0xc4203165d0)
	/usr/local/go/src/runtime/panic.go:500 +0x1a1
testing.startAlarm.func1()
	/usr/local/go/src/testing/testing.go:918 +0x10b
created by time.goFunc
	/usr/local/go/src/time/sleep.go:154 +0x44

goroutine 1 [chan receive, 4 minutes]:
testing.(*T).Run(0xc420094180, 0x155c62d, 0xd, 0x16a4b08, 0xc4202a9c01)
	/usr/local/go/src/testing/testing.go:647 +0x316
testing.RunTests.func1(0xc420094180)
	/usr/local/go/src/testing/testing.go:793 +0x6d
testing.tRunner(0xc420094180, 0xc420041da8)
	/usr/local/go/src/testing/testing.go:610 +0x81
testing.RunTests(0x16a73c8, 0x1e83e20, 0x24, 0x24, 0xc4202a9df0)
	/usr/local/go/src/testing/testing.go:799 +0x2f5
testing.(*M).Run(0xc420041ef8, 0x16a4b60)
	/usr/local/go/src/testing/testing.go:743 +0x85
github.com/cockroachdb/cockroach/pkg/acceptance.runTests(0xc420041ef8)
	/go/src/github.com/cockroachdb/cockroach/pkg/acceptance/util_test.go:138 +0x48
github.com/cockroachdb/cockroach/pkg/acceptance.TestMain(0xc420041ef8)
	/go/src/github.com/cockroachdb/cockroach/pkg/acceptance/main_test.go:38 +0x45
main.main()
	github.com/cockroachdb/cockroach/pkg/acceptance/_test/_testmain.go:122 +0xc6

goroutine 17 [syscall, 9 minutes, locked to thread]:
runtime.goexit()
	/usr/local/go/src/runtime/asm_amd64.s:2086 +0x1

goroutine 7 [chan receive, 9 minutes]:
github.com/cockroachdb/cockroach/pkg/acceptance.runTests.func1()
	/go/src/github.com/cockroachdb/cockroach/pkg/acceptance/util_test.go:129 +0xb6
created by github.com/cockroachdb/cockroach/pkg/acceptance.runTests
	/go/src/github.com/cockroachdb/cockroach/pkg/acceptance/util_test.go:137 +0x3a

goroutine 20 [chan receive]:
github.com/cockroachdb/cockroach/pkg/util/log.(*loggingT).flushDaemon(0x21ea920)
	/go/src/github.com/cockroachdb/cockroach/pkg/util/log/clog.go:1001 +0x77
created by github.com/cockroachdb/cockroach/pkg/util/log.init.1
	/go/src/github.com/cockroachdb/cockroach/pkg/util/log/clog.go:579 +0x95

goroutine 21 [syscall, 9 minutes]:
os/signal.signal_recv(0x0)
	/usr/local/go/src/runtime/sigqueue.go:116 +0x157
os/signal.loop()
	/usr/local/go/src/os/signal/signal_unix.go:22 +0x22
created by os/signal.init.1
	/usr/local/go/src/os/signal/signal_unix.go:28 +0x41

goroutine 1546 [select, 4 minutes]:
google.golang.org/grpc/transport.(*Stream).Header(0xc4201c34a0, 0x16a7200, 0xc4200e31e8, 0x1ea5ec0)
	/go/src/google.golang.org/grpc/transport/transport.go:239 +0x299
google.golang.org/grpc.recvResponse(0x1e9fe40, 0x22239e8, 0x0, 0x0, 0x0, 0x0, 0x1e93980, 0xc420539ba0, 0x0, 0x0, ...)
	/go/src/google.golang.org/grpc/call.go:62 +0xad
google.golang.org/grpc.Invoke(0x7f90cda5b528, 0xc420014558, 0x156f6c7, 0x21, 0x153a360, 0xc4202896c0, 0x14dab60, 0xc420676840, 0xc420358400, 0x0, ...)
	/go/src/google.golang.org/grpc/call.go:202 +0x8a1
github.com/cockroachdb/cockroach/pkg/roachpb.(*externalClient).Batch(0xc4204f4128, 0x7f90cda5b528, 0xc420014558, 0xc4202896c0, 0x0, 0x0, 0x0, 0xc4200e3500, 0x72b3f8, 0xc42046d680)
	/go/src/github.com/cockroachdb/cockroach/pkg/roachpb/api.pb.go:1553 +0xd2
github.com/cockroachdb/cockroach/pkg/internal/client.sender.Send(0x1e8d7c0, 0xc4204f4128, 0x7f90cda5b528, 0xc420014558, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, ...)
	/go/src/github.com/cockroachdb/cockroach/pkg/internal/client/rpc_sender.go:48 +0xbf
github.com/cockroachdb/cockroach/pkg/internal/client.(*sender).Send(0xc420595110, 0x7f90cda5b528, 0xc420014558, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, ...)
	<autogenerated>:4 +0xb0
github.com/cockroachdb/cockroach/pkg/internal/client.(*DB).send(0xc420319810, 0x7f90cda5b528, 0xc420014558, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, ...)
	/go/src/github.com/cockroachdb/cockroach/pkg/internal/client/db.go:511 +0xbf
github.com/cockroachdb/cockroach/pkg/internal/client.(*DB).Run.func1(0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, ...)
	/go/src/github.com/cockroachdb/cockroach/pkg/internal/client/db.go:445 +0x6e
github.com/cockroachdb/cockroach/pkg/internal/client.sendAndFill(0xc42055fae0, 0xc4202b9800, 0x0, 0x1ea6940)
	/go/src/github.com/cockroachdb/cockroach/pkg/internal/client/db.go:416 +0x168
github.com/cockroachdb/cockroach/pkg/internal/client.(*DB).Run(0xc420319810, 0x7f90cda5b528, 0xc420014558, 0xc4202b9800, 0x8, 0x18)
	/go/src/github.com/cockroachdb/cockroach/pkg/internal/client/db.go:447 +0xac
github.com/cockroachdb/cockroach/pkg/internal/client.(*DB).Get(0xc420319810, 0x7f90cda5b528, 0xc420014558, 0x14b2a00, 0xc42055fac0, 0xc42055fac0, 0x18, 0x18, 0x1, 0xc42055fac0, ...)
	/go/src/github.com/cockroachdb/cockroach/pkg/internal/client/db.go:214 +0x9a
github.com/cockroachdb/cockroach/pkg/internal/client.(*DB).GetProto(0xc420319810, 0x7f90cda5b528, 0xc420014558, 0x14b2a00, 0xc42055fac0, 0x1e99ec0, 0xc4206767e0, 0xc4200e39e0, 0x60033c)
	/go/src/github.com/cockroachdb/cockroach/pkg/internal/client/db.go:222 +0x69
github.com/cockroachdb/cockroach/pkg/acceptance.countRangeReplicas(0xc420319810, 0xc420626de0, 0x0, 0xc420319801)
	/go/src/github.com/cockroachdb/cockroach/pkg/acceptance/replication_test.go:36 +0x157
github.com/cockroachdb/cockroach/pkg/acceptance.checkRangeReplication.func1(0x0, 0x0)
	/go/src/github.com/cockroachdb/cockroach/pkg/acceptance/replication_test.go:71 +0x1f4
github.com/cockroachdb/cockroach/pkg/util.RetryForDuration(0xa7a358200, 0xc4200e3ce0, 0x2, 0x157c228)
	/go/src/github.com/cockroachdb/cockroach/pkg/util/testing.go:129 +0xcd
github.com/cockroachdb/cockroach/pkg/util.SucceedsSoonDepth(0x1, 0x1ea3280, 0xc4200940c0, 0xc4200e3ce0)
	/go/src/github.com/cockroachdb/cockroach/pkg/util/testing.go:115 +0x41
github.com/cockroachdb/cockroach/pkg/util.SucceedsSoon(0x1ea3280, 0xc4200940c0, 0xc4200e3ce0)
	/go/src/github.com/cockroachdb/cockroach/pkg/util/testing.go:109 +0x48
github.com/cockroachdb/cockroach/pkg/acceptance.checkRangeReplication(0xc4200940c0, 0x1ea79e0, 0xc42000ac00, 0x4a817c800)
	/go/src/github.com/cockroachdb/cockroach/pkg/acceptance/replication_test.go:83 +0x2d3
github.com/cockroachdb/cockroach/pkg/acceptance.StartCluster(0xc4200940c0, 0xc4202e24d0, 0x9, 0xc4202e4820, 0x1, 0x1, 0x12a05f200, 0x1ea79e0, 0xc42000ac00)
	/go/src/github.com/cockroachdb/cockroach/pkg/acceptance/util_test.go:345 +0x188
github.com/cockroachdb/cockroach/pkg/acceptance.runTestOnConfigs.func1(0xc4200940c0, 0xc4200e3ec8, 0x16a4c00)
	/go/src/github.com/cockroachdb/cockroach/pkg/acceptance/util_test.go:312 +0x4e
github.com/cockroachdb/cockroach/pkg/acceptance.runTestOnConfigs(0xc4200940c0, 0x16a4c00)
	/go/src/github.com/cockroachdb/cockroach/pkg/acceptance/util_test.go:315 +0xe2
github.com/cockroachdb/cockroach/pkg/acceptance.TestSingleKey(0xc4200940c0)
	/go/src/github.com/cockroachdb/cockroach/pkg/acceptance/single_key_test.go:38 +0x37
testing.tRunner(0xc4200940c0, 0x16a4b08)
	/usr/local/go/src/testing/testing.go:610 +0x81
created by testing.(*T).Run
	/usr/local/go/src/testing/testing.go:646 +0x2ec

goroutine 743 [IO wait, 7 minutes]:
net.runtime_pollWait(0x7f90cc8d9b40, 0x72, 0x1c)
	/usr/local/go/src/runtime/netpoll.go:160 +0x59
net.(*pollDesc).wait(0xc42020e290, 0x72, 0xc4203059d0, 0xc420014068)
	/usr/local/go/src/net/fd_poll_runtime.go:73 +0x38
net.(*pollDesc).waitRead(0xc42020e290, 0x1e94780, 0xc420014068)
	/usr/local/go/src/net/fd_poll_runtime.go:78 +0x34
net.(*netFD).Read(0xc42020e230, 0xc4201d7000, 0x1000, 0x1000, 0x0, 0x1e94780, 0xc420014068)
	/usr/local/go/src/net/fd_unix.go:243 +0x1a1
net.(*conn).Read(0xc4201343e0, 0xc4201d7000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/net.go:173 +0x70
net/http.(*persistConn).Read(0xc42041bd00, 0xc4201d7000, 0x1000, 0x1000, 0x30, 0xc420305b58, 0x60033c)
	/usr/local/go/src/net/http/transport.go:1261 +0x154
bufio.(*Reader).fill(0xc42059e8a0)
	/usr/local/go/src/bufio/bufio.go:97 +0x10c
bufio.(*Reader).Peek(0xc42059e8a0, 0x1, 0x0, 0x1, 0x0, 0xc420362240, 0x0)
	/usr/local/go/src/bufio/bufio.go:129 +0x62
net/http.(*persistConn).readLoop(0xc42041bd00)
	/usr/local/go/src/net/http/transport.go:1418 +0x1a1
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:1062 +0x4e9

goroutine 23 [select, 9 minutes, locked to thread]:
runtime.gopark(0x16a7c80, 0x0, 0x15549a5, 0x6, 0x18, 0x2)
	/usr/local/go/src/runtime/proc.go:259 +0x13a
runtime.selectgoImpl(0xc42002c730, 0x0, 0x18)
	/usr/local/go/src/runtime/select.go:423 +0x11d9
runtime.selectgo(0xc42002c730)
	/usr/local/go/src/runtime/select.go:238 +0x1c
runtime.ensureSigM.func1()
	/usr/local/go/src/runtime/signal1_unix.go:304 +0x2f3
runtime.goexit()
	/usr/local/go/src/runtime/asm_amd64.s:2086 +0x1

goroutine 10 [IO wait, 9 minutes]:
net.runtime_pollWait(0x7f90cda902b8, 0x72, 0x4)
	/usr/local/go/src/runtime/netpoll.go:160 +0x59
net.(*pollDesc).wait(0xc42046b790, 0x72, 0xc4200439d0, 0xc420014068)
	/usr/local/go/src/net/fd_poll_runtime.go:73 +0x38
net.(*pollDesc).waitRead(0xc42046b790, 0x1e94780, 0xc420014068)
	/usr/local/go/src/net/fd_poll_runtime.go:78 +0x34
net.(*netFD).Read(0xc42046b730, 0xc420486000, 0x1000, 0x1000, 0x0, 0x1e94780, 0xc420014068)
	/usr/local/go/src/net/fd_unix.go:243 +0x1a1
net.(*conn).Read(0xc4200320a8, 0xc420486000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/net.go:173 +0x70
net/http.(*persistConn).Read(0xc42000ae00, 0xc420486000, 0x1000, 0x1000, 0x30, 0xc420043b58, 0x60033c)
	/usr/local/go/src/net/http/transport.go:1261 +0x154
bufio.(*Reader).fill(0xc420074d20)
	/usr/local/go/src/bufio/bufio.go:97 +0x10c
bufio.(*Reader).Peek(0xc420074d20, 0x1, 0x0, 0x1, 0x0, 0xc4201444e0, 0x0)
	/usr/local/go/src/bufio/bufio.go:129 +0x62
net/http.(*persistConn).readLoop(0xc42000ae00)
	/usr/local/go/src/net/http/transport.go:1418 +0x1a1
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:1062 +0x4e9

goroutine 11 [select, 9 minutes]:
net/http.(*persistConn).writeLoop(0xc42000ae00)
	/usr/local/go/src/net/http/transport.go:1646 +0x3bd
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:1063 +0x50e

goroutine 598 [IO wait, 7 minutes]:
net.runtime_pollWait(0x7f90cc8d8a00, 0x72, 0x1a)
	/usr/local/go/src/runtime/netpoll.go:160 +0x59
net.(*pollDesc).wait(0xc420010610, 0x72, 0xc4204c99d0, 0xc420014068)
	/usr/local/go/src/net/fd_poll_runtime.go:73 +0x38
net.(*pollDesc).waitRead(0xc420010610, 0x1e94780, 0xc420014068)
	/usr/local/go/src/net/fd_poll_runtime.go:78 +0x34
net.(*netFD).Read(0xc4200105b0, 0xc42054c000, 0x1000, 0x1000, 0x0, 0x1e94780, 0xc420014068)
	/usr/local/go/src/net/fd_unix.go:243 +0x1a1
net.(*conn).Read(0xc4204f4040, 0xc42054c000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/net.go:173 +0x70
net/http.(*persistConn).Read(0xc42041b900, 0xc42054c000, 0x1000, 0x1000, 0x30, 0xc4204c9b58, 0x60033c)
	/usr/local/go/src/net/http/transport.go:1261 +0x154
bufio.(*Reader).fill(0xc420352a20)
	/usr/local/go/src/bufio/bufio.go:97 +0x10c
bufio.(*Reader).Peek(0xc420352a20, 0x1, 0x0, 0x1, 0x0, 0xc4201c0660, 0x0)
	/usr/local/go/src/bufio/bufio.go:129 +0x62
net/http.(*persistConn).readLoop(0xc42041b900)
	/usr/local/go/src/net/http/transport.go:1418 +0x1a1
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:1062 +0x4e9

goroutine 1049 [IO wait, 6 minutes]:
net.runtime_pollWait(0x7f90cc8d8b80, 0x72, 0x1f)
	/usr/local/go/src/runtime/netpoll.go:160 +0x59
net.(*pollDesc).wait(0xc4202ec530, 0x72, 0xc4205419d0, 0xc420014068)
	/usr/local/go/src/net/fd_poll_runtime.go:73 +0x38
net.(*pollDesc).waitRead(0xc4202ec530, 0x1e94780, 0xc420014068)
	/usr/local/go/src/net/fd_poll_runtime.go:78 +0x34
net.(*netFD).Read(0xc4202ec4d0, 0xc42031d000, 0x1000, 0x1000, 0x0, 0x1e94780, 0xc420014068)
	/usr/local/go/src/net/fd_unix.go:243 +0x1a1
net.(*conn).Read(0xc420134440, 0xc42031d000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/net.go:173 +0x70
net/http.(*persistConn).Read(0xc420072600, 0xc42031d000, 0x1000, 0x1000, 0x30, 0xc420541b58, 0x60033c)
	/usr/local/go/src/net/http/transport.go:1261 +0x154
bufio.(*Reader).fill(0xc420363380)
	/usr/local/go/src/bufio/bufio.go:97 +0x10c
bufio.(*Reader).Peek(0xc420363380, 0x1, 0x0, 0x1, 0x0, 0xc4201d8000, 0x0)
	/usr/local/go/src/bufio/bufio.go:129 +0x62
net/http.(*persistConn).readLoop(0xc420072600)
	/usr/local/go/src/net/http/transport.go:1418 +0x1a1
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:1062 +0x4e9

goroutine 471 [IO wait, 8 minutes]:
net.runtime_pollWait(0x7f90cda8f5f8, 0x72, 0x16)
	/usr/local/go/src/runtime/netpoll.go:160 +0x59
net.(*pollDesc).wait(0xc4202f6990, 0x72, 0xc4205909d0, 0xc420014068)
	/usr/local/go/src/net/fd_poll_runtime.go:73 +0x38
net.(*pollDesc).waitRead(0xc4202f6990, 0x1e94780, 0xc420014068)
	/usr/local/go/src/net/fd_poll_runtime.go:78 +0x34
net.(*netFD).Read(0xc4202f6930, 0xc4200f8000, 0x1000, 0x1000, 0x0, 0x1e94780, 0xc420014068)
	/usr/local/go/src/net/fd_unix.go:243 +0x1a1
net.(*conn).Read(0xc4204f40d8, 0xc4200f8000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/net.go:173 +0x70
net/http.(*persistConn).Read(0xc42000b200, 0xc4200f8000, 0x1000, 0x1000, 0x30, 0xc420590b58, 0x60033c)
	/usr/local/go/src/net/http/transport.go:1261 +0x154
bufio.(*Reader).fill(0xc4201c1440)
	/usr/local/go/src/bufio/bufio.go:97 +0x10c
bufio.(*Reader).Peek(0xc4201c1440, 0x1, 0x0, 0x1, 0x0, 0xc4200f64e0, 0x0)
	/usr/local/go/src/bufio/bufio.go:129 +0x62
net/http.(*persistConn).readLoop(0xc42000b200)
	/usr/local/go/src/net/http/transport.go:1418 +0x1a1
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:1062 +0x4e9

goroutine 417 [select, 8 minutes]:
net/http.(*persistConn).writeLoop(0xc42041bb00)
	/usr/local/go/src/net/http/transport.go:1646 +0x3bd
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:1063 +0x50e

goroutine 1020 [IO wait, 6 minutes]:
net.runtime_pollWait(0x7f90cc8d8400, 0x72, 0x22)
	/usr/local/go/src/runtime/netpoll.go:160 +0x59
net.(*pollDesc).wait(0xc420288450, 0x72, 0xc4205339d0, 0xc420014068)
	/usr/local/go/src/net/fd_poll_runtime.go:73 +0x38
net.(*pollDesc).waitRead(0xc420288450, 0x1e94780, 0xc420014068)
	/usr/local/go/src/net/fd_poll_runtime.go:78 +0x34
net.(*netFD).Read(0xc4202883f0, 0xc4201dc000, 0x1000, 0x1000, 0x0, 0x1e94780, 0xc420014068)
	/usr/local/go/src/net/fd_unix.go:243 +0x1a1
net.(*conn).Read(0xc42028e050, 0xc4201dc000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/net.go:173 +0x70
net/http.(*persistConn).Read(0xc420072700, 0xc4201dc000, 0x1000, 0x1000, 0x30, 0xc420533b58, 0x60033c)
	/usr/local/go/src/net/http/transport.go:1261 +0x154
bufio.(*Reader).fill(0xc420203020)
	/usr/local/go/src/bufio/bufio.go:97 +0x10c
bufio.(*Reader).Peek(0xc420203020, 0x1, 0x0, 0x1, 0x0, 0xc420074900, 0x0)
	/usr/local/go/src/bufio/bufio.go:129 +0x62
net/http.(*persistConn).readLoop(0xc420072700)
	/usr/local/go/src/net/http/transport.go:1418 +0x1a1
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:1062 +0x4e9

goroutine 88 [IO wait, 9 minutes]:
net.runtime_pollWait(0x7f90cda90138, 0x72, 0x6)
	/usr/local/go/src/runtime/netpoll.go:160 +0x59
net.(*pollDesc).wait(0xc4203ed3a0, 0x72, 0xc4200449d0, 0xc420014068)
	/usr/local/go/src/net/fd_poll_runtime.go:73 +0x38
net.(*pollDesc).waitRead(0xc4203ed3a0, 0x1e94780, 0xc420014068)
	/usr/local/go/src/net/fd_poll_runtime.go:78 +0x34
net.(*netFD).Read(0xc4203ed340, 0xc420266000, 0x1000, 0x1000, 0x0, 0x1e94780, 0xc420014068)
	/usr/local/go/src/net/fd_unix.go:243 +0x1a1
net.(*conn).Read(0xc4204f4058, 0xc420266000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/net.go:173 +0x70
net/http.(*persistConn).Read(0xc42041b800, 0xc420266000, 0x1000, 0x1000, 0x30, 0xc420044b58, 0x60033c)
	/usr/local/go/src/net/http/transport.go:1261 +0x154
bufio.(*Reader).fill(0xc42053ede0)
	/usr/local/go/src/bufio/bufio.go:97 +0x10c
bufio.(*Reader).Peek(0xc42053ede0, 0x1, 0x0, 0x1, 0x0, 0xc42053e4e0, 0x0)
	/usr/local/go/src/bufio/bufio.go:129 +0x62
net/http.(*persistConn).readLoop(0xc42041b800)
	/usr/local/go/src/net/http/transport.go:1418 +0x1a1
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:1062 +0x4e9

goroutine 1643 [IO wait]:
net.runtime_pollWait(0x7f90cc8d9600, 0x72, 0x36)
	/usr/local/go/src/runtime/netpoll.go:160 +0x59
net.(*pollDesc).wait(0xc420288f40, 0x72, 0xc420339828, 0xc420014068)
	/usr/local/go/src/net/fd_poll_runtime.go:73 +0x38
net.(*pollDesc).waitRead(0xc420288f40, 0x1e94780, 0xc420014068)
	/usr/local/go/src/net/fd_poll_runtime.go:78 +0x34
net.(*netFD).Read(0xc420288ee0, 0xc420476c00, 0x400, 0x400, 0x0, 0x1e94780, 0xc420014068)
	/usr/local/go/src/net/fd_unix.go:243 +0x1a1
net.(*conn).Read(0xc4200320c8, 0xc420476c00, 0x400, 0x400, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/net.go:173 +0x70
crypto/tls.(*block).readFromUntil(0xc4201bf710, 0x7f90cda927b8, 0xc4200320c8, 0x5, 0xc4200320c8, 0xc420316080)
	/usr/local/go/src/crypto/tls/conn.go:476 +0x91
crypto/tls.(*Conn).readRecord(0xc4203ad180, 0x16a7d17, 0xc4203ad288, 0xc420316590)
	/usr/local/go/src/crypto/tls/conn.go:578 +0xc4
crypto/tls.(*Conn).Read(0xc4203ad180, 0xc420330000, 0x8000, 0x8000, 0x0, 0x0, 0x0)
	/usr/local/go/src/crypto/tls/conn.go:1113 +0x116
bufio.(*Reader).fill(0xc420676180)
	/usr/local/go/src/bufio/bufio.go:97 +0x10c
bufio.(*Reader).Read(0xc420676180, 0xc4200946f8, 0x9, 0x9, 0xc420339ca8, 0x619eb0, 0xc42039b7c0)
	/usr/local/go/src/bufio/bufio.go:209 +0x1bc
io.ReadAtLeast(0x1e8ce40, 0xc420676180, 0xc4200946f8, 0x9, 0x9, 0x9, 0xc4201beca0, 0xc4201bec98, 0x1e90bc0)
	/usr/local/go/src/io/io.go:307 +0xa4
io.ReadFull(0x1e8ce40, 0xc420676180, 0xc4200946f8, 0x9, 0x9, 0xc420339eb0, 0xedcd00, 0xc4200db950)
	/usr/local/go/src/io/io.go:325 +0x58
golang.org/x/net/http2.readFrameHeader(0xc4200946f8, 0x9, 0x9, 0x1e8ce40, 0xc420676180, 0x0, 0xc400000000, 0xc420316080, 0xc)
	/go/src/golang.org/x/net/http2/frame.go:237 +0x7b
golang.org/x/net/http2.(*Framer).ReadFrame(0xc4200946c0, 0x0, 0x0, 0x0, 0x0)
	/go/src/golang.org/x/net/http2/frame.go:464 +0xa4
google.golang.org/grpc/transport.(*framer).readFrame(0xc42022e360, 0xc4204eee70, 0xc4204eee70, 0x0, 0x0)
	/go/src/google.golang.org/grpc/transport/http_util.go:505 +0x2f
google.golang.org/grpc/transport.(*http2Client).reader(0xc42000b300)
	/go/src/google.golang.org/grpc/transport/http2_client.go:921 +0xb6
created by google.golang.org/grpc/transport.newHTTP2Client
	/go/src/google.golang.org/grpc/transport/http2_client.go:203 +0x8b1

goroutine 223 [chan receive, 9 minutes]:
database/sql.(*DB).connectionOpener(0xc4203ae4d0)
	/usr/local/go/src/database/sql/sql.go:730 +0x4a
created by database/sql.Open
	/usr/local/go/src/database/sql/sql.go:493 +0x1e9

goroutine 261 [IO wait, 9 minutes]:
net.runtime_pollWait(0x7f90cda8fcb8, 0x72, 0xd)
	/usr/local/go/src/runtime/netpoll.go:160 +0x59
net.(*pollDesc).wait(0xc42027aa00, 0x72, 0xc42004e9d0, 0xc420014068)
	/usr/local/go/src/net/fd_poll_runtime.go:73 +0x38
net.(*pollDesc).waitRead(0xc42027aa00, 0x1e94780, 0xc420014068)
	/usr/local/go/src/net/fd_poll_runtime.go:78 +0x34
net.(*netFD).Read(0xc42027a9a0, 0xc42054d000, 0x1000, 0x1000, 0x0, 0x1e94780, 0xc420014068)
	/usr/local/go/src/net/fd_unix.go:243 +0x1a1
net.(*conn).Read(0xc420032110, 0xc42054d000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/net.go:173 +0x70
net/http.(*persistConn).Read(0xc420137800, 0xc42054d000, 0x1000, 0x1000, 0x30, 0xc42004eb58, 0x60033c)
	/usr/local/go/src/net/http/transport.go:1261 +0x154
bufio.(*Reader).fill(0xc420537560)
	/usr/local/go/src/bufio/bufio.go:97 +0x10c
bufio.(*Reader).Peek(0xc420537560, 0x1, 0x0, 0x1, 0x0, 0xc420362cc0, 0x0)
	/usr/local/go/src/bufio/bufio.go:129 +0x62
net/http.(*persistConn).readLoop(0xc420137800)
	/usr/local/go/src/net/http/transport.go:1418 +0x1a1
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:1062 +0x4e9

goroutine 524 [select, 8 minutes]:
net/http.(*persistConn).writeLoop(0xc420136700)
	/usr/local/go/src/net/http/transport.go:1646 +0x3bd
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:1063 +0x50e

goroutine 368 [select, 8 minutes]:
net/http.(*persistConn).writeLoop(0xc42000b900)
	/usr/local/go/src/net/http/transport.go:1646 +0x3bd
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:1063 +0x50e

goroutine 89 [select, 9 minutes]:
net/http.(*persistConn).writeLoop(0xc42041b800)
	/usr/local/go/src/net/http/transport.go:1646 +0x3bd
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:1063 +0x50e

goroutine 367 [IO wait, 8 minutes]:
net.runtime_pollWait(0x7f90cda8f9b8, 0x72, 0x11)
	/usr/local/go/src/runtime/netpoll.go:160 +0x59
net.(*pollDesc).wait(0xc42031eb50, 0x72, 0xc4203089d0, 0xc420014068)
	/usr/local/go/src/net/fd_poll_runtime.go:73 +0x38
net.(*pollDesc).waitRead(0xc42031eb50, 0x1e94780, 0xc420014068)
	/usr/local/go/src/net/fd_poll_runtime.go:78 +0x34
net.(*netFD).Read(0xc42031eaf0, 0xc4205b2000, 0x1000, 0x1000, 0x0, 0x1e94780, 0xc420014068)
	/usr/local/go/src/net/fd_unix.go:243 +0x1a1
net.(*conn).Read(0xc4204f4248, 0xc4205b2000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/net.go:173 +0x70
net/http.(*persistConn).Read(0xc42000b900, 0xc4205b2000, 0x1000, 0x1000, 0x30, 0xc420308b58, 0x60033c)
	/usr/local/go/src/net/http/transport.go:1261 +0x154
bufio.(*Reader).fill(0xc42059e900)
	/usr/local/go/src/bufio/bufio.go:97 +0x10c
bufio.(*Reader).Peek(0xc42059e900, 0x1, 0x0, 0x1, 0x0, 0xc4203b0840, 0x0)
	/usr/local/go/src/bufio/bufio.go:129 +0x62
net/http.(*persistConn).readLoop(0xc42000b900)
	/usr/local/go/src/net/http/transport.go:1418 +0x1a1
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:1062 +0x4e9

goroutine 255 [IO wait, 9 minutes]:
net.runtime_pollWait(0x7f90cda901f8, 0x72, 0xa)
	/usr/local/go/src/runtime/netpoll.go:160 +0x59
net.(*pollDesc).wait(0xc4204fa450, 0x72, 0xc42003f9d0, 0xc420014068)
	/usr/local/go/src/net/fd_poll_runtime.go:73 +0x38
net.(*pollDesc).waitRead(0xc4204fa450, 0x1e94780, 0xc420014068)
	/usr/local/go/src/net/fd_poll_runtime.go:78 +0x34
net.(*netFD).Read(0xc4204fa3f0, 0xc4205e6000, 0x1000, 0x1000, 0x0, 0x1e94780, 0xc420014068)
	/usr/local/go/src/net/fd_unix.go:243 +0x1a1
net.(*conn).Read(0xc4200320b8, 0xc4205e6000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/net.go:173 +0x70
net/http.(*persistConn).Read(0xc420137400, 0xc4205e6000, 0x1000, 0x1000, 0x30, 0xc42003fb58, 0x60033c)
	/usr/local/go/src/net/http/transport.go:1261 +0x154
bufio.(*Reader).fill(0xc4200f67e0)
	/usr/local/go/src/bufio/bufio.go:97 +0x10c
bufio.(*Reader).Peek(0xc4200f67e0, 0x1, 0x0, 0x1, 0x0, 0xc4203629c0, 0x0)
	/usr/local/go/src/bufio/bufio.go:129 +0x62
net/http.(*persistConn).readLoop(0xc420137400)
	/usr/local/go/src/net/http/transport.go:1418 +0x1a1
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:1062 +0x4e9

goroutine 1073 [IO wait, 6 minutes]:
net.runtime_pollWait(0x7f90cc8d81c0, 0x72, 0x20)
	/usr/local/go/src/runtime/netpoll.go:160 +0x59
net.(*pollDesc).wait(0xc4202ec220, 0x72, 0xc4205449d0, 0xc420014068)
	/usr/local/go/src/net/fd_poll_runtime.go:73 +0x38
net.(*pollDesc).waitRead(0xc4202ec220, 0x1e94780, 0xc420014068)
	/usr/local/go/src/net/fd_poll_runtime.go:78 +0x34
net.(*netFD).Read(0xc4202ec1c0, 0xc420313000, 0x1000, 0x1000, 0x0, 0x1e94780, 0xc420014068)
	/usr/local/go/src/net/fd_unix.go:243 +0x1a1
net.(*conn).Read(0xc4204f4090, 0xc420313000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/net.go:173 +0x70
net/http.(*persistConn).Read(0xc420358000, 0xc420313000, 0x1000, 0x1000, 0x30, 0xc420544b58, 0x60033c)
	/usr/local/go/src/net/http/transport.go:1261 +0x154
bufio.(*Reader).fill(0xc420362900)
	/usr/local/go/src/bufio/bufio.go:97 +0x10c
bufio.(*Reader).Peek(0xc420362900, 0x1, 0x0, 0x1, 0x0, 0xc4201d8060, 0x0)
	/usr/local/go/src/bufio/bufio.go:129 +0x62
net/http.(*persistConn).readLoop(0xc420358000)
	/usr/local/go/src/net/http/transport.go:1418 +0x1a1
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:1062 +0x4e9

goroutine 416 [IO wait, 8 minutes]:
net.runtime_pollWait(0x7f90cda8f778, 0x72, 0x14)
	/usr/local/go/src/runtime/netpoll.go:160 +0x59
net.(*pollDesc).wait(0xc4203846f0, 0x72, 0xc4203029d0, 0xc420014068)
	/usr/local/go/src/net/fd_poll_runtime.go:73 +0x38
net.(*pollDesc).waitRead(0xc4203846f0, 0x1e94780, 0xc420014068)
	/usr/local/go/src/net/fd_poll_runtime.go:78 +0x34
net.(*netFD).Read(0xc420384690, 0xc420138000, 0x1000, 0x1000, 0x0, 0x1e94780, 0xc420014068)
	/usr/local/go/src/net/fd_unix.go:243 +0x1a1
net.(*conn).Read(0xc420134468, 0xc420138000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/net.go:173 +0x70
net/http.(*persistConn).Read(0xc42041bb00, 0xc420138000, 0x1000, 0x1000, 0x30, 0xc420302b58, 0x60033c)
	/usr/local/go/src/net/http/transport.go:1261 +0x154
bufio.(*Reader).fill(0xc420352420)
	/usr/local/go/src/bufio/bufio.go:97 +0x10c
bufio.(*Reader).Peek(0xc420352420, 0x1, 0x0, 0x1, 0x0, 0xc42023ca20, 0x0)
	/usr/local/go/src/bufio/bufio.go:129 +0x62
net/http.(*persistConn).readLoop(0xc42041bb00)
	/usr/local/go/src/net/http/transport.go:1418 +0x1a1
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:1062 +0x4e9

goroutine 340 [select, 9 minutes]:
net/http.(*persistConn).writeLoop(0xc420137700)
	/usr/local/go/src/net/http/transport.go:1646 +0x3bd
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:1063 +0x50e

goroutine 81 [select, 9 minutes]:
net/http.(*persistConn).writeLoop(0xc42000af00)
	/usr/local/go/src/net/http/transport.go:1646 +0x3bd
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:1063 +0x50e

goroutine 1074 [select, 6 minutes]:
net/http.(*persistConn).writeLoop(0xc420358000)
	/usr/local/go/src/net/http/transport.go:1646 +0x3bd
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:1063 +0x50e

goroutine 339 [IO wait, 9 minutes]:
net.runtime_pollWait(0x7f90cda8fa78, 0x72, 0x10)
	/usr/local/go/src/runtime/netpoll.go:160 +0x59
net.(*pollDesc).wait(0xc4202ed170, 0x72, 0xc4203099d0, 0xc420014068)
	/usr/local/go/src/net/fd_poll_runtime.go:73 +0x38
net.(*pollDesc).waitRead(0xc4202ed170, 0x1e94780, 0xc420014068)
	/usr/local/go/src/net/fd_poll_runtime.go:78 +0x34
net.(*netFD).Read(0xc4202ed110, 0xc4202ea000, 0x1000, 0x1000, 0x0, 0x1e94780, 0xc420014068)
	/usr/local/go/src/net/fd_unix.go:243 +0x1a1
net.(*conn).Read(0xc42028e0f0, 0xc4202ea000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/net.go:173 +0x70
net/http.(*persistConn).Read(0xc420137700, 0xc4202ea000, 0x1000, 0x1000, 0x30, 0xc420309b58, 0x60033c)
	/usr/local/go/src/net/http/transport.go:1261 +0x154
bufio.(*Reader).fill(0xc4203639e0)
	/usr/local/go/src/bufio/bufio.go:97 +0x10c
bufio.(*Reader).Peek(0xc4203639e0, 0x1, 0x0, 0x1, 0x0, 0xc4202e55c0, 0x0)
	/usr/local/go/src/bufio/bufio.go:129 +0x62
net/http.(*persistConn).readLoop(0xc420137700)
	/usr/local/go/src/net/http/transport.go:1418 +0x1a1
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:1062 +0x4e9

goroutine 744 [select, 7 minutes]:
net/http.(*persistConn).writeLoop(0xc42041bd00)
	/usr/local/go/src/net/http/transport.go:1646 +0x3bd
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:1063 +0x50e

goroutine 1641 [chan receive, 4 minutes]:
github.com/cockroachdb/cockroach/pkg/rpc.NewContext.func1()
	/go/src/github.com/cockroachdb/cockroach/pkg/rpc/context.go:138 +0x76
github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker.func1(0xc42046d680, 0xc4205d2160)
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:196 +0x7d
created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:197 +0x66

goroutine 80 [IO wait, 9 minutes]:
net.runtime_pollWait(0x7f90cda90078, 0x72, 0x8)
	/usr/local/go/src/runtime/netpoll.go:160 +0x59
net.(*pollDesc).wait(0xc4204fa140, 0x72, 0xc4200559d0, 0xc420014068)
	/usr/local/go/src/net/fd_poll_runtime.go:73 +0x38
net.(*pollDesc).waitRead(0xc4204fa140, 0x1e94780, 0xc420014068)
	/usr/local/go/src/net/fd_poll_runtime.go:78 +0x34
net.(*netFD).Read(0xc4204fa0e0, 0xc420398000, 0x1000, 0x1000, 0x0, 0x1e94780, 0xc420014068)
	/usr/local/go/src/net/fd_unix.go:243 +0x1a1
net.(*conn).Read(0xc42028e048, 0xc420398000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/net.go:173 +0x70
net/http.(*persistConn).Read(0xc42000af00, 0xc420398000, 0x1000, 0x1000, 0x30, 0xc420055b58, 0x60033c)
	/usr/local/go/src/net/http/transport.go:1261 +0x154
bufio.(*Reader).fill(0xc4205d8540)
	/usr/local/go/src/bufio/bufio.go:97 +0x10c
bufio.(*Reader).Peek(0xc4205d8540, 0x1, 0x0, 0x1, 0x0, 0xc4203a0000, 0x0)
	/usr/local/go/src/bufio/bufio.go:129 +0x62
net/http.(*persistConn).readLoop(0xc42000af00)
	/usr/local/go/src/net/http/transport.go:1418 +0x1a1
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:1062 +0x4e9

goroutine 125 [select, 9 minutes]:
net/http.(*persistConn).writeLoop(0xc420137300)
	/usr/local/go/src/net/http/transport.go:1646 +0x3bd
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:1063 +0x50e

goroutine 1050 [select, 6 minutes]:
net/http.(*persistConn).writeLoop(0xc420072600)
	/usr/local/go/src/net/http/transport.go:1646 +0x3bd
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:1063 +0x50e

goroutine 767 [IO wait, 7 minutes]:
net.runtime_pollWait(0x7f90cc8d8940, 0x72, 0x1d)
	/usr/local/go/src/runtime/netpoll.go:160 +0x59
net.(*pollDesc).wait(0xc4202ec3e0, 0x72, 0xc4205409d0, 0xc420014068)
	/usr/local/go/src/net/fd_poll_runtime.go:73 +0x38
net.(*pollDesc).waitRead(0xc4202ec3e0, 0x1e94780, 0xc420014068)
	/usr/local/go/src/net/fd_poll_runtime.go:78 +0x34
net.(*netFD).Read(0xc4202ec380, 0xc420512000, 0x1000, 0x1000, 0x0, 0x1e94780, 0xc420014068)
	/usr/local/go/src/net/fd_unix.go:243 +0x1a1
net.(*conn).Read(0xc42028e030, 0xc420512000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/net.go:173 +0x70
net/http.(*persistConn).Read(0xc42041bf00, 0xc420512000, 0x1000, 0x1000, 0x30, 0xc420540b58, 0x60033c)
	/usr/local/go/src/net/http/transport.go:1261 +0x154
bufio.(*Reader).fill(0xc4202027e0)
	/usr/local/go/src/bufio/bufio.go:97 +0x10c
bufio.(*Reader).Peek(0xc4202027e0, 0x1, 0x0, 0x1, 0x0, 0xc420362000, 0x0)
	/usr/local/go/src/bufio/bufio.go:129 +0x62
net/http.(*persistConn).readLoop(0xc42041bf00)
	/usr/local/go/src/net/http/transport.go:1418 +0x1a1
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:1062 +0x4e9

goroutine 124 [IO wait, 9 minutes]:
net.runtime_pollWait(0x7f90cda8fef8, 0x72, 0x7)
	/usr/local/go/src/runtime/netpoll.go:160 +0x59
net.(*pollDesc).wait(0xc42046a450, 0x72, 0xc4200549d0, 0xc420014068)
	/usr/local/go/src/net/fd_poll_runtime.go:73 +0x38
net.(*pollDesc).waitRead(0xc42046a450, 0x1e94780, 0xc420014068)
	/usr/local/go/src/net/fd_poll_runtime.go:78 +0x34
net.(*netFD).Read(0xc42046a3f0, 0xc420284000, 0x1000, 0x1000, 0x0, 0x1e94780, 0xc420014068)
	/usr/local/go/src/net/fd_unix.go:243 +0x1a1
net.(*conn).Read(0xc420032098, 0xc420284000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/net.go:173 +0x70
net/http.(*persistConn).Read(0xc420137300, 0xc420284000, 0x1000, 0x1000, 0x30, 0xc420054b58, 0x60033c)
	/usr/local/go/src/net/http/transport.go:1261 +0x154
bufio.(*Reader).fill(0xc4200fa8a0)
	/usr/local/go/src/bufio/bufio.go:97 +0x10c
bufio.(*Reader).Peek(0xc4200fa8a0, 0x1, 0x0, 0x1, 0x0, 0xc4203a0000, 0x0)
	/usr/local/go/src/bufio/bufio.go:129 +0x62
net/http.(*persistConn).readLoop(0xc420137300)
	/usr/local/go/src/net/http/transport.go:1418 +0x1a1
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:1062 +0x4e9

goroutine 272 [select, 9 minutes]:
net/http.(*persistConn).writeLoop(0xc42041ba00)
	/usr/local/go/src/net/http/transport.go:1646 +0x3bd
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:1063 +0x50e

goroutine 256 [select, 9 minutes]:
net/http.(*persistConn).writeLoop(0xc420137400)
	/usr/local/go/src/net/http/transport.go:1646 +0x3bd
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:1063 +0x50e

goroutine 271 [IO wait, 9 minutes]:
net.runtime_pollWait(0x7f90cda8fb38, 0x72, 0xf)
	/usr/local/go/src/runtime/netpoll.go:160 +0x59
net.(*pollDesc).wait(0xc4205b4ca0, 0x72, 0xc4203079d0, 0xc420014068)
	/usr/local/go/src/net/fd_poll_runtime.go:73 +0x38
net.(*pollDesc).waitRead(0xc4205b4ca0, 0x1e94780, 0xc420014068)
	/usr/local/go/src/net/fd_poll_runtime.go:78 +0x34
net.(*netFD).Read(0xc4205b4c40, 0xc42022a000, 0x1000, 0x1000, 0x0, 0x1e94780, 0xc420014068)
	/usr/local/go/src/net/fd_unix.go:243 +0x1a1
net.(*conn).Read(0xc4204f40e0, 0xc42022a000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/net.go:173 +0x70
net/http.(*persistConn).Read(0xc42041ba00, 0xc42022a000, 0x1000, 0x1000, 0x30, 0xc420307b58, 0x60033c)
	/usr/local/go/src/net/http/transport.go:1261 +0x154
bufio.(*Reader).fill(0xc420353500)
	/usr/local/go/src/bufio/bufio.go:97 +0x10c
bufio.(*Reader).Peek(0xc420353500, 0x1, 0x0, 0x1, 0x0, 0xc420363320, 0x0)
	/usr/local/go/src/bufio/bufio.go:129 +0x62
net/http.(*persistConn).readLoop(0xc42041ba00)
	/usr/local/go/src/net/http/transport.go:1418 +0x1a1
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:1062 +0x4e9

goroutine 188 [chan receive, 9 minutes]:
database/sql.(*DB).connectionOpener(0xc4204e26e0)
	/usr/local/go/src/database/sql/sql.go:730 +0x4a
created by database/sql.Open
	/usr/local/go/src/database/sql/sql.go:493 +0x1e9

goroutine 208 [IO wait, 9 minutes]:
net.runtime_pollWait(0x7f90cda8ffb8, 0x72, 0x9)
	/usr/local/go/src/runtime/netpoll.go:160 +0x59
net.(*pollDesc).wait(0xc42046a530, 0x72, 0xc4200459d0, 0xc420014068)
	/usr/local/go/src/net/fd_poll_runtime.go:73 +0x38
net.(*pollDesc).waitRead(0xc42046a530, 0x1e94780, 0xc420014068)
	/usr/local/go/src/net/fd_poll_runtime.go:78 +0x34
net.(*netFD).Read(0xc42046a4d0, 0xc420560000, 0x1000, 0x1000, 0x0, 0x1e94780, 0xc420014068)
	/usr/local/go/src/net/fd_unix.go:243 +0x1a1
net.(*conn).Read(0xc4200320b0, 0xc420560000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/net.go:173 +0x70
net/http.(*persistConn).Read(0xc42000ad00, 0xc420560000, 0x1000, 0x1000, 0x30, 0xc420045b58, 0x60033c)
	/usr/local/go/src/net/http/transport.go:1261 +0x154
bufio.(*Reader).fill(0xc4200fb020)
	/usr/local/go/src/bufio/bufio.go:97 +0x10c
bufio.(*Reader).Peek(0xc4200fb020, 0x1, 0x0, 0x1, 0x0, 0xc4200f6000, 0x0)
	/usr/local/go/src/bufio/bufio.go:129 +0x62
net/http.(*persistConn).readLoop(0xc42000ad00)
	/usr/local/go/src/net/http/transport.go:1418 +0x1a1
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:1062 +0x4e9

goroutine 209 [select, 9 minutes]:
net/http.(*persistConn).writeLoop(0xc42000ad00)
	/usr/local/go/src/net/http/transport.go:1646 +0x3bd
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:1063 +0x50e

goroutine 262 [select, 9 minutes]:
net/http.(*persistConn).writeLoop(0xc420137800)
	/usr/local/go/src/net/http/transport.go:1646 +0x3bd
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:1063 +0x50e

goroutine 235 [IO wait, 9 minutes]:
net.runtime_pollWait(0x7f90cda8fbf8, 0x72, 0xe)
	/usr/local/go/src/runtime/netpoll.go:160 +0x59
net.(*pollDesc).wait(0xc42014cc30, 0x72, 0xc42036a9d0, 0xc420014068)
	/usr/local/go/src/net/fd_poll_runtime.go:73 +0x38
net.(*pollDesc).waitRead(0xc42014cc30, 0x1e94780, 0xc420014068)
	/usr/local/go/src/net/fd_poll_runtime.go:78 +0x34
net.(*netFD).Read(0xc42014cbd0, 0xc4205aa000, 0x1000, 0x1000, 0x0, 0x1e94780, 0xc420014068)
	/usr/local/go/src/net/fd_unix.go:243 +0x1a1
net.(*conn).Read(0xc420032188, 0xc4205aa000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/net.go:173 +0x70
net/http.(*persistConn).Read(0xc42000b100, 0xc4205aa000, 0x1000, 0x1000, 0x30, 0xc42036ab58, 0x60033c)
	/usr/local/go/src/net/http/transport.go:1261 +0x154
bufio.(*Reader).fill(0xc4205d7260)
	/usr/local/go/src/bufio/bufio.go:97 +0x10c
bufio.(*Reader).Peek(0xc4205d7260, 0x1, 0x0, 0x1, 0x0, 0xc4202e8360, 0x0)
	/usr/local/go/src/bufio/bufio.go:129 +0x62
net/http.(*persistConn).readLoop(0xc42000b100)
	/usr/local/go/src/net/http/transport.go:1418 +0x1a1
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:1062 +0x4e9

goroutine 656 [select, 7 minutes]:
net/http.(*persistConn).writeLoop(0xc420072300)
	/usr/local/go/src/net/http/transport.go:1646 +0x3bd
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:1063 +0x50e

goroutine 236 [select, 9 minutes]:
net/http.(*persistConn).writeLoop(0xc42000b100)
	/usr/local/go/src/net/http/transport.go:1646 +0x3bd
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:1063 +0x50e

goroutine 351 [IO wait, 8 minutes]:
net.runtime_pollWait(0x7f90cda8f8f8, 0x72, 0x12)
	/usr/local/go/src/runtime/netpoll.go:160 +0x59
net.(*pollDesc).wait(0xc42020e0d0, 0x72, 0xc42036b9d0, 0xc420014068)
	/usr/local/go/src/net/fd_poll_runtime.go:73 +0x38
net.(*pollDesc).waitRead(0xc42020e0d0, 0x1e94780, 0xc420014068)
	/usr/local/go/src/net/fd_poll_runtime.go:78 +0x34
net.(*netFD).Read(0xc42020e070, 0xc420240000, 0x1000, 0x1000, 0x0, 0x1e94780, 0xc420014068)
	/usr/local/go/src/net/fd_unix.go:243 +0x1a1
net.(*conn).Read(0xc4201344c8, 0xc420240000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/net.go:173 +0x70
net/http.(*persistConn).Read(0xc42000b600, 0xc420240000, 0x1000, 0x1000, 0x30, 0xc42036bb58, 0x60033c)
	/usr/local/go/src/net/http/transport.go:1261 +0x154
bufio.(*Reader).fill(0xc4200e69c0)
	/usr/local/go/src/bufio/bufio.go:97 +0x10c
bufio.(*Reader).Peek(0xc4200e69c0, 0x1, 0x0, 0x1, 0x0, 0xc4205070e0, 0x0)
	/usr/local/go/src/bufio/bufio.go:129 +0x62
net/http.(*persistConn).readLoop(0xc42000b600)
	/usr/local/go/src/net/http/transport.go:1418 +0x1a1
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:1062 +0x4e9

goroutine 408 [IO wait, 8 minutes]:
net.runtime_pollWait(0x7f90cda8f838, 0x72, 0x13)
	/usr/local/go/src/runtime/netpoll.go:160 +0x59
net.(*pollDesc).wait(0xc42021c6f0, 0x72, 0xc42058b9d0, 0xc420014068)
	/usr/local/go/src/net/fd_poll_runtime.go:73 +0x38
net.(*pollDesc).waitRead(0xc42021c6f0, 0x1e94780, 0xc420014068)
	/usr/local/go/src/net/fd_poll_runtime.go:78 +0x34
net.(*netFD).Read(0xc42021c690, 0xc4202c6000, 0x1000, 0x1000, 0x0, 0x1e94780, 0xc420014068)
	/usr/local/go/src/net/fd_unix.go:243 +0x1a1
net.(*conn).Read(0xc4204f4088, 0xc4202c6000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/net.go:173 +0x70
net/http.(*persistConn).Read(0xc420137900, 0xc4202c6000, 0x1000, 0x1000, 0x30, 0xc42058bb58, 0x60033c)
	/usr/local/go/src/net/http/transport.go:1261 +0x154
bufio.(*Reader).fill(0xc42053efc0)
	/usr/local/go/src/bufio/bufio.go:97 +0x10c
bufio.(*Reader).Peek(0xc42053efc0, 0x1, 0x0, 0x1, 0x0, 0xc4200f4fc0, 0x0)
	/usr/local/go/src/bufio/bufio.go:129 +0x62
net/http.(*persistConn).readLoop(0xc420137900)
	/usr/local/go/src/net/http/transport.go:1418 +0x1a1
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:1062 +0x4e9

goroutine 352 [select, 8 minutes]:
net/http.(*persistConn).writeLoop(0xc42000b600)
	/usr/local/go/src/net/http/transport.go:1646 +0x3bd
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:1063 +0x50e

goroutine 409 [select, 8 minutes]:
net/http.(*persistConn).writeLoop(0xc420137900)
	/usr/local/go/src/net/http/transport.go:1646 +0x3bd
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:1063 +0x50e

goroutine 426 [IO wait, 8 minutes]:
net.runtime_pollWait(0x7f90cda8f6b8, 0x72, 0x15)
	/usr/local/go/src/runtime/netpoll.go:160 +0x59
net.(*pollDesc).wait(0xc42037ca00, 0x72, 0xc4203049d0, 0xc420014068)
	/usr/local/go/src/net/fd_poll_runtime.go:73 +0x38
net.(*pollDesc).waitRead(0xc42037ca00, 0x1e94780, 0xc420014068)
	/usr/local/go/src/net/fd_poll_runtime.go:78 +0x34
net.(*netFD).Read(0xc42037c9a0, 0xc420098000, 0x1000, 0x1000, 0x0, 0x1e94780, 0xc420014068)
	/usr/local/go/src/net/fd_unix.go:243 +0x1a1
net.(*conn).Read(0xc420134428, 0xc420098000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/net.go:173 +0x70
net/http.(*persistConn).Read(0xc42041bc00, 0xc420098000, 0x1000, 0x1000, 0x30, 0xc420304b58, 0x60033c)
	/usr/local/go/src/net/http/transport.go:1261 +0x154
bufio.(*Reader).fill(0xc4201c13e0)
	/usr/local/go/src/bufio/bufio.go:97 +0x10c
bufio.(*Reader).Peek(0xc4201c13e0, 0x1, 0x0, 0x1, 0x0, 0xc420362f60, 0x0)
	/usr/local/go/src/bufio/bufio.go:129 +0x62
net/http.(*persistConn).readLoop(0xc42041bc00)
	/usr/local/go/src/net/http/transport.go:1418 +0x1a1
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:1062 +0x4e9

goroutine 528 [select, 8 minutes]:
net/http.(*persistConn).writeLoop(0xc420137b00)
	/usr/local/go/src/net/http/transport.go:1646 +0x3bd
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:1063 +0x50e

goroutine 427 [select, 8 minutes]:
net/http.(*persistConn).writeLoop(0xc42041bc00)
	/usr/local/go/src/net/http/transport.go:1646 +0x3bd
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:1063 +0x50e

goroutine 472 [select, 8 minutes]:
net/http.(*persistConn).writeLoop(0xc42000b200)
	/usr/local/go/src/net/http/transport.go:1646 +0x3bd
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:1063 +0x50e

goroutine 527 [IO wait, 8 minutes]:
net.runtime_pollWait(0x7f90cda8f478, 0x72, 0x17)
	/usr/local/go/src/runtime/netpoll.go:160 +0x59
net.(*pollDesc).wait(0xc42031e6f0, 0x72, 0xc4204ce9d0, 0xc420014068)
	/usr/local/go/src/net/fd_poll_runtime.go:73 +0x38
net.(*pollDesc).waitRead(0xc42031e6f0, 0x1e94780, 0xc420014068)
	/usr/local/go/src/net/fd_poll_runtime.go:78 +0x34
net.(*netFD).Read(0xc42031e690, 0xc4202ff000, 0x1000, 0x1000, 0x0, 0x1e94780, 0xc420014068)
	/usr/local/go/src/net/fd_unix.go:243 +0x1a1
net.(*conn).Read(0xc42028e0b0, 0xc4202ff000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/net.go:173 +0x70
net/http.(*persistConn).Read(0xc420137b00, 0xc4202ff000, 0x1000, 0x1000, 0x30, 0xc4204ceb58, 0x60033c)
	/usr/local/go/src/net/http/transport.go:1261 +0x154
bufio.(*Reader).fill(0xc4201c1bc0)
	/usr/local/go/src/bufio/bufio.go:97 +0x10c
bufio.(*Reader).Peek(0xc4201c1bc0, 0x1, 0x0, 0x1, 0x0, 0xc4205cacc0, 0x0)
	/usr/local/go/src/bufio/bufio.go:129 +0x62
net/http.(*persistConn).readLoop(0xc420137b00)
	/usr/local/go/src/net/http/transport.go:1418 +0x1a1
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:1062 +0x4e9

goroutine 495 [IO wait, 7 minutes]:
net.runtime_pollWait(0x7f90cda8f538, 0x72, 0x18)
	/usr/local/go/src/runtime/netpoll.go:160 +0x59
net.(*pollDesc).wait(0xc4203edb80, 0x72, 0xc42058d9d0, 0xc420014068)
	/usr/local/go/src/net/fd_poll_runtime.go:73 +0x38
net.(*pollDesc).waitRead(0xc4203edb80, 0x1e94780, 0xc420014068)
	/usr/local/go/src/net/fd_poll_runtime.go:78 +0x34
net.(*netFD).Read(0xc4203edb20, 0xc420548000, 0x1000, 0x1000, 0x0, 0x1e94780, 0xc420014068)
	/usr/local/go/src/net/fd_unix.go:243 +0x1a1
net.(*conn).Read(0xc420134460, 0xc420548000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/net.go:173 +0x70
net/http.(*persistConn).Read(0xc420137d00, 0xc420548000, 0x1000, 0x1000, 0x30, 0xc42058db58, 0x60033c)
	/usr/local/go/src/net/http/transport.go:1261 +0x154
bufio.(*Reader).fill(0xc42059e120)
	/usr/local/go/src/bufio/bufio.go:97 +0x10c
bufio.(*Reader).Peek(0xc42059e120, 0x1, 0x0, 0x1, 0x0, 0xc4201c00c0, 0x0)
	/usr/local/go/src/bufio/bufio.go:129 +0x62
net/http.(*persistConn).readLoop(0xc420137d00)
	/usr/local/go/src/net/http/transport.go:1418 +0x1a1
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:1062 +0x4e9

goroutine 523 [IO wait, 8 minutes]:
net.runtime_pollWait(0x7f90cda8f3b8, 0x72, 0x19)
	/usr/local/go/src/runtime/netpoll.go:160 +0x59
net.(*pollDesc).wait(0xc42031e290, 0x72, 0xc4203659d0, 0xc420014068)
	/usr/local/go/src/net/fd_poll_runtime.go:73 +0x38
net.(*pollDesc).waitRead(0xc42031e290, 0x1e94780, 0xc420014068)
	/usr/local/go/src/net/fd_poll_runtime.go:78 +0x34
net.(*netFD).Read(0xc42031e230, 0xc420445000, 0x1000, 0x1000, 0x0, 0x1e94780, 0xc420014068)
	/usr/local/go/src/net/fd_unix.go:243 +0x1a1
net.(*conn).Read(0xc42028e078, 0xc420445000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/net.go:173 +0x70
net/http.(*persistConn).Read(0xc420136700, 0xc420445000, 0x1000, 0x1000, 0x30, 0xc420365b58, 0x60033c)
	/usr/local/go/src/net/http/transport.go:1261 +0x154
bufio.(*Reader).fill(0xc4201c0a80)
	/usr/local/go/src/bufio/bufio.go:97 +0x10c
bufio.(*Reader).Peek(0xc4201c0a80, 0x1, 0x0, 0x1, 0x0, 0xc420362fc0, 0x0)
	/usr/local/go/src/bufio/bufio.go:129 +0x62
net/http.(*persistConn).readLoop(0xc420136700)
	/usr/local/go/src/net/http/transport.go:1418 +0x1a1
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:1062 +0x4e9

goroutine 1553 [select, 4 minutes]:
net/http.(*persistConn).writeLoop(0xc420072b00)
	/usr/local/go/src/net/http/transport.go:1646 +0x3bd
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:1063 +0x50e

goroutine 1496 [IO wait, 5 minutes]:
net.runtime_pollWait(0x7f90cc8d9840, 0x72, 0x33)
	/usr/local/go/src/runtime/netpoll.go:160 +0x59
net.(*pollDesc).wait(0xc4202ec760, 0x72, 0xc42033a9d0, 0xc420014068)
	/usr/local/go/src/net/fd_poll_runtime.go:73 +0x38
net.(*pollDesc).waitRead(0xc4202ec760, 0x1e94780, 0xc420014068)
	/usr/local/go/src/net/fd_poll_runtime.go:78 +0x34
net.(*netFD).Read(0xc4202ec700, 0xc420296000, 0x1000, 0x1000, 0x0, 0x1e94780, 0xc420014068)
	/usr/local/go/src/net/fd_unix.go:243 +0x1a1
net.(*conn).Read(0xc4201344e8, 0xc420296000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/net.go:173 +0x70
net/http.(*persistConn).Read(0xc420072a00, 0xc420296000, 0x1000, 0x1000, 0x30, 0xc42033ab58, 0x60033c)
	/usr/local/go/src/net/http/transport.go:1261 +0x154
bufio.(*Reader).fill(0xc420601320)
	/usr/local/go/src/bufio/bufio.go:97 +0x10c
bufio.(*Reader).Peek(0xc420601320, 0x1, 0x0, 0x1, 0x0, 0xc4206a0e40, 0x0)
	/usr/local/go/src/bufio/bufio.go:129 +0x62
net/http.(*persistConn).readLoop(0xc420072a00)
	/usr/local/go/src/net/http/transport.go:1418 +0x1a1
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:1062 +0x4e9

goroutine 777 [chan receive, 7 minutes]:
database/sql.(*DB).connectionOpener(0xc4204e2210)
	/usr/local/go/src/database/sql/sql.go:730 +0x4a
created by database/sql.Open
	/usr/local/go/src/database/sql/sql.go:493 +0x1e9

goroutine 496 [select, 7 minutes]:
net/http.(*persistConn).writeLoop(0xc420137d00)
	/usr/local/go/src/net/http/transport.go:1646 +0x3bd
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:1063 +0x50e

goroutine 599 [select, 7 minutes]:
net/http.(*persistConn).writeLoop(0xc42041b900)
	/usr/local/go/src/net/http/transport.go:1646 +0x3bd
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:1063 +0x50e

goroutine 1501 [IO wait, 4 minutes]:
net.runtime_pollWait(0x7f90cc8d9900, 0x72, 0x30)
	/usr/local/go/src/runtime/netpoll.go:160 +0x59
net.(*pollDesc).wait(0xc4202ecdf0, 0x72, 0xc42033c9d0, 0xc420014068)
	/usr/local/go/src/net/fd_poll_runtime.go:73 +0x38
net.(*pollDesc).waitRead(0xc4202ecdf0, 0x1e94780, 0xc420014068)
	/usr/local/go/src/net/fd_poll_runtime.go:78 +0x34
net.(*netFD).Read(0xc4202ecd90, 0xc42035b000, 0x1000, 0x1000, 0x0, 0x1e94780, 0xc420014068)
	/usr/local/go/src/net/fd_unix.go:243 +0x1a1
net.(*conn).Read(0xc420134580, 0xc42035b000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/net.go:173 +0x70
net/http.(*persistConn).Read(0xc420072d00, 0xc42035b000, 0x1000, 0x1000, 0x30, 0xc42033cb58, 0x60033c)
	/usr/local/go/src/net/http/transport.go:1261 +0x154
bufio.(*Reader).fill(0xc4206761e0)
	/usr/local/go/src/bufio/bufio.go:97 +0x10c
bufio.(*Reader).Peek(0xc4206761e0, 0x1, 0x0, 0x1, 0x0, 0xc420074180, 0x0)
	/usr/local/go/src/bufio/bufio.go:129 +0x62
net/http.(*persistConn).readLoop(0xc420072d00)
	/usr/local/go/src/net/http/transport.go:1418 +0x1a1
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:1062 +0x4e9

goroutine 1463 [IO wait, 5 minutes]:
net.runtime_pollWait(0x7f90cc8d9a80, 0x72, 0x2e)
	/usr/local/go/src/runtime/netpoll.go:160 +0x59
net.(*pollDesc).wait(0xc42021cbc0, 0x72, 0xc4202579d0, 0xc420014068)
	/usr/local/go/src/net/fd_poll_runtime.go:73 +0x38
net.(*pollDesc).waitRead(0xc42021cbc0, 0x1e94780, 0xc420014068)
	/usr/local/go/src/net/fd_poll_runtime.go:78 +0x34
net.(*netFD).Read(0xc42021cb60, 0xc420342000, 0x1000, 0x1000, 0x0, 0x1e94780, 0xc420014068)
	/usr/local/go/src/net/fd_unix.go:243 +0x1a1
net.(*conn).Read(0xc42028e0e0, 0xc420342000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/net.go:173 +0x70
net/http.(*persistConn).Read(0xc420358300, 0xc420342000, 0x1000, 0x1000, 0x30, 0xc420257b58, 0x60033c)
	/usr/local/go/src/net/http/transport.go:1261 +0x154
bufio.(*Reader).fill(0xc420231320)
	/usr/local/go/src/bufio/bufio.go:97 +0x10c
bufio.(*Reader).Peek(0xc420231320, 0x1, 0x0, 0x1, 0x0, 0xc420340ba0, 0x0)
	/usr/local/go/src/bufio/bufio.go:129 +0x62
net/http.(*persistConn).readLoop(0xc420358300)
	/usr/local/go/src/net/http/transport.go:1418 +0x1a1
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:1062 +0x4e9

goroutine 655 [IO wait, 7 minutes]:
net.runtime_pollWait(0x7f90cc8d8ac0, 0x72, 0x1b)
	/usr/local/go/src/runtime/netpoll.go:160 +0x59
net.(*pollDesc).wait(0xc42013a220, 0x72, 0xc4204cc9d0, 0xc420014068)
	/usr/local/go/src/net/fd_poll_runtime.go:73 +0x38
net.(*pollDesc).waitRead(0xc42013a220, 0x1e94780, 0xc420014068)
	/usr/local/go/src/net/fd_poll_runtime.go:78 +0x34
net.(*netFD).Read(0xc42013a1c0, 0xc4204f1000, 0x1000, 0x1000, 0x0, 0x1e94780, 0xc420014068)
	/usr/local/go/src/net/fd_unix.go:243 +0x1a1
net.(*conn).Read(0xc4204f4070, 0xc4204f1000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/net.go:173 +0x70
net/http.(*persistConn).Read(0xc420072300, 0xc4204f1000, 0x1000, 0x1000, 0x30, 0xc4204ccb58, 0x60033c)
	/usr/local/go/src/net/http/transport.go:1261 +0x154
bufio.(*Reader).fill(0xc4202e88a0)
	/usr/local/go/src/bufio/bufio.go:97 +0x10c
bufio.(*Reader).Peek(0xc4202e88a0, 0x1, 0x0, 0x1, 0x0, 0xc4203624e0, 0x0)
	/usr/local/go/src/bufio/bufio.go:129 +0x62
net/http.(*persistConn).readLoop(0xc420072300)
	/usr/local/go/src/net/http/transport.go:1418 +0x1a1
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:1062 +0x4e9

goroutine 822 [select, 6 minutes]:
net/http.(*persistConn).writeLoop(0xc420072200)
	/usr/local/go/src/net/http/transport.go:1646 +0x3bd
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:1063 +0x50e

goroutine 1488 [IO wait, 4 minutes]:
net.runtime_pollWait(0x7f90cc8d7ec0, 0x72, 0x32)
	/usr/local/go/src/runtime/netpoll.go:160 +0x59
net.(*pollDesc).wait(0xc4202885a0, 0x72, 0xc42025a9d0, 0xc420014068)
	/usr/local/go/src/net/fd_poll_runtime.go:73 +0x38
net.(*pollDesc).waitRead(0xc4202885a0, 0x1e94780, 0xc420014068)
	/usr/local/go/src/net/fd_poll_runtime.go:78 +0x34
net.(*netFD).Read(0xc420288540, 0xc42032d000, 0x1000, 0x1000, 0x0, 0x1e94780, 0xc420014068)
	/usr/local/go/src/net/fd_unix.go:243 +0x1a1
net.(*conn).Read(0xc42028e080, 0xc42032d000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/net.go:173 +0x70
net/http.(*persistConn).Read(0xc420137a00, 0xc42032d000, 0x1000, 0x1000, 0x30, 0xc42025ab58, 0x60033c)
	/usr/local/go/src/net/http/transport.go:1261 +0x154
bufio.(*Reader).fill(0xc4206a0a20)
	/usr/local/go/src/bufio/bufio.go:97 +0x10c
bufio.(*Reader).Peek(0xc4206a0a20, 0x1, 0x0, 0x1, 0x0, 0xc420626000, 0x0)
	/usr/local/go/src/bufio/bufio.go:129 +0x62
net/http.(*persistConn).readLoop(0xc420137a00)
	/usr/local/go/src/net/http/transport.go:1418 +0x1a1
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:1062 +0x4e9

goroutine 1414 [select, 5 minutes]:
net/http.(*persistConn).writeLoop(0xc420072800)
	/usr/local/go/src/net/http/transport.go:1646 +0x3bd
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:1063 +0x50e

goroutine 768 [select, 7 minutes]:
net/http.(*persistConn).writeLoop(0xc42041bf00)
	/usr/local/go/src/net/http/transport.go:1646 +0x3bd
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:1063 +0x50e

goroutine 1021 [select, 6 minutes]:
net/http.(*persistConn).writeLoop(0xc420072700)
	/usr/local/go/src/net/http/transport.go:1646 +0x3bd
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:1063 +0x50e

goroutine 821 [IO wait, 6 minutes]:
net.runtime_pollWait(0x7f90cc8d8880, 0x72, 0x1e)
	/usr/local/go/src/runtime/netpoll.go:160 +0x59
net.(*pollDesc).wait(0xc42031e370, 0x72, 0xc4200519d0, 0xc420014068)
	/usr/local/go/src/net/fd_poll_runtime.go:73 +0x38
net.(*pollDesc).waitRead(0xc42031e370, 0x1e94780, 0xc420014068)
	/usr/local/go/src/net/fd_poll_runtime.go:78 +0x34
net.(*netFD).Read(0xc42031e310, 0xc4201d6000, 0x1000, 0x1000, 0x0, 0x1e94780, 0xc420014068)
	/usr/local/go/src/net/fd_unix.go:243 +0x1a1
net.(*conn).Read(0xc420134410, 0xc4201d6000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/net.go:173 +0x70
net/http.(*persistConn).Read(0xc420072200, 0xc4201d6000, 0x1000, 0x1000, 0x30, 0xc420051b58, 0x60033c)
	/usr/local/go/src/net/http/transport.go:1261 +0x154
bufio.(*Reader).fill(0xc420362720)
	/usr/local/go/src/bufio/bufio.go:97 +0x10c
bufio.(*Reader).Peek(0xc420362720, 0x1, 0x0, 0x1, 0x0, 0xc420362a20, 0x0)
	/usr/local/go/src/bufio/bufio.go:129 +0x62
net/http.(*persistConn).readLoop(0xc420072200)
	/usr/local/go/src/net/http/transport.go:1418 +0x1a1
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:1062 +0x4e9

goroutine 1497 [select, 5 minutes]:
net/http.(*persistConn).writeLoop(0xc420072a00)
	/usr/local/go/src/net/http/transport.go:1646 +0x3bd
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:1063 +0x50e

goroutine 1569 [select]:
github.com/cockroachdb/cockroach/pkg/rpc.(*Context).runHeartbeat(0xc42012d700, 0xc420358400, 0xc4205940c0, 0xf, 0x0, 0x0)
	/go/src/github.com/cockroachdb/cockroach/pkg/rpc/context.go:292 +0x650
github.com/cockroachdb/cockroach/pkg/rpc.(*Context).GRPCDial.func1.1.1()
	/go/src/github.com/cockroachdb/cockroach/pkg/rpc/context.go:227 +0x69
github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker.func1(0xc42046d680, 0xc4202d3470)
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:196 +0x7d
created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:197 +0x66

goroutine 855 [chan receive, 7 minutes]:
database/sql.(*DB).connectionOpener(0xc4203afe40)
	/usr/local/go/src/database/sql/sql.go:730 +0x4a
created by database/sql.Open
	/usr/local/go/src/database/sql/sql.go:493 +0x1e9

goroutine 845 [chan receive, 7 minutes]:
database/sql.(*DB).connectionOpener(0xc420126d10)
	/usr/local/go/src/database/sql/sql.go:730 +0x4a
created by database/sql.Open
	/usr/local/go/src/database/sql/sql.go:493 +0x1e9

goroutine 1568 [select, 4 minutes]:
google.golang.org/grpc.(*addrConn).transportMonitor(0xc42009e3c0)
	/go/src/google.golang.org/grpc/clientconn.go:720 +0x63e
google.golang.org/grpc.(*ClientConn).resetAddrConn.func1(0xc42009e3c0)
	/go/src/google.golang.org/grpc/clientconn.go:479 +0x1de
created by google.golang.org/grpc.(*ClientConn).resetAddrConn
	/go/src/google.golang.org/grpc/clientconn.go:480 +0x3c7

goroutine 1150 [IO wait, 6 minutes]:
net.runtime_pollWait(0x7f90cc8d8580, 0x72, 0x24)
	/usr/local/go/src/runtime/netpoll.go:160 +0x59
net.(*pollDesc).wait(0xc4202d4220, 0x72, 0xc42052e9d0, 0xc420014068)
	/usr/local/go/src/net/fd_poll_runtime.go:73 +0x38
net.(*pollDesc).waitRead(0xc4202d4220, 0x1e94780, 0xc420014068)
	/usr/local/go/src/net/fd_poll_runtime.go:78 +0x34
net.(*netFD).Read(0xc4202d41c0, 0xc4202e1000, 0x1000, 0x1000, 0x0, 0x1e94780, 0xc420014068)
	/usr/local/go/src/net/fd_unix.go:243 +0x1a1
net.(*conn).Read(0xc42028e058, 0xc4202e1000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/net.go:173 +0x70
net/http.(*persistConn).Read(0xc420358100, 0xc4202e1000, 0x1000, 0x1000, 0x30, 0xc42052eb58, 0x60033c)
	/usr/local/go/src/net/http/transport.go:1261 +0x154
bufio.(*Reader).fill(0xc4201447e0)
	/usr/local/go/src/bufio/bufio.go:97 +0x10c
bufio.(*Reader).Peek(0xc4201447e0, 0x1, 0x0, 0x1, 0x0, 0xc420202000, 0x0)
	/usr/local/go/src/bufio/bufio.go:129 +0x62
net/http.(*persistConn).readLoop(0xc420358100)
	/usr/local/go/src/net/http/transport.go:1418 +0x1a1
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:1062 +0x4e9

goroutine 1151 [select, 6 minutes]:
net/http.(*persistConn).writeLoop(0xc420358100)
	/usr/local/go/src/net/http/transport.go:1646 +0x3bd
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:1063 +0x50e

goroutine 1164 [IO wait, 6 minutes]:
net.runtime_pollWait(0x7f90cc8d8700, 0x72, 0x26)
	/usr/local/go/src/runtime/netpoll.go:160 +0x59
net.(*pollDesc).wait(0xc42038a3e0, 0x72, 0xc4206ce9d0, 0xc420014068)
	/usr/local/go/src/net/fd_poll_runtime.go:73 +0x38
net.(*pollDesc).waitRead(0xc42038a3e0, 0x1e94780, 0xc420014068)
	/usr/local/go/src/net/fd_poll_runtime.go:78 +0x34
net.(*netFD).Read(0xc42038a380, 0xc4201bd000, 0x1000, 0x1000, 0x0, 0x1e94780, 0xc420014068)
	/usr/local/go/src/net/fd_unix.go:243 +0x1a1
net.(*conn).Read(0xc4200320d8, 0xc4201bd000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/net.go:173 +0x70
net/http.(*persistConn).Read(0xc420137c00, 0xc4201bd000, 0x1000, 0x1000, 0x30, 0xc4206ceb58, 0x60033c)
	/usr/local/go/src/net/http/transport.go:1261 +0x154
bufio.(*Reader).fill(0xc4206008a0)
	/usr/local/go/src/bufio/bufio.go:97 +0x10c
bufio.(*Reader).Peek(0xc4206008a0, 0x1, 0x0, 0x1, 0x0, 0xc420676480, 0x0)
	/usr/local/go/src/bufio/bufio.go:129 +0x62
net/http.(*persistConn).readLoop(0xc420137c00)
	/usr/local/go/src/net/http/transport.go:1418 +0x1a1
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:1062 +0x4e9

goroutine 1165 [select, 6 minutes]:
net/http.(*persistConn).writeLoop(0xc420137c00)
	/usr/local/go/src/net/http/transport.go:1646 +0x3bd
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:1063 +0x50e

goroutine 1413 [IO wait, 5 minutes]:
net.runtime_pollWait(0x7f90cc8d87c0, 0x72, 0x2f)
	/usr/local/go/src/runtime/netpoll.go:160 +0x59
net.(*pollDesc).wait(0xc42021c530, 0x72, 0xc4204ca9d0, 0xc420014068)
	/usr/local/go/src/net/fd_poll_runtime.go:73 +0x38
net.(*pollDesc).waitRead(0xc42021c530, 0x1e94780, 0xc420014068)
	/usr/local/go/src/net/fd_poll_runtime.go:78 +0x34
net.(*netFD).Read(0xc42021c4d0, 0xc4206ab000, 0x1000, 0x1000, 0x0, 0x1e94780, 0xc420014068)
	/usr/local/go/src/net/fd_unix.go:243 +0x1a1
net.(*conn).Read(0xc420134478, 0xc4206ab000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/net.go:173 +0x70
net/http.(*persistConn).Read(0xc420072800, 0xc4206ab000, 0x1000, 0x1000, 0x30, 0xc4204cab58, 0x60033c)
	/usr/local/go/src/net/http/transport.go:1261 +0x154
bufio.(*Reader).fill(0xc4206a0de0)
	/usr/local/go/src/bufio/bufio.go:97 +0x10c
bufio.(*Reader).Peek(0xc4206a0de0, 0x1, 0x0, 0x1, 0x0, 0xc42059e180, 0x0)
	/usr/local/go/src/bufio/bufio.go:129 +0x62
net/http.(*persistConn).readLoop(0xc420072800)
	/usr/local/go/src/net/http/transport.go:1418 +0x1a1
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:1062 +0x4e9

goroutine 1417 [IO wait, 5 minutes]:
net.runtime_pollWait(0x7f90cc8d7f80, 0x72, 0x2d)
	/usr/local/go/src/runtime/netpoll.go:160 +0x59
net.(*pollDesc).wait(0xc420288220, 0x72, 0xc4205359d0, 0xc420014068)
	/usr/local/go/src/net/fd_poll_runtime.go:73 +0x38
net.(*pollDesc).waitRead(0xc420288220, 0x1e94780, 0xc420014068)
	/usr/local/go/src/net/fd_poll_runtime.go:78 +0x34
net.(*netFD).Read(0xc4202881c0, 0xc4205c5000, 0x1000, 0x1000, 0x0, 0x1e94780, 0xc420014068)
	/usr/local/go/src/net/fd_unix.go:243 +0x1a1
net.(*conn).Read(0xc420032100, 0xc4205c5000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/net.go:173 +0x70
net/http.(*persistConn).Read(0xc420072500, 0xc4205c5000, 0x1000, 0x1000, 0x30, 0xc420535b58, 0x60033c)
	/usr/local/go/src/net/http/transport.go:1261 +0x154
bufio.(*Reader).fill(0xc420600540)
	/usr/local/go/src/bufio/bufio.go:97 +0x10c
bufio.(*Reader).Peek(0xc420600540, 0x1, 0x0, 0x1, 0x0, 0xc42069d320, 0x0)
	/usr/local/go/src/bufio/bufio.go:129 +0x62
net/http.(*persistConn).readLoop(0xc420072500)
	/usr/local/go/src/net/http/transport.go:1418 +0x1a1
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:1062 +0x4e9

goroutine 1552 [select, 4 minutes]:
net/http.(*persistConn).readLoop(0xc420072b00)
	/usr/local/go/src/net/http/transport.go:1541 +0x9c9
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:1062 +0x4e9

goroutine 1489 [select, 4 minutes]:
net/http.(*persistConn).writeLoop(0xc420137a00)
	/usr/local/go/src/net/http/transport.go:1646 +0x3bd
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:1063 +0x50e

goroutine 1331 [IO wait, 6 minutes]:
net.runtime_pollWait(0x7f90cc8d8040, 0x72, 0x2b)
	/usr/local/go/src/runtime/netpoll.go:160 +0x59
net.(*pollDesc).wait(0xc42038a220, 0x72, 0xc4206c89d0, 0xc420014068)
	/usr/local/go/src/net/fd_poll_runtime.go:73 +0x38
net.(*pollDesc).waitRead(0xc42038a220, 0x1e94780, 0xc420014068)
	/usr/local/go/src/net/fd_poll_runtime.go:78 +0x34
net.(*netFD).Read(0xc42038a1c0, 0xc4205db000, 0x1000, 0x1000, 0x0, 0x1e94780, 0xc420014068)
	/usr/local/go/src/net/fd_unix.go:243 +0x1a1
net.(*conn).Read(0xc420134420, 0xc4205db000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/net.go:173 +0x70
net/http.(*persistConn).Read(0xc420137e00, 0xc4205db000, 0x1000, 0x1000, 0x30, 0xc4206c8b58, 0x60033c)
	/usr/local/go/src/net/http/transport.go:1261 +0x154
bufio.(*Reader).fill(0xc420676780)
	/usr/local/go/src/bufio/bufio.go:97 +0x10c
bufio.(*Reader).Peek(0xc420676780, 0x1, 0x0, 0x1, 0x0, 0xc42069c000, 0x0)
	/usr/local/go/src/bufio/bufio.go:129 +0x62
net/http.(*persistConn).readLoop(0xc420137e00)
	/usr/local/go/src/net/http/transport.go:1418 +0x1a1
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:1062 +0x4e9

goroutine 1549 [select, 4 minutes]:
net/http.(*persistConn).writeLoop(0xc42000b000)
	/usr/local/go/src/net/http/transport.go:1646 +0x3bd
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:1063 +0x50e

goroutine 1332 [select, 6 minutes]:
net/http.(*persistConn).writeLoop(0xc420137e00)
	/usr/local/go/src/net/http/transport.go:1646 +0x3bd
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:1063 +0x50e

goroutine 1502 [select, 4 minutes]:
net/http.(*persistConn).writeLoop(0xc420072d00)
	/usr/local/go/src/net/http/transport.go:1646 +0x3bd
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:1063 +0x50e

goroutine 1287 [IO wait, 6 minutes]:
net.runtime_pollWait(0x7f90cc8d8100, 0x72, 0x2a)
	/usr/local/go/src/runtime/netpoll.go:160 +0x59
net.(*pollDesc).wait(0xc42024c220, 0x72, 0xc4205319d0, 0xc420014068)
	/usr/local/go/src/net/fd_poll_runtime.go:73 +0x38
net.(*pollDesc).waitRead(0xc42024c220, 0x1e94780, 0xc420014068)
	/usr/local/go/src/net/fd_poll_runtime.go:78 +0x34
net.(*netFD).Read(0xc42024c1c0, 0xc4200eb000, 0x1000, 0x1000, 0x0, 0x1e94780, 0xc420014068)
	/usr/local/go/src/net/fd_unix.go:243 +0x1a1
net.(*conn).Read(0xc4200320d0, 0xc4200eb000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/net.go:173 +0x70
net/http.(*persistConn).Read(0xc420137500, 0xc4200eb000, 0x1000, 0x1000, 0x30, 0xc420531b58, 0x60033c)
	/usr/local/go/src/net/http/transport.go:1261 +0x154
bufio.(*Reader).fill(0xc420600b40)
	/usr/local/go/src/bufio/bufio.go:97 +0x10c
bufio.(*Reader).Peek(0xc420600b40, 0x1, 0x0, 0x1, 0x0, 0xc420600ba0, 0x0)
	/usr/local/go/src/bufio/bufio.go:129 +0x62
net/http.(*persistConn).readLoop(0xc420137500)
	/usr/local/go/src/net/http/transport.go:1418 +0x1a1
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:1062 +0x4e9

goroutine 1532 [select, 4 minutes]:
github.com/cockroachdb/cockroach/pkg/acceptance/cluster.(*LocalCluster).monitor.func1(0x1)
	/go/src/github.com/cockroachdb/cockroach/pkg/acceptance/cluster/localcluster.go:608 +0x4aa
github.com/cockroachdb/cockroach/pkg/acceptance/cluster.(*LocalCluster).monitor(0xc42000ac00)
	/go/src/github.com/cockroachdb/cockroach/pkg/acceptance/cluster/localcluster.go:628 +0x61
created by github.com/cockroachdb/cockroach/pkg/acceptance/cluster.(*LocalCluster).Start
	/go/src/github.com/cockroachdb/cockroach/pkg/acceptance/cluster/localcluster.go:652 +0x57e

goroutine 1288 [select, 6 minutes]:
net/http.(*persistConn).writeLoop(0xc420137500)
	/usr/local/go/src/net/http/transport.go:1646 +0x3bd
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:1063 +0x50e

goroutine 1644 [select, 4 minutes]:
google.golang.org/grpc/transport.(*http2Client).controller(0xc42000b300)
	/go/src/google.golang.org/grpc/transport/http2_client.go:999 +0x59d
created by google.golang.org/grpc/transport.newHTTP2Client
	/go/src/google.golang.org/grpc/transport/http2_client.go:233 +0xe78

goroutine 1548 [IO wait, 4 minutes]:
net.runtime_pollWait(0x7f90cc8d9780, 0x72, 0x34)
	/usr/local/go/src/runtime/netpoll.go:160 +0x59
net.(*pollDesc).wait(0xc4202ec4c0, 0x72, 0xc42033b9d0, 0xc420014068)
	/usr/local/go/src/net/fd_poll_runtime.go:73 +0x38
net.(*pollDesc).waitRead(0xc4202ec4c0, 0x1e94780, 0xc420014068)
	/usr/local/go/src/net/fd_poll_runtime.go:78 +0x34
net.(*netFD).Read(0xc4202ec460, 0xc42048b000, 0x1000, 0x1000, 0x0, 0x1e94780, 0xc420014068)
	/usr/local/go/src/net/fd_unix.go:243 +0x1a1
net.(*conn).Read(0xc420134488, 0xc42048b000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/net.go:173 +0x70
net/http.(*persistConn).Read(0xc42000b000, 0xc42048b000, 0x1000, 0x1000, 0x8a0d30, 0xc42033bb58, 0x5c90ad)
	/usr/local/go/src/net/http/transport.go:1261 +0x154
bufio.(*Reader).fill(0xc4206a1380)
	/usr/local/go/src/bufio/bufio.go:97 +0x10c
bufio.(*Reader).Peek(0xc4206a1380, 0x1, 0xc42033bbbd, 0x1, 0x0, 0xc420690960, 0x0)
	/usr/local/go/src/bufio/bufio.go:129 +0x62
net/http.(*persistConn).readLoop(0xc42000b000)
	/usr/local/go/src/net/http/transport.go:1418 +0x1a1
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:1062 +0x4e9

goroutine 1448 [IO wait, 5 minutes]:
net.runtime_pollWait(0x7f90cc8d99c0, 0x72, 0x31)
	/usr/local/go/src/runtime/netpoll.go:160 +0x59
net.(*pollDesc).wait(0xc4202d51e0, 0x72, 0xc4202569d0, 0xc420014068)
	/usr/local/go/src/net/fd_poll_runtime.go:73 +0x38
net.(*pollDesc).waitRead(0xc4202d51e0, 0x1e94780, 0xc420014068)
	/usr/local/go/src/net/fd_poll_runtime.go:78 +0x34
net.(*netFD).Read(0xc4202d5180, 0xc420297000, 0x1000, 0x1000, 0x0, 0x1e94780, 0xc420014068)
	/usr/local/go/src/net/fd_unix.go:243 +0x1a1
net.(*conn).Read(0xc420032198, 0xc420297000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/net.go:173 +0x70
net/http.(*persistConn).Read(0xc420072900, 0xc420297000, 0x1000, 0x1000, 0x30, 0xc420256b58, 0x60033c)
	/usr/local/go/src/net/http/transport.go:1261 +0x154
bufio.(*Reader).fill(0xc4201c0e40)
	/usr/local/go/src/bufio/bufio.go:97 +0x10c
bufio.(*Reader).Peek(0xc4201c0e40, 0x1, 0x0, 0x1, 0x0, 0xc4201c1b60, 0x0)
	/usr/local/go/src/bufio/bufio.go:129 +0x62
net/http.(*persistConn).readLoop(0xc420072900)
	/usr/local/go/src/net/http/transport.go:1418 +0x1a1
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:1062 +0x4e9

goroutine 1464 [select, 5 minutes]:
net/http.(*persistConn).writeLoop(0xc420358300)
	/usr/local/go/src/net/http/transport.go:1646 +0x3bd
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:1063 +0x50e

goroutine 1340 [IO wait, 6 minutes]:
net.runtime_pollWait(0x7f90cc8d7c80, 0x72, 0x2c)
	/usr/local/go/src/runtime/netpoll.go:160 +0x59
net.(*pollDesc).wait(0xc42038a450, 0x72, 0xc4206cf9d0, 0xc420014068)
	/usr/local/go/src/net/fd_poll_runtime.go:73 +0x38
net.(*pollDesc).waitRead(0xc42038a450, 0x1e94780, 0xc420014068)
	/usr/local/go/src/net/fd_poll_runtime.go:78 +0x34
net.(*netFD).Read(0xc42038a3f0, 0xc42031b000, 0x1000, 0x1000, 0x0, 0x1e94780, 0xc420014068)
	/usr/local/go/src/net/fd_unix.go:243 +0x1a1
net.(*conn).Read(0xc4200320f8, 0xc42031b000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/net.go:173 +0x70
net/http.(*persistConn).Read(0xc420137f00, 0xc42031b000, 0x1000, 0x1000, 0x30, 0xc4206cfb58, 0x60033c)
	/usr/local/go/src/net/http/transport.go:1261 +0x154
bufio.(*Reader).fill(0xc42069c6c0)
	/usr/local/go/src/bufio/bufio.go:97 +0x10c
bufio.(*Reader).Peek(0xc42069c6c0, 0x1, 0x0, 0x1, 0x0, 0xc42053f1a0, 0x0)
	/usr/local/go/src/bufio/bufio.go:129 +0x62
net/http.(*persistConn).readLoop(0xc420137f00)
	/usr/local/go/src/net/http/transport.go:1418 +0x1a1
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:1062 +0x4e9

goroutine 1341 [select, 6 minutes]:
net/http.(*persistConn).writeLoop(0xc420137f00)
	/usr/local/go/src/net/http/transport.go:1646 +0x3bd
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:1063 +0x50e

goroutine 1418 [select, 5 minutes]:
net/http.(*persistConn).writeLoop(0xc420072500)
	/usr/local/go/src/net/http/transport.go:1646 +0x3bd
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:1063 +0x50e

goroutine 1449 [select, 5 minutes]:
net/http.(*persistConn).writeLoop(0xc420072900)
	/usr/local/go/src/net/http/transport.go:1646 +0x3bd
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:1063 +0x50e

goroutine 1550 [IO wait, 4 minutes]:
net.runtime_pollWait(0x7f90cc8d96c0, 0x72, 0x35)
	/usr/local/go/src/runtime/netpoll.go:160 +0x59
net.(*pollDesc).wait(0xc4202d41b0, 0x72, 0xc4206d9818, 0xc420014068)
	/usr/local/go/src/net/fd_poll_runtime.go:73 +0x38
net.(*pollDesc).waitRead(0xc4202d41b0, 0x1e94780, 0xc420014068)
	/usr/local/go/src/net/fd_poll_runtime.go:78 +0x34
net.(*netFD).Read(0xc4202d4150, 0xc4205a9000, 0x1000, 0x1000, 0x0, 0x1e94780, 0xc420014068)
	/usr/local/go/src/net/fd_unix.go:243 +0x1a1
net.(*conn).Read(0xc420032148, 0xc4205a9000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/net.go:173 +0x70
net/http.(*persistConn).Read(0xc420072b00, 0xc4205a9000, 0x1000, 0x1000, 0x9cc7c8, 0xc4205fc868, 0x7d)
	/usr/local/go/src/net/http/transport.go:1261 +0x154
bufio.(*Reader).fill(0xc4200f05a0)
	/usr/local/go/src/bufio/bufio.go:97 +0x10c
bufio.(*Reader).ReadSlice(0xc4200f05a0, 0x10a, 0x2, 0x0, 0x199, 0x186, 0x186)
	/usr/local/go/src/bufio/bufio.go:330 +0xb5
net/http/internal.readChunkLine(0xc4200f05a0, 0x5d3659, 0xc4200f0058, 0x0, 0x1, 0xc4206d9a88)
	/usr/local/go/src/net/http/internal/chunked.go:110 +0x34
net/http/internal.(*chunkedReader).beginChunk(0xc4204890e0)
	/usr/local/go/src/net/http/internal/chunked.go:47 +0x32
net/http/internal.(*chunkedReader).Read(0xc4204890e0, 0xc4202b8001, 0x5ff, 0x5ff, 0x619db0, 0xc4206d9b70, 0xc4206d9b78)
	/usr/local/go/src/net/http/internal/chunked.go:77 +0x7b
net/http.(*body).readLocked(0xc42035e140, 0xc4202b8001, 0x5ff, 0x5ff, 0x2, 0x2, 0x0)
	/usr/local/go/src/net/http/transfer.go:651 +0x61
net/http.(*body).Read(0xc42035e140, 0xc4202b8001, 0x5ff, 0x5ff, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/http/transfer.go:643 +0xf6
net/http.(*bodyEOFSignal).Read(0xc42035e180, 0xc4202b8001, 0x5ff, 0x5ff, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/http/transport.go:1974 +0x116
encoding/json.(*Decoder).refill(0xc4205fc820, 0xa, 0x9)
	/usr/local/go/src/encoding/json/stream.go:152 +0xfa
encoding/json.(*Decoder).readValue(0xc4205fc820, 0x0, 0x0, 0x80)
	/usr/local/go/src/encoding/json/stream.go:128 +0x198
encoding/json.(*Decoder).Decode(0xc4205fc820, 0x139c760, 0xc42000dc00, 0x0, 0x0)
	/usr/local/go/src/encoding/json/stream.go:57 +0x8e
github.com/docker/docker/client.(*Client).Events.func1(0xc4200f0060, 0xc4202ec230, 0x0, 0x0, 0x0, 0x0, 0xc420488360, 0x7f90cda0cc18, 0xc4205ba640, 0xc4200f0000)
	/go/src/github.com/docker/docker/client/events.go:50 +0x2a7
created by github.com/docker/docker/client.(*Client).Events
	/go/src/github.com/docker/docker/client/events.go:63 +0xe2

Please assign, take a look and update the issue accordingly.
@cockroach-teamcity cockroach-teamcity added this to the Q2 milestone Nov 1, 2016
@cockroach-teamcity cockroach-teamcity added O-robot Originated from a bot. C-test-failure Broken test (automatically or manually discovered). labels Nov 1, 2016
@tamird
Copy link
Contributor

tamird commented Nov 3, 2016

#10409.

@tamird tamird closed this as completed Nov 3, 2016
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
C-test-failure Broken test (automatically or manually discovered). O-robot Originated from a bot.
Projects
None yet
Milestone
Q2
Development

No branches or pull requests
2 participants
@tamird @cockroach-teamcity