net/http: net.runtime_pollWait Error of concurrent http post #16079
Comments
|
It is likely your program has a data race. Please post a complete code sample the demonstrates the problem.
|
func (this *instance) pushMultiMsg(pushList) {
defer func() {
this.model.Close()
if err := recover(); err != nil {
logs.Logger.Critical(err)
}
}()
pushChans := make(chan bool, 10000)
for {
//get userId from redis list
userId := this.model.PopList(pushList)
if userId == "" {
// task finish
break
}
pushChans <- true
go this.concurrentPush(userId)
}
waitchan(pushChans)
}
func (this *instance) concurrentPush(userId) {
client := &http.Client{}
req, err := http.NewRequest("POST", 'http://localhost/send', bytes.NewReader(data))
req.Close = true
req.Header.Set("Content-Type", "application/json; charset=utf-8")
resp, err := client.Do(req)
defer resp.Body.Close()
body, err := ioutil.ReadAll(resp.Body)
if err != nil {
return
}
.....
}
func waitchan(pushChans chan bool) {
for len(pushChans) > 0 {
time.Sleep(time.Millisecond * 500)
}
}Redis infrastructure: LVS----twemproxy----REDISRedis list key structure: push_list:abcMy program logic flow: a) get userId from redis list [500000 msg data in list]
b) use a new goroutine for post data to URL [total 10000 goroutine] |
|
@woria Please post a complete code sample the demonstrates the problem. This program must be executable so we can attempt to reproduce the problem you have reported. |
|
I'm sorry, I don't understand what the problem is. And I don't understand what the stack trace is showing us; how did you generate that trace? If the program crashed, it should have said more than that. What makes this a "net.runtime_pollWait error"? What is the error? |
|
Don't create a new http.Client for each request. If you are sending 500,000 requests with a concurrency of 10,000 goroutines, you're very quickly going to run out of ephemeral ports on the client. http.Client objects are safe for use across multiple goroutines, so you should try sharing a single one. (Or, possibly one per goroutine; I don't know if you can issue 10,000 requests simultaneously from a single Client, or if there would be blocking.) |
|
@quentinmit |
issues: #15446 it should be getting an EOF on HEAD/POST requests where putting a retry loop around the call to client.Do.
The Go version is 1.6 what I use in my program. It's the reason why I try to specifying request.Close that can keep the connection open |
It sounds like you're saying that you get a spurious error if the keepalive'd connection was closed before the request. That will be fixed in issue #15723 by retrying the request. In the meantime, however, your code (in
We are never going to fix this for Go 1.6. If you are using Go 1.6, you are going to need to work around this as I suggest above. Otherwise, you can wait for a fix in Go 1.8. There's fundamentally nothing we can do in Go. You are requesting a separate TCP connection for each request, and there are a finite number of TCP ports which can be used for that connection (65,535 theoretical ports, though the actual number is lower depending on how your kernel is configured). When those ports are all used up, it's not possible to make a new connection to the same host. You need to wait for those ports to be freed up before you can make new requests. You can look at this by running "netstat -n"; you'll see a separate connection in I'm going to close this issue as I don't think there's anything we can do in Go. Please leave a comment if you think there is still a bug here that we can fix. Thanks. |
Please answer these questions before submitting your issue. Thanks!
go version)?1.62
go env)?GOARCH="amd64"
GOBIN=""
GOEXE=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOOS="linux"
GOPATH="/mnt/windows/web_application"
GORACE=""
GOROOT="/apps/lib/go"
GOTOOLDIR="/apps/lib/go/pkg/tool/linux_amd64"
GO15VENDOREXPERIMENT="1"
CC="gcc"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0"
CXX="g++"
CGO_ENABLED="1"
code like this:
a) get userId from redis list [500000 msg data in list]
b) use a new goroutine for post data to URL [total 10000 goroutine]
c) sometimes post all successfully , and sometimes a small number of list post successfully
ERROR LOG:
http post data successfully
The text was updated successfully, but these errors were encountered: