Unsolicited response received on idle HTTP channel starting with "\r\n"; err=<nil>

[kxepal]

Long story short

By default, aiohttp closes keep-alive connections after 75 seconds of inactivity. Just before closing it, server sends \r\n data to client in order to notify it about closing. However, not all the clients are happy with that. One of those is standard golang net/http which logs errors in this case not about closed connection, but about bad data received in place when there should be any of such.

Referenced commit: bf49f8e

Question is that behaviour correct or common, or based on some RFC?
What recommendations could we give to clients which complains about to relax their communication with aiohttp servers?

Expected behaviour

2018/03/01 13:52:12 Hello, Anonymous

Actual behaviour

2018/03/01 13:52:12 Hello, Anonymous
2018/03/01 13:53:27 Unsolicited response received on idle HTTP channel starting with "\r\n"; err=<nil>

Steps to reproduce

1. Run any aiohttp example. Let's take examples/web_srv.py
2. Save the following as main.go, for example:

package main

import (
    "fmt"
    "io/ioutil"
    "log"
    "net/http"
    "net/url"
    "time"
)

func main() {
    Url, err := url.Parse("http://localhost:8080/hello")
    if err != nil {
        log.Panic(err)
        return
    }

    client := http.Client{
        Timeout: time.Duration(time.Millisecond * 10000),
    }

    resp, err := client.Get(Url.String())
    if err != nil {
        log.Panic(err)
        return
    }

    body, err := ioutil.ReadAll(resp.Body)
    resp.Body.Close()

    log.Print(string(body))
    time.Sleep(100 * time.Second)
}

3. Run it as go run main.go
4. Wait for awhile and see the error from subject.

Your environment

1. Python 3.6 / aiohttp 3.0.5
2. go 1.9.1

[kxepal]

/cc @fafhrd91

[kxepal]

Currently, I suggested to limit idleTimeout to golang client by 60 seconds, so it will close idle connection long before aiohttp does it itself with additional response data. But not sure if that's ok in general to tweak clients to let them work with aiohttp nicely.

[hubo1016]

Shouldn't the server just close the connection? Another choice is shutdown the send side first, read and discard any received data for a short while, then close the connection, so the client won't receive broken pipe errors (but usually It is not worth doing this)

[kxepal]

Shouldn't the server just close the connection?

That's how it was and the reason of current behaviour is about to fix that state

• Write to transport "\r\n" before closing after keepalive timeout, otherwise client can not detect socket disconnection. ProcessPoolExecutor in background cause freeze after keep-alive timeout #1883

I didn't found any references in actual RFCs about such behaviour as like as I didn't noticed any recommended behaviour to avoid such issues. However, I didn't dig too much so could miss something.

[fafhrd91]

#1883 describes the problem. Maybe this should be configurable. I don’t have opinion

[hubo1016]

@fafhrd91 is it an asyncio bug? Looks weird.

[fafhrd91]

It is reproducible

[Tinche]

Our backend is also Go calling out to asyncio/aiohttp, and I've been seeing this.

[lock]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a [new issue] for related bugs.
If you feel like there's important points made in this discussion, please include those exceprts into that [new issue].
[new issue]: https://github.com/aio-libs/aiohttp/issues/new