Hyper client fails if the header isn't fully received in the first read #389
Comments
|
True, I was hoping that the BufReader would fill it's entire buffer before yielding, which by default is 64kb. Is the test server head bigger than this? Or BufReader is yielding earlier? |
|
The head is a few bytes long. By looking at Wireshark I could see that the different lines of the head are spread among multiple TCP packets, which causes BufReader to yield the first received packet, which contains only the first head line. |
seanmonstar
added a commit
that referenced
this issue
Mar 27, 2015
This includes a custom BufReader, since the one in libstd doesn't allow reading additional data into the buffer without consuming it. This is required because some connections may send shorter packets, and so we need to perform multiple reads. After each read, the contents of the buffer are passed to httparse to see if have a valid message. If so, the proper amount of bytes are consumed. The additional bytes are left in the buffer since they are the beginning of the body. The buffer in this BufReader also grows in size, compared to the libstd which is sized once. This is because we start with a smaller buffer, since the majority of messages will be able to include their head in a packet or 2. Therefore, it's a wasteful performance hit to allocate the maximum size for every message. However, some headers can be quite big, and to allow for many of them to be set, we include a maximum size. Once we've hit the maximum buffer size, and still haven't determined the end of the headers, a HttpTooLargeError will be returned. Closes #389
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Some HTTP servers do not send the entire HTTP header in a single chunk. Hyper assumes that the entire header is received when calling fill_buf, which isn't always the case.
Running the client example in the documentation against a Flask test server causes it to panic:
thread '
' panicked at 'calledResult::unwrap()on anErrvalue: HttpTooLargeError'The text was updated successfully, but these errors were encountered: