feat(http1): add support for receiving trailer fields #3637
Conversation
There was a problem hiding this comment.
Thanks for the PR! It looks great overall.
If the server does not include a trailers: ... header indicating which trailer headers are being returned, should trailers be parsed and passed along? I think this one is almost certainly "yes".
Yes. I think hyper should pass this along and let users decide what to do with the extra trailers.
If the server includes trailers with invalid header names, such as Content-Length: 5, should that still be parsed and passed along?
I'm not too sure about this one. I see two possible options:
- Pass it along but mark it as unexpected
- Err it
I'm interested to see what the others think about this
src/proto/h1/decode.rs
Outdated
| trace!("found possible trailers"); | ||
| match decode_trailers( | ||
| &mut trailers_buf.take().expect("Trailer is None"), | ||
| // decoder enforces that trailers count will not exceed usize::MAX |
There was a problem hiding this comment.
There was a problem hiding this comment.
i also added a check in 545f8c4
| match byte { | ||
| b'\n' => { | ||
| if *trailers_cnt == usize::MAX as u64 { | ||
| return Poll::Ready(Err(io::Error::new( |
There was a problem hiding this comment.
Can we write some tests covering this as well?
There was a problem hiding this comment.
added in 545f8c4
i also lowered the trailer limit to 1024 as i cannot imagine a use case for sending a million trailers.
There was a problem hiding this comment.
Yea, limits are needed, both on number of field pairs, and bytes itself, or else we expose servers to OOM attacks.
There was a problem hiding this comment.
I added a size limit in f194be1
The trailer limit is now 1024 instead of usize::MAX. There is also a test proving that the trailer limit is respected.
The size of the trailer fields is now limited. The limit accounts for a single, very large trailer field or many trailer fields that exceed the limit in aggregate.
Trailer parsing now honors h1_max_headers option. It also supports a future h1_max_header_size option.
There was a problem hiding this comment.
LGTM, thanks for the PR!
Anything you want to add? @seanmonstar
There was a problem hiding this comment.
Alright, just got to review it all. Looks really good, thank you!
The one thing I think that could help is to add some docs about the support, including what is and isn't enforced. But then, as I looked around so as to suggest where, I realized that there's likely several behavioral things for HTTP/1 or HTTP/2 that aren't really documented. So, in that case, it doesn't seem fair to block this on wanting the docs. Rather, I'll open an issue to decide on where to put those, and what things are missing.
|
@seanmonstar Please feel free to assign me any relevant issues. I already some blog posts on sending trailers:
I had planned on writing one for receiving trailers. I had a note to ask you if you wanted this sort of content in examples, the hyper.rs guides, etc. |
Closes #2703
In #3375 we were strict about sending trailers per RFC 7230. In this PR, I have been much more accepting of trailers. Is this the behavior we want?
Some questions:
TE: trailersbut the server sends trailer headers, should trailers be parsed and passed along?trailers: ...header indicating which trailer headers are being returned, should trailers be parsed and passed along? I think this one is almost certainly "yes".Content-Length: 5, should that still be parsed and passed along?