Ideas for supporting HTTP Early Hints server-side #2426

acfoltzer · 2021-02-08T19:31:09Z

An important customer of ours is very interested in using HTTP 103 Early Hints to lower the perceived latency experienced by their end users. This requires the ability to return more than one response head during the message exchange. The basic example from the RFC is:

   Client request:

     GET / HTTP/1.1
     Host: example.com


   Server response:

     HTTP/1.1 103 Early Hints
     Link: </style.css>; rel=preload; as=style
     Link: </script.js>; rel=preload; as=script

     HTTP/1.1 200 OK
     Date: Fri, 26 May 2017 10:02:11 GMT
     Content-Length: 1234
     Content-Type: text/html; charset=utf-8
     Link: </style.css>; rel=preload; as=style
     Link: </script.js>; rel=preload; as=script

     <!doctype html>
     [... rest of the response body is omitted from the example ...]

With hyper's service function model, we currently only have the chance to return a single response head. Supporting multiple response heads may require some substantial API changes or additions, so I'm hoping we can discuss potential designs before diving into implementation. Please take any type signatures mentioned below to be a bit handwavy; I know things are more complex with the tower types and such.

Channel/stream-based interfaces

These are my preferred approaches from a hyper user's perspective, but I admit I tend to get lost when trying to follow all the types involved with tower services under the hood, so I'm not sure if this is particularly feasible on the implementation side.

Service function channel argument

We could add a variant of hyper::service::service_fn() whose closure takes an additional sender argument, i.e., FnMut(Request<R>, mpsc::Sender<Response<()>>) -> S. The service implementation would be free to send many interim responses, and would return the final response in the same way that existing service functions do. For example:

async fn my_service_fn(
    req: Request<Body>,
    mut interim_sender: mpsc::Sender<Response<()>>,
) -> Result<Response<Body>> {
    let early_hints = Response::builder()
        .status(103)
        .header("Link", "</style.css>; rel=preload; as=style")
        .header("Link", "</script.js>; rel=preload; as=script")
        .body(())?;
    interim_sender.send(early_hints).await?;
    let resp = todo!("build the final response")?;
    Ok(resp)
}

Service function response stream return value

A similar variant is to accept a Stream<Item = Result<Response<Body>>> as the return value from a new service function variant. This would give the flexibility for the service function to use channels internally, or return stream::once() for a basic case with no hints. For example:

async fn my_service_fn(
    req: Request<Body>,
) -> impl Stream<Item = Result<Response<Body>>> {
    let (mut result_sender, result_receiver) = mpsc::channel(1);
    tokio::task::spawn(async move {
        let early_hints = Response::builder()
            .status(103)
            .header("Link", "</style.css>; rel=preload; as=style")
            .header("Link", "</script.js>; rel=preload; as=script")
            .body(Body::empty())?;
        result_sender.send(Ok(early_hints)).await?;
        let resp = todo!("build the final response")?;
        result_sender.send(Ok(resp)).await?;
    });
    result_receiver
}

Error handling gets a bit awkward with this one, as it always does when detaching a task from a service function. Also, we don't enforce that the 103 has no body via the type system as we do when the channel is dedicated only to returning interim responses.

Functional interface

Inspired by hyper::upgrade::on(), we could add a function (strawman name hyper::interim_response()) that allows additional closures to be invoked with the current request, each of which would return an interim or final response. For example:

async fn my_service_fn(req: Request<Body>) -> Result<Response<Body>> {
    tokio::task::spawn(hyper::interim_response(async move {
        let resp = todo!("build the final response")?;
        Ok(resp)
    }));
    Ok(Response::builder()
        .status(103)
        .header("Link", "</style.css>; rel=preload; as=style")
        .header("Link", "</script.js>; rel=preload; as=script")
        .body(())?)
}

Error handling is also awkward with this one, and there's a bit of a continuation-passing style feel, but it's worth considering something that resembles the existing 1xx API. I believe the other approaches would be easier to work with, particularly if multiple 103 responses are sent in a single exchange.

Extension to body types

I could imagine implementing subsequent response heads as something that could be polled from the body like trailers. This would probably run afoul of many of the same problems that motivated #2086, though, so it seems unlikely to be the right choice.

I'm sure I'm missing some ideas here, but regardless of the final design chosen it would be great to figure out a plan forward. Please let me know how we can help.

The text was updated successfully, but these errors were encountered:

seanmonstar · 2021-02-08T21:39:02Z

For the record, this is also somewhat discussed in the issue about enabling Server Push (#1586), where I proposed something similar to hyper::upgrade, and also pondered on whether it should also send 103 links if the version is HTTP/1.1.

acfoltzer · 2021-02-08T22:44:18Z

Aha, I'd missed that thread because it didn't actually contain "hints". I like the look of that API as well; it would maintain the benefits of the channel approach without having to change the type signature of the service functions 👍 Perhaps it'd look something like this?

async fn my_service_fn(req: Request<Body>) -> Result<Response<Body>> {
    let early_hints = Response::builder()
        .status(103)
        .header("Link", "</style.css>; rel=preload; as=style")
        .header("Link", "</script.js>; rel=preload; as=script")
        .body(())?;
    hyper::interim_response(&mut req).send(early_hints).await?;
    let resp = todo!("build the final response")?;
    Ok(resp)
}

seanmonstar · 2021-02-09T00:51:08Z

Yea, something like that. I think I favor that style as well, for the reasons you mentioned. Any thoughts on whether sending early hints should be considered combined with server push? I like the name hyper::push more than interim_response, :D.

I could see the argument that perhaps they should be separate, to provide more control (and it's possible Server Push may see less use as browsers deactivate it, with Early Hints as the suggested future even in h2 and h3).

acfoltzer · 2021-02-09T18:03:52Z

Any thoughts on whether sending early hints should be considered combined with server push?

I don't have strongly held opinions about how the server push API should look, but I'd worry about confusion from exposing a single interface that can both push responses (presumably after sending a push promise elsewhere), and one that can only send 1xx responses with no associated promise. I could also see this confusion spilling over into implementation as you'd have to frame things differently in h2 for the two modes.

seanmonstar · 2021-02-10T17:52:21Z

I've felt similar. I think it'd be clever if a simpler API degraded gracefully, but probably at hyper's level we shouldn't be too clever. That leaves the cleverness to server frameworks.

In case someone doesn't actually want to use this, we could consider having it a builder option. That way, if you're not going to send early hints, we don't construct a waste channel for each request.

Also, I've been thinking about the name. What about informational?

acfoltzer · 2021-02-11T21:19:54Z

In case someone doesn't actually want to use this, we could consider having it a builder option. That way, if you're not going to send early hints, we don't construct a waste channel for each request.

That sounds great. I certainly don't imagine it'll be the common case.

Also, I've been thinking about the name. What about informational?

Much better than my strawman 😆

Kestrer · 2021-03-19T22:28:29Z

I want to suggest a slightly different, fully parameter-based API. It might look something like this:

async fn my_service_fn(req: Request<Body>, mut responder: Responder) -> Result<()> {
    let early_hints = Response::builder()
        .status(103)
        .header("Link", "</style.css>; rel=preload; as=style")
        .header("Link", "</script.js>; rel=preload; as=script")
        .body(())?;
    responder.send_interim(early_hints).await?;
    let resp = todo!("build the final response")?;
    responder.send(resp).await?;
    Ok(())
}

This API provides a more consistent interface than the previous suggested ones (apart from the stream one), since there's just one way through which responses can be sent.

Another advantage is that it would allow request handlers to easily perform work after the response has been sent. Currently in Hyper this can be achieved by spawning a task before returning from the function, but that needs 'static and so isn't always viable. This could potentially reduce latency as destructors and other cleanup work can be run after the time-sensitive window between request and response is over.

Additionally this API could potentially support sending non-'static bodies, which can reduce unnecessary allocation and copying.

vikanezrimaya · 2022-08-31T09:38:41Z

Is there still no community consensus on the API surface? I am interested in experimenting with this.

seanmonstar · 2022-08-31T17:53:37Z

Yep, this still needs an accepted proposal. I think the general leaning is to take the ideas for push and adapt them for informational, and then run through any potential downsides with that design. The more clarity there is in the design proposal, the easier it would be to merge a PR afterwards.

Anyone is welcome to write up a proposal, of course!

vikanezrimaya · 2022-09-01T17:36:11Z

@seanmonstar I am currently drafting a proposal gathering most of the proposed API ideas and evaluating their pros and cons, but the resulting Markdown document seems fairly large. Should I post it here as a comment or maybe open a new issue, PR or create a gist?

vikanezrimaya · 2022-09-01T17:50:57Z

In the end, I decided to publish the resulting document as a gist: https://gist.github.com/vikanezrimaya/037101dc7b28de37ef03a47569213236

The gist above gathers most of the current ideas in one place and tries to evaluate them. Please note that while I touched Hyper before, it was merely as a user of the library (and also via the axum framework), not as a contributor, so I might be wrong in some of my evaluations. If so, please do not hesitate to correct me.

seanmonstar · 2022-09-02T18:03:02Z

Awesome, thanks for writing that up! I'll include some comments here:

Alternate idea: providing a "pusher" as a new argument

This doesn't preserve backwards compatibility. The trait would need to change from call(Req) to call(Req, Pusher).

Alternate idea: Sending a stream of responses

This also doesn't preserve backwards compatibility, since the trait would need to change from call() -> impl Future to call() -> impl Stream.
Async streams are unstable in general, so hyper 1.0 couldn't depend on them. They exist in futures-core, but the proposal to add them to std has the trait changing a fair bit (maybe AsyncIterator, maybe the "keyword generics initiative").

API based on the old Server Push proposal

I do think this is the best option. It's kinda like adding a "pusher" argument, it's just putting it inside the request extensions, and only if enabled. If for completeness sake you wanted to keep the others, I'd put them down in an appendix with a "can't use because X" (though it's also perfectly fine to drop them from the proposal entirely).

hyper::early_hints

I'd suggest a slightly more generic approach, not just Early Hints, but any informational (1xx) response. Could be hyper::informational (or hyper::ext::informational even), or another name that implies those kinds of responses.

Consumes the mutable reference to the request

That should be fine, it's just mutably borrowing the request to "take" the pusher out of the extensions. The request is still owned by the call context.

vikanezrimaya · 2022-09-06T11:21:12Z

@seanmonstar I have updated the document with your suggestions. Should we wait for more community feedback or should I potentially start a draft implementation to have some working code to play around with?

seanmonstar · 2022-09-06T23:10:31Z

@acfoltzer did you want to take a look? This comment has a link to the gist.

DAlperin · 2023-07-03T16:58:01Z

Hey folks! Curious if there has been any movement/consensus around @vikanezrimaya proposal?

gosuwachu · 2023-10-21T11:01:02Z

I am also interested in this. It would be really useful for something like resumable upload protocol: https://datatracker.ietf.org/doc/html/draft-ietf-httpbis-resumable-upload-02

seanmonstar added A-server Area: server. B-rfc Blocked: More comments would be useful in determine next steps. C-feature Category: feature. This is adding a new feature. labels Feb 8, 2021

seanmonstar mentioned this issue Jun 1, 2021

Client support for receiving 1xx informational responses #2565

Open

acfoltzer mentioned this issue Mar 14, 2022

Define the VISION #2772

Merged

ayame113 mentioned this issue Sep 9, 2022

"Deno.serve()" - Better UX for 103 Early Hints on HTTP server denoland/deno#15827

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Ideas for supporting HTTP Early Hints server-side #2426

Ideas for supporting HTTP Early Hints server-side #2426

acfoltzer commented Feb 8, 2021 •

edited

Loading

seanmonstar commented Feb 8, 2021

acfoltzer commented Feb 8, 2021

seanmonstar commented Feb 9, 2021

acfoltzer commented Feb 9, 2021

seanmonstar commented Feb 10, 2021

acfoltzer commented Feb 11, 2021

Kestrer commented Mar 19, 2021

vikanezrimaya commented Aug 31, 2022

seanmonstar commented Aug 31, 2022 •

edited

Loading

vikanezrimaya commented Sep 1, 2022

vikanezrimaya commented Sep 1, 2022

seanmonstar commented Sep 2, 2022

vikanezrimaya commented Sep 6, 2022

seanmonstar commented Sep 6, 2022

DAlperin commented Jul 3, 2023

gosuwachu commented Oct 21, 2023

Ideas for supporting HTTP Early Hints server-side #2426

Ideas for supporting HTTP Early Hints server-side #2426

Comments

acfoltzer commented Feb 8, 2021 • edited Loading

Channel/stream-based interfaces

Service function channel argument

Service function response stream return value

Functional interface

Extension to body types

seanmonstar commented Feb 8, 2021

acfoltzer commented Feb 8, 2021

seanmonstar commented Feb 9, 2021

acfoltzer commented Feb 9, 2021

seanmonstar commented Feb 10, 2021

acfoltzer commented Feb 11, 2021

Kestrer commented Mar 19, 2021

vikanezrimaya commented Aug 31, 2022

seanmonstar commented Aug 31, 2022 • edited Loading

vikanezrimaya commented Sep 1, 2022

vikanezrimaya commented Sep 1, 2022

seanmonstar commented Sep 2, 2022

vikanezrimaya commented Sep 6, 2022

seanmonstar commented Sep 6, 2022

DAlperin commented Jul 3, 2023

gosuwachu commented Oct 21, 2023

acfoltzer commented Feb 8, 2021 •

edited

Loading

seanmonstar commented Aug 31, 2022 •

edited

Loading