'stream' vs 'request' API

[ronag]

In http2 we got a new API for handling requests where instead of emitting a 'request' event with a request and response object we emit 'stream' with a stream object.

• Is the ecosystem interested into moving towards this API?
• What are the advantages/disadvantages?
• How can we help with this transition? Would a forwards compat layer be possible/helpful/make sense?

[wesleytodd]

Is the ecosystem interested into moving towards this API?

I am not sure I understand this question. The frameworks who wish to support http2/3 will have to use this API. They obviously don't have to forward this onto their consumers. Is that what you mean when you ask this question, will frameworks expose this to their consumers?

What are the advantages/disadvantages?

The API itself is really dictated by the differences in the protocols AFAIK. I am not sure you could design an API around a stateful connection which just has request cycle like the stateless http1 API (or vice versa). Is there something more specific you wish to drill down on this question?

How can we help with this transition? Would a forwards compat layer be possible/helpful/make sense?

This is something I really think we might be able to do something about! I had some ideas around this I wrote up here: expressjs/discussions#82 (comment)

I never got feedback on that as a starting point for Express to move toward a compat layer, so if you have time and want to chime in over there it would be awesome! I think beyond Express there is a great opportunity for us to push much of the shared API into core (or a userland compat layer) which would enable better code sharing between frameworks.

[ronag]

I am not sure I understand this question. The frameworks who wish to support http2/3 will have to use this API.

The problem I see here is that no frameworks are moving towards this. They rather use the http2 compat layer to http1 since they want their frameworks to work for both use cases.

One way to reverse this trend would be a forwards compat layer. That way frameworks can move towards this new API without excluding http1 users.

[dougwilson]

To speak from the framework side: I would more say the main concern I would express is that users should be able to write their code without needing to care if a given request is http 1/2/3 unless they are going to specifically take advantage of a feature added in that new http version. Semantically they are all http, and for the current future typical public web servers will be getting http 1 traffic along with http 2.

When I create a Node.js HTTP/2 server and use the stream event, I always have to also use the request event for HTTP/1 requests, and end up with a lot of duplication, so it ends up being easier to simply use the "compat" API and write everything in just a single format. That is my 2c. Having HTTP/1 come in "upgraded" to streams would certainly change the conversation.

I also will add that I don't typically end up writing apps that can afford to only talk HTTP/2; I would bet the conversation would be different for folks who meet that case.

[ronag]

One key question here though, does the "stream" API bring enough significant improvements to justify the extra work of a forwards compat layer?

[dougwilson]

Gotcha. Unfortunately because of the above I just don't have a scenario where I can truly even test it out in a real world scenario, so don't have any feedback on that point.

[ronag]

This becomes relevant in regards to where to prioritise our resources:

• A. 'request', Improving the http2 backwards compat layer (The differences between http2 compatibility API and http[s] node#29829)
• B. 'stream', Investigating possibility of a forwards compat layer.

[wesleytodd]

I think the best long term approach and what is pragmatic right now might be in conflict here. In the long run I think we need to move http2/3 first and then make sure there is graceful degradation. I think that in the short term that is very hard and will require a huge shift in the ecosystem.

So I don't think it answers your question @ronag, but I think we need to do both things. In the long run, a good story in core around how these api's degrade to their older counterparts will not be wasted. But also I cannot see projects like Express moving quickly into this uncharted territory. Does that make sense?

[MRazvan]

If I can five my 2c, most if not all existing frameworks will have to move to a streaming api at one point if they want to take advantage of all the new functionalities exposed by http2/3, ideally it would be transparent to the framework / project what protocol it's using, and they can check for functionalities by querying the stream for capabilities for example ( If at all possible ). The question becomes request + streaming or 'stream' http1 events also for the future. Me as a developer would prefer option 2 as it will simplify development.

Moving to a stream api will break most if not all existing frameworks, so maybe keep existing API's as they are, and do a 'parallel' API for the future, the frameworks choose what to use. I imagine a lot of projects are written on top of the request pattern (projects not using any frameworks) so even if the frameworks change, those projects will very likely not move.

To summarize: I see a lot of advantages for moving to a 'streaming' api, for me as a developer also for node, it can add other protocols that can be easily picked up by the frameworks or anyone using the new API. I also don't see the request pattern going away anytime soon.

[wesleytodd]

As discussed on todays meeting, with the long timeline on this, and no clear intent from frameworks to change anything meaningful soon, we will remove the agenda until more relevant progress is made.