Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR addresses #23 using a method intimated at by one of the llhttp authors in this issue to avoid relying too heavily on the node internals. We could, instead, use the parser directly, either by importing it via _http_common or by using the WASM bindings but this seemed like the method with the least implementation and maintenance overhead.
If we were to import the parser directly, http-parser-js provides example code that is nearly compatible with llhttp, though the event callbacks now have slightly different parameters (for example kOnHeadersComplete now receives all of its properties as individual arguments rather than a single object with properties). A more definitive implementation can be found at _http_server (for requests) and _http_client (for responses). These pull from a parsers pool that we might also tap into. For the sake of completeness, the last (and worst) option is to include llhttp using the internal bindings, like so:
In implementing this PR, I've chosen to make
session.request
andsession.response
return promises (example) since the parsing method itself is asynchronous, though I'm on the fence about this and we could easily await this only within the proxy such that those properties have already been assigned by the time theinject
functions are called.Another implementation note is the use of "mirror" sockets that subscribe to and re-emit data events from their parent (
_src|dst
) sockets. I had initially thought I could simply feed in the parent sockets to node'shttp.createServer
andhttp.request
functions but doing so appears to hijack those sockets and causes the proxy to hang.One challenge in all of this (and something I suspect is an issue in the library today) is accounting for chunked headers. Using node's parsers in the way implemented here means the parsed headers are only available once the parser triggers kOnHeadersComplete but the spec allows headers to come in as chunks and you can see that node buffers them in the kOnHeaders handler. This means that transparent-proxy could fail or hang here when partial headers are sent.
Another side effect of this PR is that
session.request|response
no longer concatenate theirbody
s. The idea here is that the user would instead subscribe to data events (ex:session.request.on('data', () =>
) as one would do with node'shttp.request
orhttp.server
. The data is also, of course, available via transparent-proxy'sinject
hooks. I think there's benefit in the familiarity of using node's existing style for this but understand if the concatenatedbody
ends up being important; we could always add it as a property to theIncomingMessage
object that's used to represent node's parsed requests and responses. That said, I'm unsure what access to a partially concatenated body gets the user.All that said, feel free to consider this a first pass. Feedback welcomed.