Support custom CONNECT headers (both request and response) during HTTPS proxy requests #330
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Note: Just throwing this idea out there -- I implemented these changes for my own use case, but I'm happy to rework the implementation if it's not deemed suitable for merging. Let me know.
The Issue
Some proxies accept custom headers, for things like switching IPs, excluding certain IPs, or specifying custom behaviors (like retrying requests when certain codes are returned). Over normal HTTP, these can be sent along with the rest of your headers, and the proxy will filter the headers before passing the request along. Likewise, the proxy might inject a custom header into the response (such as one indicating the IP your request was passed through).
However, everything changes over a secured connection. When connecting to a proxy over HTTPS, this gem makes use of HTTP tunneling via the 'CONNECT' method. The underlying request and response are encrypted. As a result, the proxy is unable to read any headers you send with the underlying request or inject headers into the underlying response.
So adding custom proxy-specific headers to a HTTPS request looks something like this:
And a response from the proxy (with a custom header) looks like this:
The problem is that, aside from
Proxy-Authorization
, this gem does not support adding custom headers (likeX-My-Custom-Header
andX-Custom-Proxy-Response
above).The Change(s)
This PR adds a parameter to the
via
method which allows you to specify custom headers intended for use by the proxy:Note that if the underlying request is 'http' instead of 'https', there is no
CONNECT
tunneling, so the proxy-specific headers will be merged into the requests's primary headers.Similarly, if the proxy returns any custom headers (as shown above), they are made available as follows:
Note: I considered transparently merging these into the main request headers, but I worried that this would cause strange behavior. Either a proxy could overwrite a header you expect from the remote server, or the remote server could overwrite a header you expect from the proxy, and I wasn't sure how to handle these cases. If you have any suggestions I'd be happy to rework the behavior.
Questions?
Does all of this make sense? Let me know if you have any questions, ideas, suggestions, etc.