-
-
Notifications
You must be signed in to change notification settings - Fork 2.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Url gets manipulated in the wrong way #2188
Comments
@asvetlov I think yarl should support raw(unsafe) urls |
Isn't the main issue here that it unquotes characters? That aiohttp changes characters into their % form where needed makes sense, the other way around: not really. |
@fafhrd91 agree. |
@pvanderlinden normalizing urls by default on client is good idea. |
The issue seem to arise with anything which uses AWS cloudfront when I was looking around for issues. I wasn't aware of the |
I think it is a bug to undo pre-encoded values supplied by the user, as it can break hmac style authorisation headers that have been pre-computed. The operation "encode" should not be doing an opportunistic decode just because it can. |
@shuckc I don't get your point. |
Hi Andrew, I used python 3's urllib to quote a json snippet passed as a GET argument. urllib encodes the ":" as %3A, the partially encoded url is then used to "sign" the request by adding an extra header. Yarl inside aiohttp was then reversing the encoding of %3A back to ":" before sending the request, so the api-signature header no longer matched the request. The server checks the signature against the request value and failed. From what I have learned subsequently, colon is a permissible character in URLs, so it is actually urllib that is at fault here for encoding it.
I fixed this first by
And later by removing urllib :-) |
Yes, your fix is totally correct. |
This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a [new issue] for related bugs. |
Long story short
Url's passed into aiohttp gets changed in certain ways the servers don't always accept. In this case a
:
in the url which is quoted to%3A
already, will get unquoted by aiohttp.Expected behaviour
%3A
should stay%3A
Actual behaviour
%3A
in the path get's translated to:
causing errorsSteps to reproduce
pass in a url with
%3A
. Example url:https://thumbor-static.factorymedia.com/_9eOzvEVNwVVRsG0qxCqt-9-KxI=/1280x720/smart/http%3A%2F%2Fcoresites-cdn.factorymedia.com%2Fsidewalk%2Fwp-content%2Fuploads%2F2017%2F08%2Ffifty-fifty-twenty-years.jpg
works in your browser, and in python requests, doesn't work in aiohttp.Your environment
aiohttp 2.2.5, python 3.5, linux
The text was updated successfully, but these errors were encountered: