Imroc can't read response correctly #350
Comments
|
I tried to reproduce: package main
import (
"github.com/imroc/req/v3"
)
func main() {
client := req.C().DevMode()
headers := map[string]string{
"Connection": "keep-alive",
"sec-ch-ua": `"Chromium";v="124", "Microsoft Edge";v="124", "Not-A.Brand";v="99"`,
"sec-ch-ua-mobile": "?0",
"sec-ch-ua-platform": "Windows",
"Upgrade-Insecure-Requests": "1",
"User-Agent": `Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/124.0.0.0 Safari/537.36 Edg/124.0.0.0`,
"Accept": `text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7`,
"Sec-Fetch-Site": "same-origin",
"Sec-Fetch-Mode": "navigate",
"Sec-Fetch-User": "?1",
"Sec-Fetch-Dest": "document",
"Referer": "https://www.loves.com/en/lovesconnect",
"Accept-Encoding": "gzip, deflate, br, zstd",
"Accept-Language": "es-419,es;q=0.9,es-ES;q=0.8,en;q=0.7,en-GB;q=0.6,en-US;q=0.5,es-CO;q=0.4",
}
api := "https://www.loves.com/api/sitecore/Login/LoyaltyAuth0Login"
req := client.R().SetHeaders(headers)
req.MustGet(api)
}Got 403 response: 2024/05/06 09:51:56.430997 DEBUG [req] HTTP/1.1 GET https://www.loves.com/api/sitecore/Login/LoyaltyAuth0Login
GET /api/sitecore/Login/LoyaltyAuth0Login HTTP/1.1
Host: www.loves.com
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/124.0.0.0 Safari/537.36 Edg/124.0.0.0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7
Accept-Encoding: gzip, deflate, br, zstd
Accept-Language: es-419,es;q=0.9,es-ES;q=0.8,en;q=0.7,en-GB;q=0.6,en-US;q=0.5,es-CO;q=0.4
Connection: keep-alive
Referer: https://www.loves.com/en/lovesconnect
Sec-Ch-Ua: "Chromium";v="124", "Microsoft Edge";v="124", "Not-A.Brand";v="99"
Sec-Ch-Ua-Mobile: ?0
Sec-Ch-Ua-Platform: Windows
Sec-Fetch-Dest: document
Sec-Fetch-Mode: navigate
Sec-Fetch-Site: same-origin
Sec-Fetch-User: ?1
Upgrade-Insecure-Requests: 1
HTTP/1.1 403 Forbidden
Content-Type: text/html
Cache-Control: no-cache, no-store
Connection: close
Content-Length: 810
X-Iinfo: 5-2087529-0 0NNN RT(1714960315772 74) q(0 -1 -1 1) r(0 -1) B16 U18
Set-Cookie: visid_incap_727601=Swrj3DYjSyavkMn5fDPXm7s3OGYAAAAAQUIPAAAAAADcMhJ3MQEULe6MqQMgb7pl; expires=Mon, 05 May 2025 16:58:30 GMT; HttpOnly; path=/; Domain=.loves.com
Set-Cookie: incap_ses_627_727601=YVHfakqHNxYzQKk2OY2zCLs3OGYAAAAA03Q5eFoEgn3R9BBdcesxiw==; path=/; Domain=.loves.com
<html style="height:100%"><head><META NAME="ROBOTS" CONTENT="NOINDEX, NOFOLLOW"><meta name="format-detection" content="telephone=no"><meta name="viewport" content="initial-scale=1.0"><meta http-equiv="X-UA-Compatible" content="IE=edge,chrome=1"><script src="/ee-not-sword-not-forgotter-and-where-What-hee-th" async></script></head><body style="margin:0px;height:100%"><iframe id="main-iframe" src="/_Incapsula_Resource?CWUDNSAI=23&xinfo=5-2087529-0%200NNN%20RT%281714960315772%2074%29%20q%280%20-1%20-1%201%29%20r%280%20-1%29%20B16%20U18&incident_id=627000050004344883-10128756990738757&edet=16&cinfo=ffffffff&rpinfo=0&mth=GET" frameborder=0 width="100%" height="100%" marginheight="0px" marginwidth="0px">Request unsuccessful. Incapsula incident ID: 627000050004344883-10128756990738757</iframe></body></html>
Can you show the reproducable code? Here what I know: If you set |
|
hi i use your code and i get this `:status: 403 �8[→����►u�.������/��~)O%N6@�+ng�↑wB�E�s��,=@↑►- ▼�<2↨��2�k��� I don't understand why the text appeared to you while it didn't to me, I am using the version github.com/imroc/req/v3 v3.43.3 |
|
Your response header is different: content-encoding: brIt means you need uncompress the response body using br (standrard library If you don't want to uncompress manully, you should not set |
|
It worked by removing the header encoding, thank you very much for your time, although if it is not too much to ask, it would be good if the library or imroc/req module did it automatically, that is, it would detect the encoding and do this automatically, it would be a plus for you library, think about it, since if I remove the encoding the response would not be compressed so it would weigh more I understand, in any case I appreciate your help and the time and if you can add automatic encoding it would be very good, since it seemed strange to me that you Those strange letters didn't appear to you and they did to me and you are using encoding, Well I just want to say thank you for what you do. |
ghost
closed this as
|
req will uncompress automatically by default, but only gzip algorithm. If |
This becomes problematic when trying to copy a browsers fingerprint. The default accept-encoding will lead to blocks on some sites due to not matching the stock browser. If req is able to decode everything we're asking for the accept-encoding, it should decode it unless explicitly told not to. |
|
I've made a PR example of how this could be handled. I would appreciate if you could add this or it's a huge headache. |
Hello, I am doing a normal get and when trying to display the response it appears in strange characters
I think it is "zstd"
Accept-Encoding": "gzip, deflate, br, zstd
but I'm not sure and I don't know how to solve it, here is my get
client := req.C()
headers := map[string]string{
"Connection": "keep-alive",
"sec-ch-ua":
"Chromium";v="124", "Microsoft Edge";v="124", "Not-A.Brand";v="99","sec-ch-ua-mobile": "?0",
"sec-ch-ua-platform":
"Windows","Upgrade-Insecure-Requests": "1",
"User-Agent":
Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/124.0.0.0 Safari/537.36 Edg/124.0.0.0,"Accept":
text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7,"Sec-Fetch-Site": "same-origin",
"Sec-Fetch-Mode": "navigate",
"Sec-Fetch-User": "?1",
"Sec-Fetch-Dest": "document",
"Referer": "https://www.loves.com/en/lovesconnect",
"Accept-Encoding": "gzip, deflate, br, zstd",
"Accept-Language": "es-419,es;q=0.9,es-ES;q=0.8,en;q=0.7,en-GB;q=0.6,en-US;q=0.5,es-CO;q=0.4",
}
req := client.R().SetHeaders(headers)
response := req.MustGet(api)
fmt.Println(response.String())
Also if I use client.DevMode() the same thing happens too, help please
The text was updated successfully, but these errors were encountered: