Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add basic redirect of input #164

Open
marcelo321 opened this issue Sep 17, 2024 · 0 comments
Open

add basic redirect of input #164

marcelo321 opened this issue Sep 17, 2024 · 0 comments

Comments

@marcelo321
Copy link

echo https://paypal.com | hakrawler gives no output.

we need to make it follow the basic redirect for hakrawler to work.. and could be prevented if we just make it follow the initials redirects

# curl https://paypal.com -IL
HTTP/2 301 
date: Tue, 17 Sep 2024 23:03:42 GMT
location: https://www.paypal.com/
server: ECAcc (agc/7F3B)
content-length: 0

HTTP/2 302 
server: Varnish
retry-after: 0
location: https://www.paypal.com/us/home
accept-ranges: bytes
date: Tue, 17 Sep 2024 23:03:42 GMT
via: 1.1 varnish
strict-transport-security: max-age=63072000; includeSubDomains; preload
x-served-by: cache-pdk-kfty2130050-PDK
x-cache: HIT
x-cache-hits: 0
server-timing: content-encoding;desc="",x-cdn;desc="fastly"
content-length: 0

can we implement this so we don't have to do a curl request before doing hakrawler???

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant