Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

HI blocking access #600

Closed
stucka opened this issue Jan 11, 2024 · 1 comment
Closed

HI blocking access #600

stucka opened this issue Jan 11, 2024 · 1 comment

Comments

@stucka
Copy link
Contributor

stucka commented Jan 11, 2024

Hawaii appears to be blocking access to requests directly, throwing a "blocked" into the title tag. If requests offers up a page with a regular browser's User-Agent, Hawaii throws a challenge error to non-Javascript-enabled browsers.

stucka added a commit that referenced this issue Jan 17, 2024
stucka added a commit that referenced this issue Jan 17, 2024
stucka added a commit that referenced this issue Jan 19, 2024
Scraper will break, transformer will come back.
stucka added a commit that referenced this issue Jan 19, 2024
@stucka
Copy link
Contributor Author

stucka commented Jan 30, 2024

HI seems to have re-enabled access. Code in #605 now allows a quick one-line change back to Google's cache.

But something else broke in the code -- I think problems with tracking location in the array that @Ash1R had set up -- and so I've replaced some of the parsing code to go line by line through the data, while building a list of dictionaries. Closing this.

@stucka stucka closed this as completed Jan 30, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant