-
Notifications
You must be signed in to change notification settings - Fork 10
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add GA scraper #324
Add GA scraper #324
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This looks great. Thank you for stepping up and taking on an important state!
One thing we're trying to be more consistent about is saving all of the raw files we gather into the cache, before any manipulation happens. Would you mind adding that to your submission?
Earlier this morning I added a simple example that I think will work for you to the docs.
Okay. Now writes files to cache, reads prior years from cache. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for diving in on the cache, @chriszs! I've raised a few minor issues next. Let me know what you think about them.
I think we might be ready for @zstumgoren to weigh in.
Co-authored-by: Ben Welsh <[email protected]>
At a glance, this looks fantastic! Thanks for adding in the caching behavior @chriszs !! Looks ready to merge if @palewire agrees. Somewhat related, but I'd like for us to hash out and formalize our caching policy in a new issue. I've been mulling some ideas about different caches approach in various situations, but some of the questions are more editorial judgement-y than technical. Wouldn't mind having us discuss as a group. 'll file a question Issue today and CC you both to get your thoughts. Meantime, thanks again @chriszs for shipping us GA!! |
Agreed we need to settle a caching policy. Chris set a smart example here. |
We're in. Thank you, @chriszs! |
Adds a scaper for Georgia WARN Act data.
Steps to test
Should create
exports/ga.csv
Closes #63