-
Notifications
You must be signed in to change notification settings - Fork 24
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Florida scraper #24
base: master
Are you sure you want to change the base?
Florida scraper #24
Conversation
Hey Ed, this is amazing, what a terrible site to parse, heh. The scraper seems to be choking on two issues:
You mind taking a look? :) |
In case opts.limit is not provided, specify a default value 9999
Hey Adam Thanks for pointing out that --limit might not be supplied. I added a default value for --limit and tested it by passing in an empty object for opts. However I am not able to replicate the issues you described. I tried running with various values for --limit and also with no limit and it would not fail. What data did you use? |
If opts.limit not supplied use 9999
Sorry for the delay, I'll try to be a bit more responsive so we can get this solved. Here's what's happening:
|
The page is essentially a mass of text and there are no obvious hard-and-fast rules as to when a particular element would be included or excluded. I still have not been able to find an example that makes it fail, but I recoded where you identified so if an element label is not found then it will not try to 'trim' the non-existent text. |
Here is a scraper for Florida. I hope it helps.
Ed