Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Recover data from Israel #1100

Merged
merged 8 commits into from
Apr 8, 2024
Merged

Recover data from Israel #1100

merged 8 commits into from
Apr 8, 2024

Conversation

majesticio
Copy link
Contributor

Adapter upgraded, and updated to use new data URL.

Adds data from ~182 stations.

@majesticio majesticio requested a review from caparker April 3, 2024 22:23
Copy link
Collaborator

@caparker caparker left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Lets filter the data down a little, or maybe there is a timestamp error somewhere. But this is what I got the first time I ran it

182 locations from 2019-01-01T08:55:00.000Z - 2024-04-04T23:50:00.000Z | 
Parameters for israel-envista {
"co":{"count":16,"errors":3,"max":1,"min":0,"nulls":0},
"no":{"count":164,"errors":0,"max":0.7908,"min":-9.999,"nulls":0},
"no2":{"count":165,"errors":0,"max":0.0456,"min":-9.999,"nulls":0},
"nox":{"count":165,"errors":0,"max":0.7958999999999999,"min":-9.999,"nulls":0},
"o3":{"count":88,"errors":0,"max":0.1971,"min":-9.999,"nulls":0},
"pm10":{"count":62,"errors":5,"max":263,"min":0,"nulls":0},
"pm25":{"count":112,"errors":4,"max":95,"min":0,"nulls":0},
"so2":{"count":107,"errors":0,"max":9.999,"min":-9.999,"nulls":0}}

Notice the min timestamp is in 2019? How about we make it default to the past 6hrs or something like that.

And then notice the various -9.999 and 9.999's ? Could you find out if those are values that are starting out as -9999 and 9999 and then going through our units converter (divided by 1000)? If they are we need to make sure they are not converted because if they are submitted as 9.9 and -9.9 I think our system will treat them as real values and not error codes. If they are not being divided we will need to figure out another way to deal with them.

@majesticio
Copy link
Contributor Author

I have added updates to unifyMeasurementUnits that will ignore values of -9999 or 9999 and them pass them along, as these are known error codes, and dividing them by 1000 for example to convert ppb to ppm will create values like -9.999 which may be treated as a measurement rather than an error

@majesticio
Copy link
Contributor Author

The API provides latest measurements every 5 minutes. The adapter has been updated to reflect this, and I am updating the docs as well.

@caparker caparker merged commit fd4860c into main Apr 8, 2024
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants