[WIP] #16 add geocode processor #88
Conversation
| @@ -124,6 +125,15 @@ entities-sync: | |||
| - run: dump.to_path | |||
| parameters: | |||
| out-path: ../data/clearmash/new-entities-sync | |||
| # dump geocode_location table | |||
| - run: dump.to_sql | |||
There was a problem hiding this comment.
I think it will be better to have the sync processor update the geocode_entities table
| @@ -149,3 +159,23 @@ entities-delete: | |||
| id-field: item_id | |||
| source: clearmash | |||
| delete-all-input: true | |||
|
|
|||
| geocode: | |||
There was a problem hiding this comment.
this pipeline should be part of a common pipeline spec file, it's not specific to clearmash
| "house_number": ["street_number", "house_number"], | ||
| "city": ["city"], | ||
| "country": ["country"], | ||
| "po_box": ["pob", "zipcode", "postal_code", "pob_postal_code"],} |
There was a problem hiding this comment.
this stuff is specific to budget key - it's irrelevant for us
There was a problem hiding this comment.
I thought maybe to leave it incase we'd like to extend location to other items in the future, like a detailed address of a photo subject (e.g painting, synagogue, person's residence and so on). But I guess it could be added if and when needed.
There was a problem hiding this comment.
all this logic is specific only to open budget, even if we did need street number and this stuff - this code will not work..
|
|
||
| # returns a location string to geocode by | ||
| # returns None if no suitable location string is found | ||
| def get_location_string(self, entity_details): |
There was a problem hiding this comment.
this function is not needed, we get the location string as-is from the source data
There was a problem hiding this comment.
should I change this processor to extend the BaseProcessor, like the rest?
There was a problem hiding this comment.
would be better to have all processor use similar logic, but not critical
sync should add location field with lat/long (using geocoding of the place name)
#16