Add filtering, normalisation and variants for postcodes #2757
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
The postcodes will only be removed as a 'computed postcode' they are still searchable for the given object.
Adds patterns for countries that have simple numeric-only postcodes.
Moves postcodes that are either in countries without a postcode system or don't correspond to the local pattern for postcodes into a field for a normal address part. Makes them searchable but not as a special address. This has two consequences: they are no longer a skippable part of the address and the postcodes cannot be searched on their own.
If the country code is not part of the mandatory output, the country code filter will do the correct handling.
Now includes all postcodes that have optional parts.
Optional groups are not implemented yet.
Also documents the changes to the SQL functions of the tokenizer.
Includes smaller code fixes found by the tests.
update_postcodes_from_db() needs to do the full postcode treatment in order to derive the correct word table entries.
It can happen for bogus names and this will not get fixed anymore.
Hierarchical postcodes need a different treatment.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This adds a special sanitizer and tokenizer for postcodes. The sanitizer filters the postcodes from the OSM tags, so that only those can pass that conform to the official postcode format of the country. The tokenizer creates variants for postcodes that have optional spaces in them.
Per-country postcode formats can be configured in
settings/country_settings.yaml. There is a new section in the customization documentation that explains the details.The main effect of this change is that only postcodes in the proper format will end up in the location postcode table. Postcodes with typos and bad postcode entries will no longer propagate to other objects. Nominatim also has some special treatment of words recognized as postcodes in a search request. This will now work much better because the recognition as a postcode is based on 'whatever can be found in the location_postcode table'.
Still to do are handling of hierarchical postcodes (#1011) and recognition of unknown postcodes during search (#1452).
Fixes #927. Fixes #1207.