Add forward search capability for SQLite databases #3273
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Also define a custom set of operators in preparation of differences in implementation.
Connecting is reasonably expensive because the spatialite extension needs to be loaded. Disable pooling for tests because there is some memory leak when quickly opening and closing QueuePools with sqlite connections.
Use JSON arrays which can have mixed types and therefore have a more logical structure than separate arrays. Avoid JSON dicts because of their verboseness.
A non-index version of ST_DWithin is not necessary. ST_Distance can be used for that purpose. Index use for intersects can be covered with a simple parameter.
Use function classes which can be instantiated directly.
Also improves on the decision if an indexed is used or not.
The first parameter is in fact the self parameter referring to the function class.
Requires to mark the databse r/w when it is newly created in the convert function.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This adds the remaining search capabilities to SQLite databases including category search.
SQLite has neither an intarray type nor GIN indexes. Both are needed for Nominatim's central search index. The SQLite implementation uses an explicit reverse lookup table instead and uses a custom type that saves intarrays in text fields.
This PR also contains some cleanup of the SQLAlchemy wrapper code. Types that are different between SQLite and PostgreSQL are not properly wrapped in a single class with dynamic instantiation according to the database used. Geometry functions now clearly state when they should use an index. The typing of @compile functions should now be more correct.
Finally, connection pools are enabled for SQLite. SQLAlchemy doesn't do that by default because connecting to a SQLite database is superfast. However, given that we need to load the spatialite extension, the delay in opening a new connection is noticable for Nominatim. The old behaviour can be reenabled by setting NOMINATIM_API_POOL_SIZE to 0.