Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for SQLite DBs in frontend: reverse #3233

Merged
merged 16 commits into from
Oct 24, 2023

Conversation

lonvia
Copy link
Member

@lonvia lonvia commented Oct 24, 2023

This is the first part of enabling Nominatim to be run from a SQLite (or rather SpatialLite) database.

The major changes are:

  • add a new CLI function nominatim convert to dump an existing PostGIS database into an SQLite file,
  • adapt the API functions reverse(), details() and lookup() to work with SQLite,
  • change unit tests for these API functions, so that they are run against SQLite and PostGIS respectively
  • change BDD API tests so that they can be run against a SQLite database (SQLite file needs to created manually by running nominatim convert against the API tests database right now)

CI does run the unit tests for SQLite but not the BDD tests.

SQLite support is highly experimental and currently undocumented. To try it out, use the new convert function to create a databse. Then set the database DSN to sqlite:dbname=<filename> to use it. It also works with the web frontend and nominatim serve when using one of the Python engines.

Differences between PostgreSQL and SQLite are hidden behind custom SQLAlchemy functions, so the code should not interfere with normal operations. If other interference shows up in production, I reserve the right to pull the code out again.

lonvia added 16 commits October 23, 2023 17:19
Remove all information not strictly used by the frontend as well as
any index information. This will make it easier to create a SQLite
database from the schema.
Includes porting unit tests.
The database must currently be created by hand and the name handed
in via -DAPI_TEST_DB='sqlite:...'.
Transfer is_address_point into SQLAlchemy function, so that
json.has_key() can use the older json_extract() function.
And work around broken Distance function.
A boolean type makes the SQLite dialect produce a costruct like
'func() = 1' in WHERE condition. While syntactically correct, it tends
to confuse the query planer.
@lonvia lonvia merged commit ca782e2 into osm-search:master Oct 24, 2023
8 checks passed
@lonvia lonvia deleted the support-for-sqlite branch October 25, 2023 07:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant