Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Compress search database to make grains much smaller #58

Open
orblivion opened this issue Nov 8, 2023 · 0 comments
Open

Compress search database to make grains much smaller #58

orblivion opened this issue Nov 8, 2023 · 0 comments
Labels
search Related to searchable items on the map

Comments

@orblivion
Copy link
Owner

orblivion commented Nov 8, 2023

I realized that even with the base data (major cities, before any downloaded areas) it's already 183 megs. But if you gzip search.db it's 16 megs. No wonder, it's a bunch of text.

I bet there's an sql feature or plugin we could use to make it smaller. Especially since it's read-only 99% of the time. The goal is to find and implement such a thing for the search database.

Though if there's a notable performance hit for reading, it would give me pause. In that case maybe we could look into something awful like decompressing on startup.

Or maybe we just need to run VACUUM or set auto_vacuum. I'd guess auto_vacuum only happens on writes. I just tried it and it at least cuts size in half. https://www.sqlite.org/lang_vacuum.html

@orblivion orblivion added the search Related to searchable items on the map label Nov 8, 2023
@orblivion orblivion modified the milestone: Launch Nov 8, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
search Related to searchable items on the map
Projects
None yet
Development

No branches or pull requests

1 participant