Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Better persistence #10

Open
2 tasks
jgraeger opened this issue Jun 4, 2022 · 0 comments
Open
2 tasks

Better persistence #10

jgraeger opened this issue Jun 4, 2022 · 0 comments

Comments

@jgraeger
Copy link
Member

jgraeger commented Jun 4, 2022

After merging #9, the service will persist data to disk. However, the implementation is very naive. Read times will decrease over time, as for each database query, the service will read through the whole file. Second, everytime a whole new record containing each key for each known service will created. This isn`t a problem per sé und maybe we should leave it like that for simplicity. However, the database file size will grow and grow over time. The persistence layer needs to be improved.

We want to:

  • Significantly improve read performance (most obvious solution: Create a database index)
  • Squash records that are not needed anymore, but keep them accessible in some way. Using an append-only DB is intended.
@ldb ldb mentioned this issue Jun 4, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant