Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support for pluggable backend storage #301

Open
tookko opened this issue Jan 25, 2022 · 1 comment
Open

Support for pluggable backend storage #301

tookko opened this issue Jan 25, 2022 · 1 comment

Comments

@tookko
Copy link

tookko commented Jan 25, 2022

Currently Karapace requires a Kafka cluster for storing schemas. While the topic name can be changed, it creates management / mapping overhead in cases where multiple schema registries are needed, or when schemas are needed outside Kafka, or when dealing with limited HW resources.

Suggestion is to abstract backend storage details and allow different storage plugins to be developed (e.g. relational DB or k/v store).

@jjaakola-aiven
Copy link
Contributor

The https://github.com/Aiven-Open/karapace/blob/main/karapace/in_memory_database.py has an API that can be extended for pluggable backend.

The API can be extracted to abstract base class. The point where some extra care is required is the get_schema_id function, when relational database can provide monotonically increasing identifier. Also some locking that protects the schema id generation can also be removed when using database as storage.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants