-
-
Notifications
You must be signed in to change notification settings - Fork 61
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Create a unified Snuba consumer #1670
Comments
Here it is @JoshFerge. :) |
This is too much work that our team does not have time for at the moment. |
I think we should eventually do that but it's too early to create tickets. I think with lightweight deletes, EAP mutations, subscriptions the code became a lot more duplicated, the friction for creating new consumers in snuba feels very high, and the refactor in this issue would solve it. we did a similar project in sentry where we have way more consumers and it took me less than a quarter. I think it can be done in a limited fashion for rust only. |
Currently we have many Snuba consumers and seems like we are going to add more as we add more features around Snuba. This poses a maintenance and synchronization issue for on-premise as the growing number of consumers is hard to maintain. People also keep forgetting to add new consumers to the self-hosted repo, causing issues or missing features (this has happened twice now).
Proposal: introduce a
consumers
command to Snuba CLI that runs all core consumers with default settings (the settings we use in self-hosted) using multiprocessing so the consumers are still logically separate./cc @fpacifici @lynnagara @chadwhitacre
The text was updated successfully, but these errors were encountered: