-
Notifications
You must be signed in to change notification settings - Fork 41
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
common: improve caching in API/SQL enrichments #747
Comments
pondzix
added a commit
that referenced
this issue
Feb 16, 2023
pondzix
added a commit
that referenced
this issue
Feb 16, 2023
pondzix
added a commit
that referenced
this issue
Feb 17, 2023
pondzix
added a commit
that referenced
this issue
Feb 17, 2023
spenes
pushed a commit
that referenced
this issue
Apr 13, 2023
spenes
pushed a commit
that referenced
this issue
Apr 26, 2023
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
If we get an error from external API/database (e.g. because it's down), such error is cached by enrich. It's cached because we don't want to overload API/database experiencing some issues.
TTL for errors stored in the cache is the same as for successful results. If this TTL is some big integer (which is often reasonable in case of successful/valid results) we can get stuck with errors in the cache for while. So even though external API, which originally produced errors, is now working correctly we don't know that cause we use errors stored in the cache.
Potential improvements:
Those improvements can be implemented for both API and SQL enrichments.
It all resembles behaviour of circuit-breaker. Adding proper circuit-breaker implementation (most likely coming from some external lib) could be problematic as we still have to support
Id
monad, hence improvements for existing solution relying on errors caching.The text was updated successfully, but these errors were encountered: