eval: add a rewrite module to handle queries with namespaces. #1675
Conversation
atlas-eval/src/test/scala/com/netflix/atlas/eval/stream/EvaluatorSuite.scala
Outdated
Show resolved
Hide resolved
atlas-eval/src/main/scala/com/netflix/atlas/eval/stream/EvaluatorImpl.scala
Outdated
Show resolved
Hide resolved
atlas-eval/src/main/scala/com/netflix/atlas/eval/stream/EvaluatorImpl.scala
Outdated
Show resolved
Hide resolved
atlas-eval/src/main/scala/com/netflix/atlas/eval/stream/NamespaceRewriter.scala
Outdated
Show resolved
Hide resolved
atlas-eval/src/main/scala/com/netflix/atlas/eval/stream/NamespaceRewriter.scala
Outdated
Show resolved
Hide resolved
atlas-eval/src/main/scala/com/netflix/atlas/eval/stream/NamespaceRewriter.scala
Outdated
Show resolved
Hide resolved
| } | ||
| case _ => failWithBody(promise, response) | ||
| } | ||
| case Failure(ex) => |
There was a problem hiding this comment.
This is typically handled with settings on the pool:
https://pekko.apache.org/docs/pekko-http/current/client-side/host-level.html#retrying-a-request
There was a problem hiding this comment.
Ah that's how to do it, thanks.
There was a problem hiding this comment.
#1677 updates the helper to have a flow option and instrument it better.
I'll have to think about the degradation a bit. Will need to ensure that the rewrite call failing overall will have no impact other than preventing data source changes and that those would get picked up automatically when it recovers.
There was a problem hiding this comment.
Caching the rewrites for a reconnect? That's the only time we'd need to worry about it right? Otherwise if a new eval was submitted the caller would need to retry.
atlas-eval/src/main/scala/com/netflix/atlas/eval/stream/NamespaceRewriter.scala
Outdated
Show resolved
Hide resolved
atlas-eval/src/main/scala/com/netflix/atlas/eval/stream/NamespaceRewriter.scala
Outdated
Show resolved
Hide resolved
manolama
force-pushed
the
tweaks
branch
2 times, most recently
from
a3cd8bd
to
2598f1d
Compare
If configured via `atlas.eval.stream.rewrite-url`, data source URIs are sent to a rewriting service. Responses with errors are then dropped and the caller notified. Valid rewrites continue processing.
Some uses, e.g. the atlas-stream case, entail updating the data sources for a flow when new subscriptions are added or existing subscriptions dropped. This will return cached rewrites for individual queries in the event the rewrite backend is unavailable. Calls will continue for the rewrite service until a successful response is received, at which point it will update the cache and forward the rewrites downstream. f
DataSource URIs are sent via HTTP (with retries) and if they incorporate namespace predicates, the service will rewrite them to route to the proper host, removing the namespace predicate.