Shards Series API. #3856
Shards Series API. #3856
Conversation
This PR introduces Series Queries Sharding. It does not check the boundaries of ingesters data since I'm assuming grafana#3852 will be merge first. Signed-off-by: Cyril Tovena <[email protected]>
Signed-off-by: Cyril Tovena <[email protected]>
| requests := make([]queryrange.Request, 0, conf.RowShards) | ||
| for i := 0; i < int(conf.RowShards); i++ { | ||
| shardedRequest := *req | ||
| shardedRequest.Shards = []string{astmapper.ShardAnnotation{ | ||
| Shard: i, | ||
| Of: int(conf.RowShards), | ||
| }.String()} | ||
| requests = append(requests, &shardedRequest) | ||
| } | ||
| requestResponses, err := queryrange.DoRequests(ctx, ss.next, requests, ss.limits) |
There was a problem hiding this comment.
I guess this is where some of the magic happens. Multiple requests are made. Are they going to the same endpoint? How are they load balanced? Also, how is the endpoint defined?
There was a problem hiding this comment.
These are dispatched from the query frontend to queriers according to the sharding factor. The queriers then will request from the store (and ingesters if necessary - determined by the time range of the request since ingesters hold recent data). Both the store and the ingesters now have the ability to discern and filter results down to a specified shard.
There was a problem hiding this comment.
Ok. How are the queries routed to the inhesters?
There was a problem hiding this comment.
frontend -> queriers -> ingester
There was a problem hiding this comment.
And if we have multiple queriers? Is the a load balancer?
This PR introduces Series Queries Sharding. It does not check the boundaries of ingesters data since I'm assuming
#3852 will be merge first.
Signed-off-by: Cyril Tovena [email protected]