From 46b4a73145fc0b0723a8ae0087b1f7ece3de3acc Mon Sep 17 00:00:00 2001 From: Benjamin Trent Date: Tue, 16 Apr 2024 08:53:41 -0400 Subject: [PATCH] Add note about random sampler consistency (#107479) (#107530) --- .../bucket/random-sampler-aggregation.asciidoc | 13 ++++++++++++- 1 file changed, 12 insertions(+), 1 deletion(-) diff --git a/docs/reference/aggregations/bucket/random-sampler-aggregation.asciidoc b/docs/reference/aggregations/bucket/random-sampler-aggregation.asciidoc index efd025ed67ee6..9fc533104024d 100644 --- a/docs/reference/aggregations/bucket/random-sampler-aggregation.asciidoc +++ b/docs/reference/aggregations/bucket/random-sampler-aggregation.asciidoc @@ -94,6 +94,17 @@ higher sampling rates, the relative error is still low. NOTE: This represents the result of aggregations against a typical positively skewed APM data set which also has outliers in the upper tail. The linear dependence of the relative error on the sample size is found to hold widely, but the slope depends on the variation in the quantity being aggregated. As such, the variance in your own data may cause relative error rates to increase or decrease at a different rate. +[[random-sampler-consistency]] +==== Random sampler consistency + +For a given `probability` and `seed`, the random sampler aggregation is consistent when sampling unchanged data from the same shard. +However, this is background random sampling if a particular document is included in the sampled set or not is dependent on current number of segments. + +Meaning, replica vs. primary shards could return different values as different particular documents are sampled. + +If the shard changes in via doc addition, update, deletion, or segment merging, the particular documents sampled could change, and thus the resulting statistics could change. + +The resulting statistics used from the random sampler aggregation are approximate and should be treated as such. [[random-sampler-special-cases]] ==== Random sampling special cases @@ -105,6 +116,6 @@ for a bucket is `10,000` with `probability: 0.1`, the actual number of documents An exception to this is <>. Unique item counts are not suitable for automatic scaling. When interpreting the cardinality count, compare it -to the number of sampled docs provided in the top level `doc_count` within the random_sampler aggregation. It gives +to the number of sampled docs provided in the top level `doc_count` within the random_sampler aggregation. It gives you an idea of unique values as a percentage of total values. It may not reflect, however, the exact number of unique values for the given field.