Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cardinality agg on constant_keyword fields returning zero in 8.9+ #99776

Closed
ebeahan opened this issue Sep 21, 2023 · 2 comments · Fixed by #99814
Closed

Cardinality agg on constant_keyword fields returning zero in 8.9+ #99776

ebeahan opened this issue Sep 21, 2023 · 2 comments · Fixed by #99814
Assignees
Labels
:Analytics/Aggregations Aggregations >bug Team:Analytics Meta label for analytical engine team (ESQL/Aggs/Geo)

Comments

@ebeahan
Copy link
Member

ebeahan commented Sep 21, 2023

Elasticsearch Version

8.9.0+

Installed Plugins

No response

Java Version

bundled

OS Version

all

Problem Description

Beginning in 8.9.0, observing cardinality aggs returning zero against constant_keyword mapped fields.

Steps to Reproduce

Performed simple setup of an index with constant_keyword mapping for field test in 8.8.2 and 8.9.0:

PUT test-index
{
  "mappings": {
    "properties": {
      "test": {
        "type": "constant_keyword"
      }
    }
  }
}

PUT test-index/_bulk
{ "index" : { "_index" : "test-index", "_id" : "1" } }
{ "test" : "value1" }



GET test-index/_search
{
  "aggs": {
    "0": {
      "cardinality": {
        "field": "test"
      }
    }
  },
  "size": 0
}

8.8.2 result

{
  "took": 1,
  "timed_out": false,
  "_shards": {
    "total": 1,
    "successful": 1,
    "skipped": 0,
    "failed": 0
  },
  "hits": {
    "total": {
      "value": 1,
      "relation": "eq"
    },
    "max_score": null,
    "hits": []
  },
  "aggregations": {
    "0": {
      "value": 1 <= expected
    }
  }
}

8.9.0 result

{
  "took": 2,
  "timed_out": false,
  "_shards": {
    "total": 1,
    "successful": 1,
    "skipped": 0,
    "failed": 0
  },
  "hits": {
    "total": {
      "value": 1,
      "relation": "eq"
    },
    "max_score": null,
    "hits": []
  },
  "aggregations": {
    "0": {
      "value": 0 <= unexpected
    }
  }
}

Also see a zero value when querying constant_keyword values across indices:

# `test` already mapped as `constant_keyword` for `test-index-*`
PUT test-index/_bulk
{ "index" : { "_index" : "test-index-0001", "_id" : "1" } }
{ "test" : "value1" }
{ "index" : { "_index" : "test-index-0002", "_id" : "2" } }
{ "test" : "value2" }
{ "index" : { "_index" : "test-index-0003", "_id" : "3" } }
{ "test" : "value3" }
{ "index" : { "_index" : "test-index-0004", "_id" : "4" } }
{ "test" : "value4" }
{ "index" : { "_index" : "test-index-0005", "_id" : "5" } }
{ "test" : "value5" }



GET test-index-*/_search
{
  "aggs": {
    "0": {
      "cardinality": {
        "field": "test"
      }
    }
  },
  "size": 0
}

Logs (if relevant)

No response

@ebeahan ebeahan added >bug :Analytics/Aggregations Aggregations needs:triage Requires assignment of a team area label labels Sep 21, 2023
@elasticsearchmachine elasticsearchmachine added the Team:Analytics Meta label for analytical engine team (ESQL/Aggs/Geo) label Sep 21, 2023
@elasticsearchmachine
Copy link
Collaborator

Pinging @elastic/es-analytics-geo (Team:Analytics)

@kkrik-es
Copy link
Contributor

This is due to #92060, I think. indexTerms is null for a const_term field, returning LeafBucketCollector.NO_OP_COLLECTOR. I'll look into adding a check for this case.

kkrik-es added a commit to kkrik-es/elasticsearch that referenced this issue Sep 22, 2023
const_keyword fields don't show up in the leafReader, since they have
a const value. elastic#92060 modified the logic to return no results in case
the leaf reader contains no information about the requested field in a
cardinality aggregation. This is wrong for const_keyword fields, as they
contain up to 1 distinct value.

To fix this, we fall back to the old logic in this case that can
handle const_keyword fields properly.

Fixes elastic#99776
kkrik-es added a commit that referenced this issue Sep 25, 2023
* Fix cardinality agg for const_keyword

const_keyword fields don't show up in the leafReader, since they have
a const value. #92060 modified the logic to return no results in case
the leaf reader contains no information about the requested field in a
cardinality aggregation. This is wrong for const_keyword fields, as they
contain up to 1 distinct value.

To fix this, we fall back to the old logic in this case that can
handle const_keyword fields properly.

Fixes #99776

* Update docs/changelog/99814.yaml

* Update skip ranges for broken releases.
kkrik-es added a commit to kkrik-es/elasticsearch that referenced this issue Sep 25, 2023
* Fix cardinality agg for const_keyword

const_keyword fields don't show up in the leafReader, since they have
a const value. elastic#92060 modified the logic to return no results in case
the leaf reader contains no information about the requested field in a
cardinality aggregation. This is wrong for const_keyword fields, as they
contain up to 1 distinct value.

To fix this, we fall back to the old logic in this case that can
handle const_keyword fields properly.

Fixes elastic#99776

* Update docs/changelog/99814.yaml

* Update skip ranges for broken releases.
elasticsearchmachine pushed a commit that referenced this issue Sep 25, 2023
* Fix cardinality agg for const_keyword

const_keyword fields don't show up in the leafReader, since they have
a const value. #92060 modified the logic to return no results in case
the leaf reader contains no information about the requested field in a
cardinality aggregation. This is wrong for const_keyword fields, as they
contain up to 1 distinct value.

To fix this, we fall back to the old logic in this case that can
handle const_keyword fields properly.

Fixes #99776

* Update docs/changelog/99814.yaml

* Update skip ranges for broken releases.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
:Analytics/Aggregations Aggregations >bug Team:Analytics Meta label for analytical engine team (ESQL/Aggs/Geo)
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants