Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Can't use time_series_metric mapping parameter with scaled_float or unsigned_long fields #78100

Closed
jrodewig opened this issue Sep 21, 2021 · 2 comments · Fixed by #78204
Closed
Labels
>bug :Search Foundations/Mapping Index mappings, including merging and defining field types :StorageEngine/Rollup Turn fine-grained time-based data into coarser-grained data :StorageEngine/TSDB You know, for Metrics Team:Analytics Meta label for analytical engine team (ESQL/Aggs/Geo) Team:Search Foundations Meta label for the Search Foundations team in Elasticsearch

Comments

@jrodewig
Copy link
Contributor

Elasticsearch version (bin/elasticsearch --version):
Latest elasticsearch-8.0.0-SNAPSHOT-darwin-x86_64.tar.gz
Version: 8.0.0-SNAPSHOT, Build: default/tar/a3213137789b03e114a0bfa97db953e2ecf82671/2021-09-21T13:21:46.873945Z", JVM: 16.0.1

Plugins installed: []

JVM version (java -version):
openjdk version "16.0.1" 2021-04-20
OpenJDK Runtime Environment AdoptOpenJDK-16.0.1+9 (build 16.0.1+9)
OpenJDK 64-Bit Server VM AdoptOpenJDK-16.0.1+9 (build 16.0.1+9, mixed mode, sharing)

OS version (uname -a if on a Unix-like system):
Darwin Kernel Version 20.6.0: Wed Jun 23 00:26:31 PDT 2021; root:xnu-7195.141.2~5/RELEASE_X86_64 x86_64

Description of the problem including expected versus actual behavior:

#76766 adds the time_series_metric mapping parameter to numeric fields. However, you can't use the parameter in scaled_float or unsigned_long fields.

Steps to reproduce:

  1. Attempt to create an index containing a scaled_float field with the time_series_metric parameter.
# Unscaled float as counter metric
PUT my-scaled-float-counter-index
{
  "mappings": {
    "properties": {
      "my-scaled-float-field": {
        "type": "scaled_float",
        "scaling_factor": 2.3,
        "time_series_metric": "counter"
      }
    }
  }
}

# Unscaled float as gauge metric
PUT my-scaled-float-gauge-index
{
  "mappings": {
    "properties": {
      "my-scaled-float-field": {
        "type": "scaled_float",
        "scaling_factor": 2.3,
        "time_series_metric": "gauge"
      }
    }
  }
}

The request returns the following error:

{
  "error" : {
    "root_cause" : [
      {
        "type" : "mapper_parsing_exception",
        "reason" : "unknown parameter [time_series_metric] on mapper [my-scaled-float-field] of type [scaled_float]"
      }
    ],
    "type" : "mapper_parsing_exception",
    "reason" : "Failed to parse mapping: unknown parameter [time_series_metric] on mapper [my-scaled-float-field] of type [scaled_float]",
    "caused_by" : {
      "type" : "mapper_parsing_exception",
      "reason" : "unknown parameter [time_series_metric] on mapper [my-scaled-float-field] of type [scaled_float]"
    }
  },
  "status" : 400
}
  1. Attempt to create an index containing an unsigned_long field with the time_series_metric parameter.
# Unsigned long as counter
PUT my-unsigned-long-counter-index
{
  "mappings": {
    "properties": {
      "my-unsigned-long-field": {
        "type": "unsigned_long",
        "time_series_metric": "counter"
      }
    }
  }
}

# Unsigned long as gauge
PUT my-unsigned-long-gauge-index
{
  "mappings": {
    "properties": {
      "my-unsigned-long-field": {
        "type": "unsigned_long",
        "time_series_metric": "gauge"
      }
    }
  }
}

The request returns the following error:

{
  "error" : {
    "root_cause" : [
      {
        "type" : "mapper_parsing_exception",
        "reason" : "unknown parameter [time_series_metric] on mapper [my-unsigned-long-field] of type [unsigned_long]"
      }
    ],
    "type" : "mapper_parsing_exception",
    "reason" : "Failed to parse mapping: unknown parameter [time_series_metric] on mapper [my-unsigned-long-field] of type [unsigned_long]",
    "caused_by" : {
      "type" : "mapper_parsing_exception",
      "reason" : "unknown parameter [time_series_metric] on mapper [my-unsigned-long-field] of type [unsigned_long]"
    }
  },
  "status" : 400
}

@jrodewig jrodewig added >bug :Search Foundations/Mapping Index mappings, including merging and defining field types :StorageEngine/Rollup Turn fine-grained time-based data into coarser-grained data :StorageEngine/TSDB You know, for Metrics labels Sep 21, 2021
@elasticmachine elasticmachine added Team:Analytics Meta label for analytical engine team (ESQL/Aggs/Geo) Team:Search Meta label for search team labels Sep 21, 2021
@elasticmachine
Copy link
Collaborator

Pinging @elastic/es-search (Team:Search)

@elasticmachine
Copy link
Collaborator

Pinging @elastic/es-analytics-geo (Team:Analytics)

csoulios added a commit that referenced this issue Sep 23, 2021
    Added the time_series_metric mapping parameter to the unsigned_long and scaled_float field types
    Added the time_series_dimension mapping parameter to the unsigned_long field type

Fixes #78100

Relates to #76766, #74450 and #74014
csoulios added a commit to csoulios/elasticsearch that referenced this issue Sep 23, 2021
…#78204)

    Added the time_series_metric mapping parameter to the unsigned_long and scaled_float field types
    Added the time_series_dimension mapping parameter to the unsigned_long field type

Fixes elastic#78100

Relates to elastic#76766, elastic#74450 and elastic#74014
csoulios added a commit that referenced this issue Sep 27, 2021
…rameters (#78265)

Backports the following PRs:

* Add dimension mapping parameter (#74450)

Added the dimension parameter to the following field types:

    keyword
    ip
    Numeric field types (integer, long, byte, short)

The dimension parameter is of type boolean (default: false) and is used
to mark that a field is a time series dimension field.

Relates to #74014

* Add constraints to dimension fields (#74939)

This PR adds the following constraints to dimension fields:

    It must be an indexed field and must has doc values
    It cannot be multi-valued
    The number of dimension fields in the index mapping must not be more than 16. This should be configurable through an index property (index.mapping.dimension_fields.limit)
    keyword fields cannot be more than 1024 bytes long
    keyword fields must not use a normalizer

Based on the code added in PR #74450
Relates to #74660

* Expand DocumentMapperTests (#76368)

Adds a test for setting the maximum number of dimensions setting and
tests the names and types of the metadata fields in the index.
Previously we just asserted the count of metadata fields. That made it
hard to read failures.

* Fix broken test for dimension keywords (#75408)

Test was failing because it was testing 1024 bytes long keyword and assertion was failing.

Closes #75225

* Checkstyle

* Add time_series_metric parameter (#76766)

This PR adds the time_series_metric parameter to the following field types:

    Numeric field types
    histogram
    aggregate_metric_double

* Rename `dimension` mapping parameter to `time_series_dimension` (#78012)

This PR renames dimension mapping parameter to time_series_dimension to make it consistent with time_series_metric parameter (#76766)

Relates to #74450 and #74014

* Add time series params to `unsigned_long` and `scaled_float` (#78204)

    Added the time_series_metric mapping parameter to the unsigned_long and scaled_float field types
    Added the time_series_dimension mapping parameter to the unsigned_long field type

Fixes #78100

Relates to #76766, #74450 and #74014

Co-authored-by: Nik Everett <[email protected]>
@javanna javanna added Team:Search Foundations Meta label for the Search Foundations team in Elasticsearch and removed Team:Search Meta label for search team labels Jul 16, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
>bug :Search Foundations/Mapping Index mappings, including merging and defining field types :StorageEngine/Rollup Turn fine-grained time-based data into coarser-grained data :StorageEngine/TSDB You know, for Metrics Team:Analytics Meta label for analytical engine team (ESQL/Aggs/Geo) Team:Search Foundations Meta label for the Search Foundations team in Elasticsearch
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants