Improve SyntheticSourceIndexSettingsProvider to not always create a MapperService. #116070
Assignees
Labels
Comments
|
Pinging @elastic/es-storage-engine (Team:StorageEngine) |
martijnvg
added a commit
to martijnvg/elasticsearch
that referenced
this issue
Nov 1, 2024
…n required. In case the index.mapping.source.mode is specified, there is no need to create a mapper service to determine whether synthetic source is used. In case of logsdb/tsdb there is also no reason to create a mapper service. If _source.mode attribute is specified, then it doesn't really matter whether what its value is for the SyntheticSourceIndexSettingsProvider. If it is synthetic, then that is the same as the index mode's default source mode. If it is stored, we just will add set `index.mapping.source.mode` to `stored`, which has no effect. And disabled source mode isn't allowed in the case of logsdb and tsdb. Closes elastic#116070
martijnvg
added a commit
to martijnvg/elasticsearch
that referenced
this issue
Nov 5, 2024
…n required (elastic#116075) In case the index.mapping.source.mode is specified, there is no need to create a mapper service to determine whether synthetic source is used. In case of logsdb/tsdb there is also no reason to create a mapper service. If _source.mode attribute is specified, then it doesn't really matter whether what its value is for the SyntheticSourceIndexSettingsProvider. If it is synthetic, then that is the same as the index mode's default source mode. If it is stored, we just will add set index.mapping.source.mode to stored, which has no effect. And disabled source mode isn't allowed in the case of logsdb and tsdb. Closes elastic#116070
elasticsearchmachine
pushed a commit
that referenced
this issue
Nov 5, 2024
…n required (#116075) (#116233) In case the index.mapping.source.mode is specified, there is no need to create a mapper service to determine whether synthetic source is used. In case of logsdb/tsdb there is also no reason to create a mapper service. If _source.mode attribute is specified, then it doesn't really matter whether what its value is for the SyntheticSourceIndexSettingsProvider. If it is synthetic, then that is the same as the index mode's default source mode. If it is stored, we just will add set index.mapping.source.mode to stored, which has no effect. And disabled source mode isn't allowed in the case of logsdb and tsdb. Closes #116070
jozala
pushed a commit
that referenced
this issue
Nov 13, 2024
…n required (#116075) In case the index.mapping.source.mode is specified, there is no need to create a mapper service to determine whether synthetic source is used. In case of logsdb/tsdb there is also no reason to create a mapper service. If _source.mode attribute is specified, then it doesn't really matter whether what its value is for the SyntheticSourceIndexSettingsProvider. If it is synthetic, then that is the same as the index mode's default source mode. If it is stored, we just will add set index.mapping.source.mode to stored, which has no effect. And disabled source mode isn't allowed in the case of logsdb and tsdb. Closes #116070
alexey-ivanov-es
pushed a commit
to alexey-ivanov-es/elasticsearch
that referenced
this issue
Nov 28, 2024
…n required (elastic#116075) In case the index.mapping.source.mode is specified, there is no need to create a mapper service to determine whether synthetic source is used. In case of logsdb/tsdb there is also no reason to create a mapper service. If _source.mode attribute is specified, then it doesn't really matter whether what its value is for the SyntheticSourceIndexSettingsProvider. If it is synthetic, then that is the same as the index mode's default source mode. If it is stored, we just will add set index.mapping.source.mode to stored, which has no effect. And disabled source mode isn't allowed in the case of logsdb and tsdb. Closes elastic#116070
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
A regression was detected in
nightly-elastic/logs-many-shards-quantitative-indexing-throughput-1visualization in the many shards nightly benchmark. Investigation has indicated that the decrease in throughput was caused when logsdb was enabled by default. This mainly keeps track of index / data stream creation.We should be able to add an optimization that avoids creating a new mapper service when we can figure out that synthetic source is used by just looking in the index settings (the
index.modeandindex.mapping.source.modeindex settings). Only if the deprecated source field mapper'smodeattribute is used we have to fall back to creating mapper service.The text was updated successfully, but these errors were encountered: