Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Revert "Deprecate _source.mode in mappings (#116689)" #117150

Merged
merged 1 commit into from
Nov 20, 2024

Conversation

martijnvg
Copy link
Member

@martijnvg martijnvg commented Nov 20, 2024

This reverts #116689. Bwc tests fail, because older nodes are killed with the following error:

[2024-11-20T10:54:58,600][ERROR][o.e.b.ElasticsearchUncaughtExceptionHandler] [v8.17.0-0] fatal error in thread [elasticsearch[v8.17.0-0
][clusterApplierService#updateTask][T#1]], exiting java.lang.AssertionError: provided source [{"_doc":{"_data_stream_timestamp":{"enabled":true},"_source":{},"properties":{"@timestamp":{"type":"date"},"k8s":{"properties":{"pod":{"properties":{"ip":{"type":"ip"},"name":{"type":"keyword"},"network":{"properties":{"rx":{"type":"long"},"tx":{"type":"long"}}},"uid":{"type":"keyword","time_series_dimension":true}}}}},"metricset":{"type":"keyword","time_series_dimension":true}}}}] differs from mapping [{"_doc":{"_data_stream_timestamp":{"enabled":true},"_source":{"mode":"synthetic"},"properties":{"@timestamp":{"type":"date"},"k8s":{"properties":{"pod":{"properties":{"ip":{"type":"ip"},"name":{"type":"keyword"},"network":{"properties":{"rx":{"type":"long"},"tx":{"type":"long"}}},"uid":{"type":"keyword","time_series_dimension":true}}}}},"metricset":{"type":"keyword","time_series_dimension":true}}}}]
        at [email protected]/org.elasticsearch.index.mapper.DocumentMapper.<init>(DocumentMapper.java:66)
        at [email protected]/org.elasticsearch.index.mapper.MapperService.newDocumentMapper(MapperService.java:588)
        at [email protected]/org.elasticsearch.index.mapper.MapperService.updateMapping(MapperService.java:346)
        at [email protected]/org.elasticsearch.index.IndexService.updateMapping(IndexService.java:840)
        at [email protected]/org.elasticsearch.indices.cluster.IndicesClusterStateService.createIndicesAndUpdateShards(IndicesClusterStateService.java:583)
        at [email protected]/org.elasticsearch.indices.cluster.IndicesClusterStateService.doApplyClusterState(IndicesClusterStateService.java:306)
        at [email protected]/org.elasticsearch.indices.cluster.IndicesClusterStateService.applyClusterState(IndicesClusterStateService.java:260)
        at [email protected]/org.elasticsearch.cluster.service.ClusterApplierService.callClusterStateAppliers(ClusterApplierService.java:544)
        at [email protected]/org.elasticsearch.cluster.service.ClusterApplierService.callClusterStateAppliers(ClusterApplierService.java:530)
        at [email protected]/org.elasticsearch.cluster.service.ClusterApplierService.applyChanges(ClusterApplierService.java:503)
        at [email protected]/org.elasticsearch.cluster.service.ClusterApplierService.runTask(ClusterApplierService.java:432)
        at [email protected]/org.elasticsearch.cluster.service.ClusterApplierService$UpdateTask.run(ClusterApplierService.java:157)
        at [email protected]/org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingRunnable.run(ThreadContext.java:956)
        at [email protected]/org.elasticsearch.common.util.concurrent.PrioritizedEsThreadPoolExecutor$TieBreakingPrioritizedRunnable.runAndClean(PrioritizedEsThreadPoolExecutor.java:218)
        at [email protected]/org.elasticsearch.common.util.concurrent.PrioritizedEsThreadPoolExecutor$TieBreakingPrioritizedRunnable.run(PrioritizedEsThreadPoolExecutor.java:184)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)
        at java.base/java.lang.Thread.run(Thread.java:1575)

The mode parameter no longer gets serialized for new indices. However on the older nodes still serialize the mode parameter, which caused the menioned assertion to fail. Reverting for now and see how best to address this bwc serialization issue.

We can only stop serializing mode, when all nodes are on the same version. Unfortunately we can't invoke c.clusterTransportVersion().get() from parser or builder, because that calling thread isn't allowed to call clusterService.state().

@martijnvg martijnvg added >non-issue :StorageEngine/Mapping The storage related side of mappings v9.0.0 labels Nov 20, 2024
@martijnvg martijnvg requested review from a team as code owners November 20, 2024 12:27
@martijnvg martijnvg changed the title Revert "Deprecate _source.mode in mappings (#)" Revert "Deprecate _source.mode in mappings (#116689)" Nov 20, 2024
@elasticsearchmachine
Copy link
Collaborator

Pinging @elastic/es-storage-engine (Team:StorageEngine)

@martijnvg martijnvg merged commit ac06a84 into elastic:main Nov 20, 2024
4 of 16 checks passed
martijnvg added a commit to martijnvg/elasticsearch that referenced this pull request Nov 20, 2024
martijnvg added a commit to martijnvg/elasticsearch that referenced this pull request Nov 20, 2024
martijnvg added a commit to martijnvg/elasticsearch that referenced this pull request Nov 20, 2024
rjernst pushed a commit to rjernst/elasticsearch that referenced this pull request Nov 20, 2024
alexey-ivanov-es pushed a commit to alexey-ivanov-es/elasticsearch that referenced this pull request Nov 28, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants