Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add data stream timestamp validation via metadata field mapper #59002

Merged
merged 6 commits into from
Jul 6, 2020

Conversation

martijnvg
Copy link
Member

Backport of #58582 to 7.x branch.

This commit adds a new metadata field mapper that validates,
that a document has exactly a single timestamp value in the data stream timestamp field and
that the timestamp field mapping only has type, meta or format attributes configured.
Other attributes can affect the guarantee that an index with this meta field mapper has a
useable timestamp field.

The MetadataCreateIndexService inserts a data stream timestamp field mapper whenever
a new backing index of a data stream is created.

Relates to #53100

martijnvg added 2 commits July 3, 2020 11:59
Backport of elastic#58582 to 7.x branch.

This commit adds a new metadata field mapper that validates,
that a document has exactly a single timestamp value in the data stream timestamp field and
that the timestamp field mapping only has `type`, `meta` or `format` attributes configured.
Other attributes can affect the guarantee that an index with this meta field mapper has a
useable timestamp field.

The MetadataCreateIndexService inserts a data stream timestamp field mapper whenever
a new backing index of a data stream is created.

Relates to elastic#53100
@martijnvg martijnvg added backport :Data Management/Data streams Data streams and their lifecycles v7.9.0 labels Jul 3, 2020
@elasticmachine
Copy link
Collaborator

Pinging @elastic/es-core-features (:Core/Features/Data streams)

@elasticmachine elasticmachine added the Team:Data Management Meta label for data/management team label Jul 3, 2020
@martijnvg
Copy link
Member Author

@elasticmachine update branch

@martijnvg martijnvg merged commit f0dd9b4 into elastic:7.x Jul 6, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backport :Data Management/Data streams Data streams and their lifecycles Team:Data Management Meta label for data/management team v7.9.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants