Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add "synthetics-*-*" templates for synthetics fleet data #62193

Merged
merged 6 commits into from
Sep 14, 2020

Conversation

dakrone
Copy link
Member

@dakrone dakrone commented Sep 9, 2020

For the Elastic Agent we currently have logs and metrics, however, synthetic data doesn't belong
with those and thus we should have a place for it to live. This would be data reported from
heartbeat and under the 'monitoring' category.

This commit adds a composable index template for synthetics-*-* indices similar to the work in
#56709 and #57629.

Resolves #61665

For the Elastic Agent we currently have `logs` and `metrics`, however, synthetic data doesn't belong
with those and thus we should have a place for it to live. This would be data reported from
heartbeat and under the 'monitoring' category.

This commit adds a composable index template for `synthetics-*-*` indices similar to the work in
 elastic#56709 and elastic#57629.

Resolves elastic#61665
@dakrone dakrone added :Data Management/Indices APIs APIs to create and manage indices and templates v8.0.0 v7.10.0 labels Sep 9, 2020
@elasticmachine
Copy link
Collaborator

Pinging @elastic/es-core-features (:Core/Features/Indices APIs)

@elasticmachine elasticmachine added the Team:Data Management Meta label for data/management team label Sep 9, 2020
Copy link
Contributor

@andreidan andreidan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks Lee (it's only the docs tests that need some adjustments)

Copy link
Contributor

@ruflin ruflin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall LGTM but I want some others to chime in too.

"hot": {
"actions": {
"rollover": {
"max_size": "50gb",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@andrewvc I assume this also works for synthetics for now?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

TBH we haven't done enough benchmarking to know, but it seems reasonable

"priority": 100,
"data_stream": {},
"composed_of": [
"synthetics-mappings",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jpountz I remember you wanted to "not" use components here. Perhaps it is a good time to adjust?

@ruflin
Copy link
Contributor

ruflin commented Sep 10, 2020

@ph @neptunian As soon as this lands, we also probably need to add a fake doc for the index pattern in Kibana.

@neptunian
Copy link

neptunian commented Sep 14, 2020

@ruflin if we expect to create an index pattern, like synthetics-* when a package installs we need to add this to our list of index pattern types. Then a placeholder would be created automatically.

@dakrone
Copy link
Member Author

dakrone commented Sep 14, 2020

@elasticmachine update branch

@dakrone
Copy link
Member Author

dakrone commented Sep 14, 2020

@elasticmachine run elasticsearch-ci/2

@dakrone dakrone merged commit d21ddb5 into elastic:master Sep 14, 2020
@dakrone dakrone deleted the add-synthetics-templates branch September 14, 2020 22:37
dakrone added a commit to dakrone/elasticsearch that referenced this pull request Sep 14, 2020
* Add "synthetics-*-*" templates for synthetics fleet data

For the Elastic Agent we currently have `logs` and `metrics`, however, synthetic data doesn't belong
with those and thus we should have a place for it to live. This would be data reported from
heartbeat and under the 'monitoring' category.

This commit adds a composable index template for `synthetics-*-*` indices similar to the work in
 elastic#56709 and elastic#57629.

Resolves elastic#61665
dakrone added a commit that referenced this pull request Sep 14, 2020
… (#62346)

* Add "synthetics-*-*" templates for synthetics fleet data

For the Elastic Agent we currently have `logs` and `metrics`, however, synthetic data doesn't belong
with those and thus we should have a place for it to live. This would be data reported from
heartbeat and under the 'monitoring' category.

This commit adds a composable index template for `synthetics-*-*` indices similar to the work in
 #56709 and #57629.

Resolves #61665
@ruflin
Copy link
Contributor

ruflin commented Sep 15, 2020

@neptunian SGTM

dakrone added a commit to dakrone/elasticsearch that referenced this pull request Sep 15, 2020
We need to ensure these are mapped as 'ip' instead of a keyword, even if they do end up not being
used.

Relates to elastic#62193
dakrone added a commit that referenced this pull request Sep 17, 2020
…2412)

We need to ensure these are mapped as 'ip' instead of a keyword, even if they do end up not being
used.

Relates to #62193
dakrone added a commit to dakrone/elasticsearch that referenced this pull request Sep 17, 2020
…astic#62412)

We need to ensure these are mapped as 'ip' instead of a keyword, even if they do end up not being
used.

Relates to elastic#62193
dakrone added a commit that referenced this pull request Sep 17, 2020
…gs (#62412) (#62553)

We need to ensure these are mapped as 'ip' instead of a keyword, even if they do end up not being
used.

Relates to #62193
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
:Data Management/Indices APIs APIs to create and manage indices and templates Team:Data Management Meta label for data/management team v7.10.0 v8.0.0-alpha1
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add new index for synthetics
8 participants