Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Send period in metricbeat data #12616

Closed
skh opened this issue Jun 19, 2019 · 6 comments · Fixed by #13242
Closed

Send period in metricbeat data #12616

skh opened this issue Jun 19, 2019 · 6 comments · Fixed by #13242
Assignees
Labels
candidate Candidate to be added to the current iteration enhancement Metricbeat Metricbeat Team:Integrations Label for the Integrations team

Comments

@skh
Copy link

skh commented Jun 19, 2019

For querying metricbeat documents for the Kibana Infra UI app, it would be very helpful to know the interval in which data is sent by the various metricbeat modules.

One idea that came up in a call is to include the configured period in the documents themselves.

@kaiyan-sheng kaiyan-sheng added the Team:Integrations Label for the Integrations team label Jun 19, 2019
@kaiyan-sheng
Copy link
Contributor

Nicolas had an old PR for doing the same thing: #6929

@exekias
Copy link
Contributor

exekias commented Jun 24, 2019

Thank you for opening, is there anywhere where we can read how you plan to use it?

@exekias exekias added the discuss Issue needs further discussion. label Jun 24, 2019
@skh
Copy link
Author

skh commented Jul 24, 2019

@exekias It would be a solution to the problem outlined in elastic/kibana#36774

@simianhacker
Copy link
Member

@exekias If the period is stored with the event data then we would need to get a sample from the time range. Either using an aggregation (terms?) or just plucking the last document where the field exists (along with filtering for time range and whatnot).

@exekias
Copy link
Contributor

exekias commented Jul 26, 2019

As discussed yesterday:

In order to detect gaps, it would be nice to have the reporting period in docs. The UI can then query for the max period across docs for some metric and use it as the minimum bucket size for the query. Then it will now that an empty bucket is a gap int he data.

@exekias exekias removed the discuss Issue needs further discussion. label Jul 26, 2019
@exekias
Copy link
Contributor

exekias commented Jul 26, 2019

@sorantis FYI ^

@exekias exekias added the candidate Candidate to be added to the current iteration label Jul 26, 2019
@jsoriano jsoriano self-assigned this Aug 14, 2019
jsoriano added a commit that referenced this issue Aug 23, 2019
Having the period as part of each event makes it possible
for Kibana or ML to predict when the next event is
potentially missing or delayed based on the period of the
previous events. It can always be that the period changed
but as soon as the next event comes in, this can be used
as the new expected period.

This field is not added to push fetchers, as they don't do
periodic fetching.

Fixes #12616

Co-authored-by: ruflin <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
candidate Candidate to be added to the current iteration enhancement Metricbeat Metricbeat Team:Integrations Label for the Integrations team
Projects
None yet
Development

Successfully merging a pull request may close this issue.

5 participants