Cherry-pick #20875 to 7.x: [Metricbeat] Add latency config option into aws module #21537
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Cherry-pick of PR #20875 to 7.x branch. Original message:
What does this PR do?
This PR adds the
latency
config parameter foraws
module. With this config parameter, users can collect CloudWatch metrics with a latency larger than Metricbeat collection period. For example, for S3 request metrics, the latency can be 3 minutes. The recommended collection period is 1min. Without thelatency
config parameter, no data points will be collected ever.In this screenshot: current timestamp is 14:21 but we only see the latest data at 14:18 with a 3-minute delay. In this case, without this fix, our Metricbeat module will keep trying to collect data from the last 2-minute time frame, which will always be empty:
(@gbanasiak Thank you for your screenshot!)
Why is it important?
latency
config sets CloudWatch API start time and end time with a delay. For example, if the current time is2020-08-31T18:30:00.000Z
with collection period1m
.Without
latency
:startTime =
2020-08-31T18:29:00.000Z
endTime =
2020-08-31T18:30:00.000Z
With
latency
set to5min
:startTime =
2020-08-31T18:24:00.000Z
endTime =
2020-08-31T18:25:00.000Z
Checklist
CHANGELOG.next.asciidoc
orCHANGELOG-developer.next.asciidoc
.How to test this PR locally
./metricbeat modules enable aws
modules.d/aws.yml
to:./metricbeat -e -d "*"