Skip to content

Commit

Permalink
Cherry-pick elastic#15964 to 7.6: [Metricbeat] Update aws module docu…
Browse files Browse the repository at this point in the history
…mentation (elastic#16012)

(cherry picked from commit 7aa2f7a)
  • Loading branch information
kaiyan-sheng authored Feb 3, 2020
1 parent e3a7c9c commit 486784f
Show file tree
Hide file tree
Showing 4 changed files with 154 additions and 26 deletions.
88 changes: 76 additions & 12 deletions metricbeat/docs/modules/aws.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -39,6 +39,37 @@ Collecting `tags` for `ec2`, `cloudwatch`, `ebs` and `elb` metricset is supporte

* *tags.*: Tag key value pairs from aws resources. A tag is a label that user assigns to an AWS resource.

[float]
=== `billing`
Billing metric data includes the estimated charges for every service in the AWS
account and the estimated overall total charge for the AWS account. The estimated
charges are calculated and sent several times daily to CloudWatch. Therefore,
`period` in aws module configuration is set to `12h`.

The billing metricset comes with a predefined dashboard:

image::./images/metricbeat-aws-billing-overview.png[]

[float]
=== `cloudwatch`
This metricset allows users to query metrics from AWS CloudWatch with any given
namespaces or specific instance with a given period.
Please see https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/aws-services-cloudwatch-metrics.html[AWS Services That Publish CloudWatch Metrics]
for a list of AWS services that publish metrics to CloudWatch.

[float]
=== `ebs`
For basic monitoring in AWS EBS volumes, data is available automatically in
5-minute periods at no charge. This includes data for the root device volumes
for EBS-backed instances. User can also enable detailed monitoring for
provisioned IOPS SSD (io1) volumes to automatically send one-minute metrics to
CloudWatch. Default period in aws module configuration is set to `5m` for ebs
metricset.

The ebs metricset comes with a predefined dashboard:

image::./images/metricbeat-aws-ebs-overview.png[]

[float]
=== `ec2`
By default, Amazon EC2 sends metric data to CloudWatch every 5 minutes. With this basic monitoring, `period` in aws module
Expand All @@ -53,14 +84,34 @@ larger than `60s`. Since AWS sends metric data to CloudWatch in 1-minute periods
than `60s` will cause extra API requests which means extra charges on AWS. To avoid unnecessary charges, `period` is
preferred to be set to `60s` or multiples of `60s`, such as `120s` and `180s`.

The ec2 metricset comes with a predefined dashboard. For example:
The ec2 metricset comes with a predefined dashboard:

image::./images/metricbeat-aws-ec2-overview.png[]

[float]
=== `sqs`
CloudWatch metrics for Amazon SQS queues are automatically collected and pushed to CloudWatch every 5 minutes,
the `period` for `sqs` metricset is recommended to be `300s` or multiples of `300s`.
=== `elb`
elb metricset collects CloudWatch metrics from classic load balancer, application
load balancer and network load balancer.

All three kinds of elastic load balancing reports metrics to Cloudwatch only when requests are flowing
through the load balancer. If there are requests flowing through the load balancer,
Elastic Load Balancing measures and sends its metrics in 60-second intervals.
If there are no requests flowing through the load balancer or no data for a metric,
the metric is not reported.
Therefore, `period` in aws module configuration is set to `1m`.

The elb metricset comes with a predefined dashboard:

image::./images/metricbeat-aws-elb-overview.png[]

[float]
=== `rds`
`period` for `rds` metricset is recommended to be `60s` or multiples of `60s` because Amazon RDS sends metrics and
dimensions to Amazon CloudWatch every minute.

The rds metricset comes with a predefined dashboard:

image::./images/metricbeat-aws-rds-overview.png[]

[float]
=== `s3_daily_storage`
Expand All @@ -78,17 +129,30 @@ https://docs.aws.amazon.com/AmazonS3/latest/user-guide/configure-metrics.html[Ho
Configure Request Metrics for S3] for instructions on how to enable request metrics for
each S3 bucket.

The s3_daily_storage and s3_request metricset comes with a predefined combined dashboard:

image::./images/metricbeat-aws-s3-overview.png[]

[float]
=== `cloudwatch`
This metricset gives users the freedom to query metrics from AWS CloudWatch with
any given namespaces or specific instance with a given period.
Please see https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/aws-services-cloudwatch-metrics.html[AWS Services That Publish CloudWatch Metrics]
for a list of AWS services that publish metrics to CloudWatch.
=== `sqs`
CloudWatch metrics for Amazon SQS queues are automatically collected and pushed to CloudWatch every 5 minutes,
the `period` for `sqs` metricset is recommended to be `300s` or multiples of `300s`.

The sqs metricset comes with a predefined dashboard:

image::./images/metricbeat-aws-sqs-overview.png[]

[float]
=== `rds`
`period` for `rds` metricset is recommended to be `60s` or multiples of `60s` because Amazon RDS sends metrics and
dimensions to Amazon CloudWatch every minute.
=== `usage`
CloudWatch collects metrics that track the usage of some AWS resources. These
metrics correspond to AWS service quotas. Tracking these metrics can help
proactively manage quotas. Service quota usage metrics are in the AWS/Usage
namespace and are collected every minute. Therefore, `period` in aws module
configuration for usage metricset is set to `1m`.

The usage metricset comes with a predefined dashboard:

image::./images/metricbeat-aws-usage-overview.png[]

[float]
== AWS API requests count per metricset
Expand Down
2 changes: 1 addition & 1 deletion x-pack/metricbeat/module/aws/_meta/config.yml
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
- module: aws
period: 1m
metricsets:
- elb
- usage
- module: aws
period: 5m
Expand All @@ -19,7 +20,6 @@
metricsets:
- ebs
- ec2
- elb
- sns
- sqs
- rds
Expand Down
88 changes: 76 additions & 12 deletions x-pack/metricbeat/module/aws/_meta/docs.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,37 @@ Collecting `tags` for `ec2`, `cloudwatch`, `ebs` and `elb` metricset is supporte

* *tags.*: Tag key value pairs from aws resources. A tag is a label that user assigns to an AWS resource.

[float]
=== `billing`
Billing metric data includes the estimated charges for every service in the AWS
account and the estimated overall total charge for the AWS account. The estimated
charges are calculated and sent several times daily to CloudWatch. Therefore,
`period` in aws module configuration is set to `12h`.

The billing metricset comes with a predefined dashboard:

image::./images/metricbeat-aws-billing-overview.png[]

[float]
=== `cloudwatch`
This metricset allows users to query metrics from AWS CloudWatch with any given
namespaces or specific instance with a given period.
Please see https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/aws-services-cloudwatch-metrics.html[AWS Services That Publish CloudWatch Metrics]
for a list of AWS services that publish metrics to CloudWatch.

[float]
=== `ebs`
For basic monitoring in AWS EBS volumes, data is available automatically in
5-minute periods at no charge. This includes data for the root device volumes
for EBS-backed instances. User can also enable detailed monitoring for
provisioned IOPS SSD (io1) volumes to automatically send one-minute metrics to
CloudWatch. Default period in aws module configuration is set to `5m` for ebs
metricset.

The ebs metricset comes with a predefined dashboard:

image::./images/metricbeat-aws-ebs-overview.png[]

[float]
=== `ec2`
By default, Amazon EC2 sends metric data to CloudWatch every 5 minutes. With this basic monitoring, `period` in aws module
Expand All @@ -45,14 +76,34 @@ larger than `60s`. Since AWS sends metric data to CloudWatch in 1-minute periods
than `60s` will cause extra API requests which means extra charges on AWS. To avoid unnecessary charges, `period` is
preferred to be set to `60s` or multiples of `60s`, such as `120s` and `180s`.

The ec2 metricset comes with a predefined dashboard. For example:
The ec2 metricset comes with a predefined dashboard:

image::./images/metricbeat-aws-ec2-overview.png[]

[float]
=== `sqs`
CloudWatch metrics for Amazon SQS queues are automatically collected and pushed to CloudWatch every 5 minutes,
the `period` for `sqs` metricset is recommended to be `300s` or multiples of `300s`.
=== `elb`
elb metricset collects CloudWatch metrics from classic load balancer, application
load balancer and network load balancer.

All three kinds of elastic load balancing reports metrics to Cloudwatch only when requests are flowing
through the load balancer. If there are requests flowing through the load balancer,
Elastic Load Balancing measures and sends its metrics in 60-second intervals.
If there are no requests flowing through the load balancer or no data for a metric,
the metric is not reported.
Therefore, `period` in aws module configuration is set to `1m`.

The elb metricset comes with a predefined dashboard:

image::./images/metricbeat-aws-elb-overview.png[]

[float]
=== `rds`
`period` for `rds` metricset is recommended to be `60s` or multiples of `60s` because Amazon RDS sends metrics and
dimensions to Amazon CloudWatch every minute.

The rds metricset comes with a predefined dashboard:

image::./images/metricbeat-aws-rds-overview.png[]

[float]
=== `s3_daily_storage`
Expand All @@ -70,17 +121,30 @@ https://docs.aws.amazon.com/AmazonS3/latest/user-guide/configure-metrics.html[Ho
Configure Request Metrics for S3] for instructions on how to enable request metrics for
each S3 bucket.

The s3_daily_storage and s3_request metricset comes with a predefined combined dashboard:

image::./images/metricbeat-aws-s3-overview.png[]

[float]
=== `cloudwatch`
This metricset gives users the freedom to query metrics from AWS CloudWatch with
any given namespaces or specific instance with a given period.
Please see https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/aws-services-cloudwatch-metrics.html[AWS Services That Publish CloudWatch Metrics]
for a list of AWS services that publish metrics to CloudWatch.
=== `sqs`
CloudWatch metrics for Amazon SQS queues are automatically collected and pushed to CloudWatch every 5 minutes,
the `period` for `sqs` metricset is recommended to be `300s` or multiples of `300s`.

The sqs metricset comes with a predefined dashboard:

image::./images/metricbeat-aws-sqs-overview.png[]

[float]
=== `rds`
`period` for `rds` metricset is recommended to be `60s` or multiples of `60s` because Amazon RDS sends metrics and
dimensions to Amazon CloudWatch every minute.
=== `usage`
CloudWatch collects metrics that track the usage of some AWS resources. These
metrics correspond to AWS service quotas. Tracking these metrics can help
proactively manage quotas. Service quota usage metrics are in the AWS/Usage
namespace and are collected every minute. Therefore, `period` in aws module
configuration for usage metricset is set to `1m`.

The usage metricset comes with a predefined dashboard:

image::./images/metricbeat-aws-usage-overview.png[]

[float]
== AWS API requests count per metricset
Expand Down
2 changes: 1 addition & 1 deletion x-pack/metricbeat/modules.d/aws.yml.disabled
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@
- module: aws
period: 1m
metricsets:
- elb
- usage
- module: aws
period: 5m
Expand All @@ -22,7 +23,6 @@
metricsets:
- ebs
- ec2
- elb
- sns
- sqs
- rds
Expand Down

0 comments on commit 486784f

Please sign in to comment.