diff --git a/metricbeat/docs/modules/aws.asciidoc b/metricbeat/docs/modules/aws.asciidoc index c0857b61007..61180ef7d99 100644 --- a/metricbeat/docs/modules/aws.asciidoc +++ b/metricbeat/docs/modules/aws.asciidoc @@ -39,6 +39,37 @@ Collecting `tags` for `ec2`, `cloudwatch`, `ebs` and `elb` metricset is supporte * *tags.*: Tag key value pairs from aws resources. A tag is a label that user assigns to an AWS resource. +[float] +=== `billing` +Billing metric data includes the estimated charges for every service in the AWS +account and the estimated overall total charge for the AWS account. The estimated +charges are calculated and sent several times daily to CloudWatch. Therefore, +`period` in aws module configuration is set to `12h`. + +The billing metricset comes with a predefined dashboard: + +image::./images/metricbeat-aws-billing-overview.png[] + +[float] +=== `cloudwatch` +This metricset allows users to query metrics from AWS CloudWatch with any given +namespaces or specific instance with a given period. +Please see https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/aws-services-cloudwatch-metrics.html[AWS Services That Publish CloudWatch Metrics] +for a list of AWS services that publish metrics to CloudWatch. + +[float] +=== `ebs` +For basic monitoring in AWS EBS volumes, data is available automatically in +5-minute periods at no charge. This includes data for the root device volumes +for EBS-backed instances. User can also enable detailed monitoring for +provisioned IOPS SSD (io1) volumes to automatically send one-minute metrics to +CloudWatch. Default period in aws module configuration is set to `5m` for ebs +metricset. + +The ebs metricset comes with a predefined dashboard: + +image::./images/metricbeat-aws-ebs-overview.png[] + [float] === `ec2` By default, Amazon EC2 sends metric data to CloudWatch every 5 minutes. With this basic monitoring, `period` in aws module @@ -53,14 +84,34 @@ larger than `60s`. Since AWS sends metric data to CloudWatch in 1-minute periods than `60s` will cause extra API requests which means extra charges on AWS. To avoid unnecessary charges, `period` is preferred to be set to `60s` or multiples of `60s`, such as `120s` and `180s`. -The ec2 metricset comes with a predefined dashboard. For example: +The ec2 metricset comes with a predefined dashboard: image::./images/metricbeat-aws-ec2-overview.png[] [float] -=== `sqs` -CloudWatch metrics for Amazon SQS queues are automatically collected and pushed to CloudWatch every 5 minutes, -the `period` for `sqs` metricset is recommended to be `300s` or multiples of `300s`. +=== `elb` +elb metricset collects CloudWatch metrics from classic load balancer, application +load balancer and network load balancer. + +All three kinds of elastic load balancing reports metrics to Cloudwatch only when requests are flowing +through the load balancer. If there are requests flowing through the load balancer, +Elastic Load Balancing measures and sends its metrics in 60-second intervals. +If there are no requests flowing through the load balancer or no data for a metric, +the metric is not reported. +Therefore, `period` in aws module configuration is set to `1m`. + +The elb metricset comes with a predefined dashboard: + +image::./images/metricbeat-aws-elb-overview.png[] + +[float] +=== `rds` +`period` for `rds` metricset is recommended to be `60s` or multiples of `60s` because Amazon RDS sends metrics and +dimensions to Amazon CloudWatch every minute. + +The rds metricset comes with a predefined dashboard: + +image::./images/metricbeat-aws-rds-overview.png[] [float] === `s3_daily_storage` @@ -78,17 +129,30 @@ https://docs.aws.amazon.com/AmazonS3/latest/user-guide/configure-metrics.html[Ho Configure Request Metrics for S3] for instructions on how to enable request metrics for each S3 bucket. +The s3_daily_storage and s3_request metricset comes with a predefined combined dashboard: + +image::./images/metricbeat-aws-s3-overview.png[] + [float] -=== `cloudwatch` -This metricset gives users the freedom to query metrics from AWS CloudWatch with -any given namespaces or specific instance with a given period. -Please see https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/aws-services-cloudwatch-metrics.html[AWS Services That Publish CloudWatch Metrics] -for a list of AWS services that publish metrics to CloudWatch. +=== `sqs` +CloudWatch metrics for Amazon SQS queues are automatically collected and pushed to CloudWatch every 5 minutes, +the `period` for `sqs` metricset is recommended to be `300s` or multiples of `300s`. + +The sqs metricset comes with a predefined dashboard: + +image::./images/metricbeat-aws-sqs-overview.png[] [float] -=== `rds` -`period` for `rds` metricset is recommended to be `60s` or multiples of `60s` because Amazon RDS sends metrics and -dimensions to Amazon CloudWatch every minute. +=== `usage` +CloudWatch collects metrics that track the usage of some AWS resources. These +metrics correspond to AWS service quotas. Tracking these metrics can help +proactively manage quotas. Service quota usage metrics are in the AWS/Usage +namespace and are collected every minute. Therefore, `period` in aws module +configuration for usage metricset is set to `1m`. + +The usage metricset comes with a predefined dashboard: + +image::./images/metricbeat-aws-usage-overview.png[] [float] == AWS API requests count per metricset diff --git a/x-pack/metricbeat/module/aws/_meta/config.yml b/x-pack/metricbeat/module/aws/_meta/config.yml index 513719e5681..daad69ba51c 100644 --- a/x-pack/metricbeat/module/aws/_meta/config.yml +++ b/x-pack/metricbeat/module/aws/_meta/config.yml @@ -1,6 +1,7 @@ - module: aws period: 1m metricsets: + - elb - usage - module: aws period: 5m @@ -19,7 +20,6 @@ metricsets: - ebs - ec2 - - elb - sns - sqs - rds diff --git a/x-pack/metricbeat/module/aws/_meta/docs.asciidoc b/x-pack/metricbeat/module/aws/_meta/docs.asciidoc index 0b3565a001a..32726b80ccb 100644 --- a/x-pack/metricbeat/module/aws/_meta/docs.asciidoc +++ b/x-pack/metricbeat/module/aws/_meta/docs.asciidoc @@ -31,6 +31,37 @@ Collecting `tags` for `ec2`, `cloudwatch`, `ebs` and `elb` metricset is supporte * *tags.*: Tag key value pairs from aws resources. A tag is a label that user assigns to an AWS resource. +[float] +=== `billing` +Billing metric data includes the estimated charges for every service in the AWS +account and the estimated overall total charge for the AWS account. The estimated +charges are calculated and sent several times daily to CloudWatch. Therefore, +`period` in aws module configuration is set to `12h`. + +The billing metricset comes with a predefined dashboard: + +image::./images/metricbeat-aws-billing-overview.png[] + +[float] +=== `cloudwatch` +This metricset allows users to query metrics from AWS CloudWatch with any given +namespaces or specific instance with a given period. +Please see https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/aws-services-cloudwatch-metrics.html[AWS Services That Publish CloudWatch Metrics] +for a list of AWS services that publish metrics to CloudWatch. + +[float] +=== `ebs` +For basic monitoring in AWS EBS volumes, data is available automatically in +5-minute periods at no charge. This includes data for the root device volumes +for EBS-backed instances. User can also enable detailed monitoring for +provisioned IOPS SSD (io1) volumes to automatically send one-minute metrics to +CloudWatch. Default period in aws module configuration is set to `5m` for ebs +metricset. + +The ebs metricset comes with a predefined dashboard: + +image::./images/metricbeat-aws-ebs-overview.png[] + [float] === `ec2` By default, Amazon EC2 sends metric data to CloudWatch every 5 minutes. With this basic monitoring, `period` in aws module @@ -45,14 +76,34 @@ larger than `60s`. Since AWS sends metric data to CloudWatch in 1-minute periods than `60s` will cause extra API requests which means extra charges on AWS. To avoid unnecessary charges, `period` is preferred to be set to `60s` or multiples of `60s`, such as `120s` and `180s`. -The ec2 metricset comes with a predefined dashboard. For example: +The ec2 metricset comes with a predefined dashboard: image::./images/metricbeat-aws-ec2-overview.png[] [float] -=== `sqs` -CloudWatch metrics for Amazon SQS queues are automatically collected and pushed to CloudWatch every 5 minutes, -the `period` for `sqs` metricset is recommended to be `300s` or multiples of `300s`. +=== `elb` +elb metricset collects CloudWatch metrics from classic load balancer, application +load balancer and network load balancer. + +All three kinds of elastic load balancing reports metrics to Cloudwatch only when requests are flowing +through the load balancer. If there are requests flowing through the load balancer, +Elastic Load Balancing measures and sends its metrics in 60-second intervals. +If there are no requests flowing through the load balancer or no data for a metric, +the metric is not reported. +Therefore, `period` in aws module configuration is set to `1m`. + +The elb metricset comes with a predefined dashboard: + +image::./images/metricbeat-aws-elb-overview.png[] + +[float] +=== `rds` +`period` for `rds` metricset is recommended to be `60s` or multiples of `60s` because Amazon RDS sends metrics and +dimensions to Amazon CloudWatch every minute. + +The rds metricset comes with a predefined dashboard: + +image::./images/metricbeat-aws-rds-overview.png[] [float] === `s3_daily_storage` @@ -70,17 +121,30 @@ https://docs.aws.amazon.com/AmazonS3/latest/user-guide/configure-metrics.html[Ho Configure Request Metrics for S3] for instructions on how to enable request metrics for each S3 bucket. +The s3_daily_storage and s3_request metricset comes with a predefined combined dashboard: + +image::./images/metricbeat-aws-s3-overview.png[] + [float] -=== `cloudwatch` -This metricset gives users the freedom to query metrics from AWS CloudWatch with -any given namespaces or specific instance with a given period. -Please see https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/aws-services-cloudwatch-metrics.html[AWS Services That Publish CloudWatch Metrics] -for a list of AWS services that publish metrics to CloudWatch. +=== `sqs` +CloudWatch metrics for Amazon SQS queues are automatically collected and pushed to CloudWatch every 5 minutes, +the `period` for `sqs` metricset is recommended to be `300s` or multiples of `300s`. + +The sqs metricset comes with a predefined dashboard: + +image::./images/metricbeat-aws-sqs-overview.png[] [float] -=== `rds` -`period` for `rds` metricset is recommended to be `60s` or multiples of `60s` because Amazon RDS sends metrics and -dimensions to Amazon CloudWatch every minute. +=== `usage` +CloudWatch collects metrics that track the usage of some AWS resources. These +metrics correspond to AWS service quotas. Tracking these metrics can help +proactively manage quotas. Service quota usage metrics are in the AWS/Usage +namespace and are collected every minute. Therefore, `period` in aws module +configuration for usage metricset is set to `1m`. + +The usage metricset comes with a predefined dashboard: + +image::./images/metricbeat-aws-usage-overview.png[] [float] == AWS API requests count per metricset diff --git a/x-pack/metricbeat/modules.d/aws.yml.disabled b/x-pack/metricbeat/modules.d/aws.yml.disabled index e581490b967..8069d0d6c0c 100644 --- a/x-pack/metricbeat/modules.d/aws.yml.disabled +++ b/x-pack/metricbeat/modules.d/aws.yml.disabled @@ -4,6 +4,7 @@ - module: aws period: 1m metricsets: + - elb - usage - module: aws period: 5m @@ -22,7 +23,6 @@ metricsets: - ebs - ec2 - - elb - sns - sqs - rds