Skip to content

Commit

Permalink
Add cost warning to aws, googlecloud and azure doc (elastic#20033) (e…
Browse files Browse the repository at this point in the history
…lastic#20120)

* Add cost warning to aws googlecloud and azure doc

(cherry picked from commit 62b390f)
  • Loading branch information
kaiyan-sheng authored Jul 22, 2020
1 parent 36d28ba commit 13f429c
Show file tree
Hide file tree
Showing 6 changed files with 28 additions and 6 deletions.
5 changes: 4 additions & 1 deletion metricbeat/docs/modules/aws.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -10,10 +10,12 @@ This file is generated! See scripts/mage/docs_collector.go

This module periodically fetches monitoring metrics from AWS CloudWatch using
https://docs.aws.amazon.com/AmazonCloudWatch/latest/APIReference/API_GetMetricData.html[GetMetricData API] for AWS services.
Note: extra AWS charges on GetMetricData API requests will be generated by this module.

All metrics are enabled by default.

IMPORTANT: Extra AWS charges on CloudWatch API requests will be generated by this
module. Please see <<aws-api-requests,AWS API requests>> for more details.

[float]
== Module-specific configuration notes

Expand Down Expand Up @@ -196,6 +198,7 @@ real-time metrics for users to better understand the performance of their web
applications and services.

[float]
[[aws-api-requests]]
== AWS API requests count per metricset
This session is to document what are the AWS API called made by each metricset
in `aws` module. This will be useful for users to estimate costs for using `aws`
Expand Down
4 changes: 4 additions & 0 deletions metricbeat/docs/modules/azure.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,9 @@ Additional azure API calls will be executed in order to retrieve information reg

The azure module mericsets are `monitor`, `compute_vm` and `compute_vm_scaleset`

IMPORTANT: Extra Azure charges on metric queries may be generated by this module.
Please see <<<<azure-api-cost,additional notes about metrics and costs>> for more details.

[float]
=== Dashboards

Expand Down Expand Up @@ -119,6 +122,7 @@ so the `period` for `billing` metricset should be `24h` or multiples of `24h`.
This metricset will collect application insights metrics, the `period` (interval) for the `app-insights` metricset is set by default at `300s`.

[float]
[[azure-api-cost]]
== Additional notes about metrics and costs

Costs: Metric queries are charged based on the number of standard API calls. More information on pricing here https://azure.microsoft.com/id-id/pricing/details/monitor/.
Expand Down
8 changes: 6 additions & 2 deletions metricbeat/docs/modules/googlecloud.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,10 @@ beta[]

This module periodically fetches monitoring metrics from Google Cloud Platform using
https://cloud.google.com/monitoring/api/metrics_gcp[Stackdriver Monitoring API] for Google Cloud Platform services.
Note: extra GCP charges on Stackdriver Monitoring API requests will be generated by this module.

IMPORTANT: Extra GCP charges on Stackdriver Monitoring API requests may be
generated by this module. Please see <<gcp-api-requests,rough estimation of the number of API calls>>
for more details.

[float]
== Module config and parameters
Expand Down Expand Up @@ -138,7 +141,8 @@ GCP monitoring data has a up to 240 seconds latency, which means latest monitori
In googlecloud module, metrics are collected based on this ingest delay, which is also obtained from ListMetricDescriptors API.

[float]
=== Rough estimation of the number of API Calls
[[gcp-api-requests]]
=== Rough estimation of the number of API calls
Google Cloud Platform pricing depends of the number of requests you do to their API's. Here you have some information that you can use to make an estimation of the pricing you should expect. For example, imagine that you have a Compute Metricset activated and you don't want to exclude labels. You have a total of 20 instances running in a particular GCP project, region and zone.

For example, if Compute Metricset fetches 14 metrics (which is the number of metrics fetched in the early beta version). Each of those metrics will attempt an API call to Compute API to retrieve also their metadata. Because you have 20 different instances, the total number of API calls that will be done on each refresh period are: 14 metrics + 20 instances = 34 API requests every 5 minutes if that is your current Period. 9792 API requests per day with one zone. If you add 2 zones more with the same amount of instances you'll have 19584 API requests per day (9792 on each zone) or around 587520 per month for the Compute Metricset. This maths must be done for each different Metricset with slight variations.
Expand Down
5 changes: 4 additions & 1 deletion x-pack/metricbeat/module/aws/_meta/docs.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -2,10 +2,12 @@

This module periodically fetches monitoring metrics from AWS CloudWatch using
https://docs.aws.amazon.com/AmazonCloudWatch/latest/APIReference/API_GetMetricData.html[GetMetricData API] for AWS services.
Note: extra AWS charges on GetMetricData API requests will be generated by this module.

All metrics are enabled by default.

IMPORTANT: Extra AWS charges on CloudWatch API requests will be generated by this
module. Please see <<aws-api-requests,AWS API requests>> for more details.

[float]
== Module-specific configuration notes

Expand Down Expand Up @@ -188,6 +190,7 @@ real-time metrics for users to better understand the performance of their web
applications and services.

[float]
[[aws-api-requests]]
== AWS API requests count per metricset
This session is to document what are the AWS API called made by each metricset
in `aws` module. This will be useful for users to estimate costs for using `aws`
Expand Down
4 changes: 4 additions & 0 deletions x-pack/metricbeat/module/azure/_meta/docs.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,9 @@ Additional azure API calls will be executed in order to retrieve information reg

The azure module mericsets are `monitor`, `compute_vm` and `compute_vm_scaleset`

IMPORTANT: Extra Azure charges on metric queries may be generated by this module.
Please see <<<<azure-api-cost,additional notes about metrics and costs>> for more details.

[float]
=== Dashboards

Expand Down Expand Up @@ -111,6 +114,7 @@ so the `period` for `billing` metricset should be `24h` or multiples of `24h`.
This metricset will collect application insights metrics, the `period` (interval) for the `app-insights` metricset is set by default at `300s`.

[float]
[[azure-api-cost]]
== Additional notes about metrics and costs

Costs: Metric queries are charged based on the number of standard API calls. More information on pricing here https://azure.microsoft.com/id-id/pricing/details/monitor/.
Expand Down
8 changes: 6 additions & 2 deletions x-pack/metricbeat/module/googlecloud/_meta/docs.asciidoc
Original file line number Diff line number Diff line change
@@ -1,6 +1,9 @@
This module periodically fetches monitoring metrics from Google Cloud Platform using
https://cloud.google.com/monitoring/api/metrics_gcp[Stackdriver Monitoring API] for Google Cloud Platform services.
Note: extra GCP charges on Stackdriver Monitoring API requests will be generated by this module.

IMPORTANT: Extra GCP charges on Stackdriver Monitoring API requests may be
generated by this module. Please see <<gcp-api-requests,rough estimation of the number of API calls>>
for more details.

[float]
== Module config and parameters
Expand Down Expand Up @@ -128,7 +131,8 @@ GCP monitoring data has a up to 240 seconds latency, which means latest monitori
In googlecloud module, metrics are collected based on this ingest delay, which is also obtained from ListMetricDescriptors API.

[float]
=== Rough estimation of the number of API Calls
[[gcp-api-requests]]
=== Rough estimation of the number of API calls
Google Cloud Platform pricing depends of the number of requests you do to their API's. Here you have some information that you can use to make an estimation of the pricing you should expect. For example, imagine that you have a Compute Metricset activated and you don't want to exclude labels. You have a total of 20 instances running in a particular GCP project, region and zone.

For example, if Compute Metricset fetches 14 metrics (which is the number of metrics fetched in the early beta version). Each of those metrics will attempt an API call to Compute API to retrieve also their metadata. Because you have 20 different instances, the total number of API calls that will be done on each refresh period are: 14 metrics + 20 instances = 34 API requests every 5 minutes if that is your current Period. 9792 API requests per day with one zone. If you add 2 zones more with the same amount of instances you'll have 19584 API requests per day (9792 on each zone) or around 587520 per month for the Compute Metricset. This maths must be done for each different Metricset with slight variations.
Expand Down

0 comments on commit 13f429c

Please sign in to comment.