[GCP] Update fields docs and fix inconsistencies after #27974 (#33006)

Co-authored-by: DeDe Morton <[email protected]>
(cherry picked from commit 923e11f)

# Conflicts:
#	x-pack/metricbeat/module/gcp/dataproc/_meta/docs.asciidoc
#	x-pack/metricbeat/module/gcp/firestore/_meta/docs.asciidoc

x-pack/metricbeat/module/gcp/compute/_meta/docs.asciidoc

@@ -1,96 +1,30 @@
 Compute metricset to fetch metrics from https://cloud.google.com/compute/[Compute Engine] Virtual Machines in Google Cloud Platform. No Monitoring or Logging agent is required in your instances to use this metricset.
 
-The `compute` metricset contains all metrics exported from the https://cloud.google.com/monitoring/api/metrics_gcp#gcp-compute[Stackdriver API]. The field names have been left untouched for people already familiar with them.
+The `compute` metricset contains all metrics exported from the https://cloud.google.com/monitoring/api/metrics_gcp#gcp-compute[Stackdriver API]. The field names are aligned to {beats-devguide}/event-conventions.html[Beats naming conventions] with minor modifications to their GCP metrics name counterpart.
 
 Extra labels and metadata are also extracted using the https://cloud.google.com/compute/docs/reference/rest/v1/instances/get[Compute API]. This is enough to get most of the info associated with a metric like Compute labels and metadata and metric specific Labels.
 
 [float]
-=== Metrics and labels
-Here is a list of metrics collected by `compute` metricset:
-
-[float]
-==== firewall
-
-* `instance.firewall.dropped_bytes_count`: Incoming bytes dropped by the firewall.
-- `instance_name`: The name of the VM instance.
-
-* `instance.firewall.dropped_packets_count`: Incoming packets dropped by the firewall.
-- `instance_name`: The name of the VM instance.
-
-[float]
-==== cpu
-
-* `instance.cpu.reserved_cores`: Number of cores reserved on the host of the `instance`.
-- `instance_name`: The name of the VM instance.
-
-* `instance.cpu.utilization`: The fraction of the allocated CPU that is currently in use on the `instance`.
-- `instance_name`: The name of the VM instance.
-
-* `instance.cpu.usage_time`: Usage for all cores in seconds.
-- `instance_name`: The name of the VM instance.
-
-[float]
-==== disk
-
-* `instance.disk.read_bytes_count`: Count of bytes read from disk.
-- `instance_name`: The name of the VM instance.
-- `device_name`: The name of the disk device.
-- `storage_type`: The storage type: `pd-standard`, `pd-ssd`, or `local-ssd`.
-- `device_type`: The disk type: `ephemeral` or `permanent`.
-
-* `instance.disk.read_ops_count`: Count of disk read IO operations.
-- `instance_name`: The name of the VM instance.
-- `device_name`: The name of the disk device.
-- `storage_type`: The storage type: `pd-standard`, `pd-ssd`, or `local-ssd`.
-- `device_type`: The disk type: `ephemeral` or `permanent`.
-
-* `instance.disk.write_bytes_count`: Count of bytes written to disk
-- `instance_name`: The name of the VM instance.
-- `device_name`: The name of the disk device.
-- `storage_type`: The storage type: `pd-standard`, `pd-ssd`, or `local-ssd`.
-- `device_type`: The disk type: `ephemeral` or `permanent`.
-
-* `instance.disk.write_ops_count`: Count of disk write IO operations.
-- `instance_name`: The name of the VM instance.
-- `device_name`: The name of the disk device.
-- `storage_type`: The storage type: `pd-standard`, `pd-ssd`, or `local-ssd`.
-- `device_type`: The disk type: `ephemeral` or `permanent`.
-
-[float]
-==== uptime
-
-* `instance.uptime`: How long the VM has been running, in seconds
-- `instance_name`: The name of the VM instance.
-
-[float]
-==== network
-
-* `instance.network.received_bytes_count`: Count of bytes received from the network
-- `instance_name`: The name of the VM instance.
-- `loadBalanced`: Whether traffic was sent from an L3 loadbalanced IP address assigned to the VM. Traffic that is externally routed from the VM's standard internal or external IP address, such as L7 loadbalanced traffic, is not considered to be loadbalanced in this metric.
-
-* `instance.network.received_packets_count`: Count of packets received from the network
-- `instance_name`: The name of the VM instance.
-- `loadBalanced`: Whether traffic was sent from an L3 loadbalanced IP address assigned to the VM. Traffic that is externally routed from the VM's standard internal or external IP address, such as L7 loadbalanced traffic, is not considered to be loadbalanced in this metric.
-
-* `instance.network.sent_bytes_count`: Count of bytes sent over the network
-- `instance_name`: The name of the VM instance.
-- `loadBalanced`: Whether traffic was sent from an L3 loadbalanced IP address assigned to the VM. Traffic that is externally routed from the VM's standard internal or external IP address, such as L7 loadbalanced traffic, is not considered to be loadbalanced in this metric.
-
-* `instance.network.sent_packets_count`: Count of packets sent over the network
-- `instance_name`: The name of the VM instance.
-- `loadBalanced`: Whether traffic was sent from an L3 loadbalanced IP address assigned to the VM. Traffic that is externally routed from the VM's standard internal or external IP address, such as L7 loadbalanced traffic, is not considered to be loadbalanced in this metric.
-
-[float]
-==== memory
-* `instance.memory.balloon.ram_size`: The total amount of memory in the VM. This metric is only available for VMs that belong to the e2 family.
-- `instance_name`: The name of the VM instance.
-
-* `instance.memory.balloon.ram_used`: Memory currently used in the VM. This metric is only available for VMs that belong to the e2 family.
-- `instance_name`: The name of the VM instance.
-
-* `instance.memory.balloon.swap_in_bytes_count`: The amount of memory read into the guest from its own swap space. This metric is only available for VMs that belong to the e2 family.
-- `instance_name`: The name of the VM instance.
-
-* `instance.memory.balloon.swap_out_bytes_count`: The amount of memory written from the guest to its own swap space. This metric is only available for VMs that belong to the e2 family.
-- `instance_name`: The name of the VM instance.
+=== Labels
+Here is a list of labels collected by `compute` metricset depending on the type of metric being collected:
+
+* `instance_name`: The name of the VM instance.
+Collected with:
+** `gcp.instance.firewall.*`
+** `gcp.instance.cpu.*`
+** `gcp.instance.disk.*`
+** `gcp.instance.memory.*`
+** `gcp.instance.network.*`
+** `gcp.instance.uptime`
+* `device_name`: The name of the disk device.
+Collected with:
+** `gcp.instance.disk.*`
+* `storage_type`: The storage type: `pd-standard`, `pd-ssd`, or `local-ssd`.
+Collected with:
+** `gcp.instance.disk.*`
+* `device_type`: The disk type: `ephemeral` or `permanent`.
+Collected with:
+** `gcp.instance.disk.*`
+* `loadBalanced`: Whether traffic was sent from an L3 loadbalanced IP address assigned to the VM. Traffic that is externally routed from the VM's standard internal or external IP address, such as L7 loadbalanced traffic, is not considered to be loadbalanced in this metric.
+Collected with:
+** `gcp.instance.network.*`

x-pack/metricbeat/module/gcp/dataproc/_meta/docs.asciidoc

@@ -0,0 +1,5 @@
+Dataproc metricset fetches metrics from https://cloud.google.com/dataproc/[Dataproc] in Google Cloud Platform.
+
+The `dataproc` metricset contains all metrics exported from the https://cloud.google.com/monitoring/api/metrics_gcp#gcp-dataproc[GCP Dataproc Monitoring API]. The field names are aligned to {beats-devguide}/event-conventions.html[Beats naming conventions] with minor modifications to their GCP metrics name counterpart.
+
+You can specify a single region to fetch metrics like `us-central1`. Be aware that GCP Storage does not use zones so `us-central1-a` will return nothing. If no region is specified, metrics are returned from all buckets.

x-pack/metricbeat/module/gcp/firestore/_meta/docs.asciidoc

@@ -0,0 +1,5 @@
+Firestore metricset fetches metrics from https://cloud.google.com/firestore/[Firestore] in Google Cloud Platform.
+
+The `firestore` metricset contains all metrics exported from the https://cloud.google.com/monitoring/api/metrics_gcp#gcp-firestore[GCP Firestore Monitoring API]. The field names are aligned to {beats-devguide}/event-conventions.html[Beats naming conventions] with minor modifications to their GCP metrics name counterpart.
+
+You can specify a single region to fetch metrics like `us-central1`. Be aware that GCP Storage does not use zones so `us-central1-a` will return nothing. If no region is specified, metrics are returned from all buckets.

x-pack/metricbeat/module/gcp/gke/_meta/docs.asciidoc

@@ -1,14 +1,9 @@
-`gke` metricset is designed for collecting metrics from https://cloud.google.com/kubernetes-engine[Google Kubernetes
-Engine].
-Google Cloud Monitoring supports Google Kubernetes Engine metrics, as listed in
-https://cloud.google.com/monitoring/api/metrics_kubernetes[Google Cloud Monitoring Kubernetes metrics].
+`gke` metricset is designed for collecting metrics from https://cloud.google.com/kubernetes-engine[Google Kubernetes Engine].
+Google Cloud Monitoring supports Google Kubernetes Engine metrics, as listed in https://cloud.google.com/monitoring/api/metrics_kubernetes[Google Cloud Monitoring Kubernetes metrics].
 
-This metricset collects all GA Kubernetes metrics from Google Cloud Monitoring
-APIs. It leverages under the hood the `metrics` metricset.
+This metricset collects all GA Kubernetes metrics from Google Cloud Monitoring APIs. It leverages under the hood the `metrics` metricset. The field names are aligned to {beats-devguide}/event-conventions.html[Beats naming conventions] with minor modifications to their GCP metrics name counterpart.
 
-We recommend users to define `period: 1m` for this metricset because in Google
-Cloud, GKE monitoring metrics are sampled every 60 seconds.
-Some of the metrics have an ingest delay up to 240 seconds.
+We recommend users to define `period: 1m` for this metricset because in Google Cloud, GKE monitoring metrics are sampled every 60 seconds. Some of the metrics have an ingest delay up to 240 seconds.
 
 [float]
 === Metricset-specific configuration notes

x-pack/metricbeat/module/gcp/loadbalancing/_meta/docs.asciidoc

@@ -1,34 +1,3 @@
 Load Balancing metricset fetches metrics from https://cloud.google.com/load-balancing/[Load Balancing] in Google Cloud Platform.
 
-The `loadbalancing` metricset contains all metrics exported from the https://cloud.google.com/monitoring/api/metrics_gcp#gcp-loadbalancing[Stackdriver API]. The field names have been left untouched for people already familiar with them.
-
-[float]
-=== Metrics
-Here is a list of metrics collected by `loadbalancing` metricset:
-
-[float]
-==== https
-
-- `loadbalancing.https.backend_request_bytes_count`: The number of bytes sent as requests from HTTP/S load balancer to backends.
-- `loadbalancing.https.backend_request_count`: The number of requests served by backends of HTTP/S load balancer.
-- `loadbalancing.https.backend_response_bytes_count`: The number of bytes sent as responses from backends (or cache) to HTTP/S load balancer.
-- `loadbalancing.https.request_bytes_count`: The number of bytes sent as requests from clients to HTTP/S load balancer.
-- `loadbalancing.https.request_count`: The number of requests served by HTTP/S load balancer.
-- `loadbalancing.https.response_bytes_count`: The number of bytes sent as responses from HTTP/S load balancer to clients.
-
-[float]
-==== l3
-
-- `loadbalancing.l3.internal.egress_bytes_count`: The number of bytes sent from ILB backend to client (for TCP flows it's counting bytes on application stream only).
-- `loadbalancing.l3.internal.egress_packets_count`: The number of packets sent from ILB backend to client of the flow.
-- `loadbalancing.l3.internal.ingress_bytes_count`: The number of bytes sent from client to ILB backend (for TCP flows it's counting bytes on application stream only).
-- `loadbalancing.l3.internal.ingress_packets_count`: The number of packets sent from client to ILB backend.
-
-[float]
-==== tcp_ssl_proxy
-
-- `loadbalancing.tcp_ssl_proxy.closed_connections`: Number of connections that were terminated over TCP/SSL proxy.
-- `loadbalancing.tcp_ssl_proxy.egress_bytes_count`: Number of bytes sent from VM to client using proxy.
-- `loadbalancing.tcp_ssl_proxy.ingress_bytes_count`: Number of bytes sent from client to VM using proxy.
-- `loadbalancing.tcp_ssl_proxy.new_connections`: Number of connections that were created over TCP/SSL proxy.
-- `loadbalancing.tcp_ssl_proxy.open_connections`: Current number of outstanding connections through the TCP/SSL proxy.
+The `loadbalancing` metricset contains all metrics exported from the https://cloud.google.com/monitoring/api/metrics_gcp#gcp-loadbalancing[Stackdriver API]. The field names are aligned to {beats-devguide}/event-conventions.html[Beats naming conventions] with minor modifications to their GCP metrics name counterpart.

x-pack/metricbeat/module/gcp/pubsub/_meta/docs.asciidoc

@@ -1,63 +1,5 @@
 PubSub metricsetf fetches metrics from https://cloud.google.com/pubsub/[Pub/Sub] topics and subscriptions in Google Cloud Platform.
 
-The `pubsub` metricset contains all GA stage metrics exported from the https://cloud.google.com/monitoring/api/metrics_gcp#gcp-pubsub[Stackdriver API]. The field names have been left untouched for people already familiar with them.
+The `pubsub` metricset contains all GA stage metrics exported from the https://cloud.google.com/monitoring/api/metrics_gcp#gcp-pubsub[Stackdriver API]. The field names are aligned to {beats-devguide}/event-conventions.html[Beats naming conventions] with minor modifications to their GCP metrics name counterpart.
 
-No special permissions are needed apart from the ones detailed in the module section of the docs.
-
-[float]
-=== Metrics
-Here is a list of metrics collected by `pubsub` metricset:
-
-[float]
-==== Snapshot Metrics
-- `pubsub.snapshot.backlog_bytes`: Total byte size of the messages retained in a snapshot.
-- `pubsub.snapshot.backlog_bytes_by_region`: Total byte size of the messages retained in a snapshot, broken down by Cloud region.
-- `pubsub.snapshot.config_updates_count`: Cumulative count of configuration changes, grouped by operation type and result.
-- `pubsub.snapshot.num_messages`: Number of messages retained in a snapshot. Sampled every 60 seconds.
-- `pubsub.snapshot.num_messages_by_region`: Number of messages retained in a snapshot, broken down by Cloud region.
-- `pubsub.snapshot.oldest_message_age`: Age (in seconds) of the oldest message retained in a snapshot.
-- `pubsub.snapshot.oldest_message_age_by_region`: Age (in seconds) of the oldest message retained in a snapshot, broken down by Cloud region.
-
-[float]
-==== Subscription Metrics
-- `pubsub.subscription.ack_message_count`: Cumulative count of messages acknowledged by Acknowledge requests, grouped by delivery type.
-- `pubsub.subscription.backlog_bytes`: Total byte size of the unacknowledged messages (a.k.a. backlog messages) in a subscription.
-- `pubsub.subscription.byte_cost`: Cumulative cost of operations, measured in bytes. This is used to measure quota utilization.
-- `pubsub.subscription.config_updates_count`: Cumulative count of configuration changes for each subscription, grouped by operation type and result.
-- `pubsub.subscription.dead_letter_message_count`: Cumulative count of messages published to dead letter topic, grouped by result.
-- `pubsub.subscription.mod_ack_deadline_message_count`: Cumulative count of messages whose deadline was updated by ModifyAckDeadline requests, grouped by delivery type.
-- `pubsub.subscription.mod_ack_deadline_message_operation_count`: Cumulative count of ModifyAckDeadline message operations, grouped by result.
-- `pubsub.subscription.mod_ack_deadline_request_count`: Cumulative count of ModifyAckDeadline requests, grouped by result.
-- `pubsub.subscription.num_outstanding_messages`: Number of messages delivered to a subscription's push endpoint, but not yet acknowledged.
-- `pubsub.subscription.num_undelivered_messages`: Number of unacknowledged messages (a.k.a. backlog messages) in a subscription.
-- `pubsub.subscription.oldest_retained_acked_message_age`: Age (in seconds) of the oldest acknowledged message retained in a subscription.
-- `pubsub.subscription.oldest_retained_acked_message_age_by_region`: Age (in seconds) of the oldest acknowledged message retained in a subscription, broken down by Cloud region.
-- `pubsub.subscription.oldest_unacked_message_age`: Age (in seconds) of the oldest unacknowledged message (a.k.a. backlog message) in a subscription.
-- `pubsub.subscription.oldest_unacked_message_age_by_region`: Age (in seconds) of the oldest unacknowledged message in a subscription, broken down by Cloud region.
-- `pubsub.subscription.pull_ack_message_operation_count`: Cumulative count of acknowledge message operations, grouped by result. For a definition of message operations, see Cloud Pub/Sub metric subscription/mod_ack_deadline_message_operation_count.
-- `pubsub.subscription.pull_ack_request_count`: Cumulative count of acknowledge requests, grouped by result.
-- `pubsub.subscription.pull_message_operation_count`: Cumulative count of pull message operations, grouped by result. For a definition of message operations, see Cloud Pub/Sub metric subscription/mod_ack_deadline_message_operation_count.
-- `pubsub.subscription.pull_request_count`: Cumulative count of pull requests, grouped by result.
-- `pubsub.subscription.push_request_count`: Cumulative count of push attempts, grouped by result. Unlike pulls, the push server implementation does not batch user messages. So each request only contains one user message. The push server retries on errors, so a given user message can appear multiple times.
-- `pubsub.subscription.retained_acked_bytes`: otal byte size of the acknowledged messages retained in a subscription.
-- `pubsub.subscription.retained_acked_bytes_by_region`: Total byte size of the acknowledged messages retained in a subscription, broken down by Cloud region.
-- `pubsub.subscription.seek_request_count`: Cumulative count of seek attempts, grouped by result.
-- `pubsub.subscription.sent_message_count`: Cumulative count of messages sent by Cloud Pub/Sub to subscriber clients, grouped by delivery type.
-- `pubsub.subscription.streaming_pull_ack_message_operation_count`: Cumulative count of StreamingPull acknowledge message operations, grouped by result. For a definition of message operations, see Cloud Pub/Sub metric subscription/mod_ack_deadline_message_operation_count.
-- `pubsub.subscription.streaming_pull_ack_request_count`: Cumulative count of streaming pull requests with non-empty acknowledge ids, grouped by result.
-- `pubsub.subscription.streaming_pull_message_operation_count`: Cumulative count of streaming pull message operations, grouped by result. For a definition of message operations, see Cloud Pub/Sub metric <code>subscription/mod_ack_deadline_message_operation_count
-- `pubsub.subscription.streaming_pull_mod_ack_deadline_message_operation_count`: Cumulative count of StreamingPull ModifyAckDeadline operations, grouped by result.
-- `pubsub.subscription.streaming_pull_mod_ack_deadline_request_count`: Cumulative count of streaming pull requests with non-empty ModifyAckDeadline fields, grouped by result.
-- `pubsub.subscription.streaming_pull_response_count`: Cumulative count of streaming pull responses, grouped by result.
-- `pubsub.subscription.unacked_bytes_by_region`: Total byte size of the unacknowledged messages in a subscription, broken down by Cloud region.
-
-[float]
-==== Topic Metrics
-- `pubsub.topic.byte_cost`: Cost of operations, measured in bytes. This is used to measure utilization for quotas.
-- `pubsub.topic.config_updates_count`: Cumulative count of configuration changes, grouped by operation type and result.
-- `pubsub.topic.oldest_retained_acked_message_age_by_region`: Age (in seconds) of the oldest acknowledged message retained in a topic, broken down by Cloud region.
-- `pubsub.topic.oldest_unacked_message_age_by_region`: Age (in seconds) of the oldest unacknowledged message in a topic, broken down by Cloud region.
-- `pubsub.topic.retained_acked_bytes_by_region`: Total byte size of the acknowledged messages retained in a topic, broken down by Cloud region.
-- `pubsub.topic.send_message_operation_count`: Cumulative count of publish message operations, grouped by result. For a definition of message operations, see Cloud Pub/Sub metric subscription/mod_ack_deadline_message_operation_count.
-- `pubsub.topic.send_request_count`: Cumulative count of publish requests, grouped by result.
-- `pubsub.topic.unacked_bytes_by_region`: Total byte size of the unacknowledged messages in a topic, broken down by Cloud region.
+No special permissions are needed apart from the ones detailed in the module section of the docs.