-
Notifications
You must be signed in to change notification settings - Fork 4.9k
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
[Metricbeat] Add PubSub metricset to Google Cloud Platform module (#1…
- Loading branch information
Showing
15 changed files
with
277 additions
and
4 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,18 @@ | ||
| //// | ||
| This file is generated! See scripts/mage/docs_collector.go | ||
| //// | ||
|
|
||
| [[metricbeat-metricset-googlecloud-pubsub]] | ||
| === Google Cloud Platform pubsub metricset | ||
|
|
||
| beta[] | ||
|
|
||
| include::../../../../x-pack/metricbeat/module/googlecloud/pubsub/_meta/docs.asciidoc[] | ||
|
|
||
| This is a default metricset. If the host module is unconfigured, this metricset is enabled by default. | ||
|
|
||
| ==== Fields | ||
|
|
||
| For a description of each field in the metricset, see the | ||
| <<exported-fields-googlecloud,exported fields>> section. | ||
|
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.
Oops, something went wrong.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,4 +1,5 @@ | ||
| name: googlecloud | ||
| metricsets: | ||
| - compute | ||
| - pubsub | ||
| - loadbalancing |
53 changes: 53 additions & 0 deletions
53
x-pack/metricbeat/module/googlecloud/pubsub/_meta/data_subscription.json
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,53 @@ | ||
| { | ||
| "@timestamp": "2020-01-14T11:46:00.000Z", | ||
| "@metadata": { | ||
| "beat": "metricbeat", | ||
| "type": "_doc", | ||
| "version": "8.0.0" | ||
| }, | ||
| "googlecloud": { | ||
| "labels": { | ||
| "resource": { | ||
| "subscription_id": "test-subscription" | ||
| } | ||
| }, | ||
| "pubsub": { | ||
| "subscription": { | ||
| "num_undelivered_messages": 0, | ||
| "oldest_unacked_message_age": 0, | ||
| "backlog_bytes": 0 | ||
| } | ||
| } | ||
| }, | ||
| "service": { | ||
| "type": "googlecloud" | ||
| }, | ||
| "cloud": { | ||
| "account": { | ||
| "id": "elastic-metricbeat" | ||
| }, | ||
| "provider": "googlecloud" | ||
| }, | ||
| "host": { | ||
| "name": "mcastro" | ||
| }, | ||
| "agent": { | ||
| "version": "8.0.0", | ||
| "type": "metricbeat", | ||
| "ephemeral_id": "b3b8d9da-0094-4d54-b1dc-4d5e103d38aa", | ||
| "hostname": "mcastro", | ||
| "id": "7e36a073-1a32-4a94-b65b-4c7f971fb228" | ||
| }, | ||
| "ecs": { | ||
| "version": "1.4.0" | ||
| }, | ||
| "event": { | ||
| "dataset": "googlecloud.pubsub", | ||
| "module": "googlecloud", | ||
| "duration": 933264720 | ||
| }, | ||
| "metricset": { | ||
| "name": "pubsub", | ||
| "period": 300000 | ||
| } | ||
| } |
61 changes: 61 additions & 0 deletions
61
x-pack/metricbeat/module/googlecloud/pubsub/_meta/data_topic.json
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,61 @@ | ||
| { | ||
| "@timestamp": "2020-01-14T11:42:00.000Z", | ||
| "@metadata": { | ||
| "beat": "metricbeat", | ||
| "type": "_doc", | ||
| "version": "8.0.0" | ||
| }, | ||
| "ecs": { | ||
| "version": "1.4.0" | ||
| }, | ||
| "cloud": { | ||
| "provider": "googlecloud", | ||
| "account": { | ||
| "id": "elastic-metricbeat" | ||
| } | ||
| }, | ||
| "metricset": { | ||
| "name": "pubsub", | ||
| "period": 300000 | ||
| }, | ||
| "event": { | ||
| "duration": 932060852, | ||
| "dataset": "googlecloud.pubsub", | ||
| "module": "googlecloud" | ||
| }, | ||
| "googlecloud": { | ||
| "labels": { | ||
| "resource": { | ||
| "subscription_id": "test-subscription" | ||
| } | ||
| }, | ||
| "pubsub": { | ||
| "topic": { | ||
| "message_sizes": { | ||
| "bucket_options": { | ||
| "Options": { | ||
| "ExponentialBuckets": { | ||
| "growth_factor": 4, | ||
| "scale": 1, | ||
| "num_finite_buckets": 16 | ||
| } | ||
| } | ||
| } | ||
| } | ||
| } | ||
| } | ||
| }, | ||
| "service": { | ||
| "type": "googlecloud" | ||
| }, | ||
| "host": { | ||
| "name": "mcastro" | ||
| }, | ||
| "agent": { | ||
| "ephemeral_id": "b3b8d9da-0094-4d54-b1dc-4d5e103d38aa", | ||
| "hostname": "mcastro", | ||
| "id": "7e36a073-1a32-4a94-b65b-4c7f971fb228", | ||
| "version": "8.0.0", | ||
| "type": "metricbeat" | ||
| } | ||
| } |
25 changes: 25 additions & 0 deletions
25
x-pack/metricbeat/module/googlecloud/pubsub/_meta/docs.asciidoc
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,25 @@ | ||
| PubSub Metricset to fetch metrics from https://cloud.google.com/pubsub/[Pub/Sub] topics and subscriptions in Google Cloud Platform. | ||
|
|
||
| The `pubsub` Metricset contains all metrics exported from the https://cloud.google.com/monitoring/api/metrics_gcp#gcp-pubsub[Stackdriver API]. The field names have been left untouched for people already familiar with them. | ||
|
|
||
| No special permissions are needed apart from the ones detailed in the module section of the docs. | ||
|
|
||
| - `pubsub.subscription.ack_message_count`: Cumulative count of messages acknowledged by Acknowledge requests, grouped by delivery type. | ||
| - `pubsub.subscription.backlog_bytes`: Total byte size of the unacknowledged messages (a.k.a. backlog messages) in a subscription. | ||
| - `pubsub.subscription.num_outstanding_messages`: Number of messages delivered to a subscription's push endpoint, but not yet acknowledged. | ||
| - `pubsub.subscription.num_undelivered_messages`: Number of unacknowledged messages (a.k.a. backlog messages) in a subscription. | ||
| - `pubsub.subscription.oldest_unacked_message_age`: Age (in seconds) of the oldest unacknowledged message (a.k.a. backlog message) in a subscription. | ||
| - `pubsub.subscription.pull_ack_message_operation_count`: Cumulative count of acknowledge message operations, grouped by result. For a definition of message operations, see Cloud Pub/Sub metric subscription/mod_ack_deadline_message_operation_count. | ||
| - `pubsub.subscription.pull_ack_request_count`: Cumulative count of acknowledge requests, grouped by result. | ||
| - `pubsub.subscription.pull_message_operation_count`: Cumulative count of pull message operations, grouped by result. For a definition of message operations, see Cloud Pub/Sub metric subscription/mod_ack_deadline_message_operation_count. | ||
| - `pubsub.subscription.pull_request_count`: Cumulative count of pull requests, grouped by result. | ||
| - `pubsub.subscription.push_request_count`: Cumulative count of push attempts, grouped by result. Unlike pulls, the push server implementation does not batch user messages. So each request only contains one user message. The push server retries on errors, so a given user message can appear multiple times. | ||
| - `pubsub.subscription.push_request_latencies`: Distribution of push request latencies (in microseconds), grouped by result. | ||
| - `pubsub.subscription.sent_message_count`: Cumulative count of messages sent by Cloud Pub/Sub to subscriber clients, grouped by delivery type. | ||
| - `pubsub.subscription.streaming_pull_ack_message_operation_count`: Cumulative count of StreamingPull acknowledge message operations, grouped by result. For a definition of message operations, see Cloud Pub/Sub metric subscription/mod_ack_deadline_message_operation_count. | ||
| - `pubsub.subscription.streaming_pull_ack_request_count`: Cumulative count of streaming pull requests with non-empty acknowledge ids, grouped by result. | ||
| - `pubsub.subscription.streaming_pull_message_operation_count`: Cumulative count of streaming pull message operations, grouped by result. For a definition of message operations, see Cloud Pub/Sub metric <code>subscription/mod_ack_deadline_message_operation_count | ||
| - `pubsub.subscription.streaming_pull_response_count`: Cumulative count of streaming pull responses, grouped by result. | ||
| - `pubsub.topic.streaming_pull_response_count`: Cumulative count of streaming pull responses, grouped by result. | ||
| - `pubsub.topic.send_message_operation_count`: Cumulative count of publish message operations, grouped by result. For a definition of message operations, see Cloud Pub/Sub metric subscription/mod_ack_deadline_message_operation_count. | ||
| - `pubsub.topic.send_request_count`: Cumulative count of publish requests, grouped by result. |
70 changes: 70 additions & 0 deletions
70
x-pack/metricbeat/module/googlecloud/pubsub/_meta/fields.yml
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,70 @@ | ||
| - name: pubsub | ||
| release: beta | ||
| type: group | ||
| description: Google Cloud PubSub metrics | ||
| fields: | ||
| - name: subscription | ||
| type: group | ||
| description: Suscription related metrics | ||
| fields: | ||
| - name: ack_message_count | ||
| type: long | ||
| description: Cumulative count of messages acknowledged by Acknowledge requests, grouped by delivery type. | ||
| - name: backlog_bytes | ||
| type: long | ||
| description: Total byte size of the unacknowledged messages (a.k.a. backlog messages) in a subscription. | ||
| - name: num_outstanding_messages | ||
| type: long | ||
| description: Number of messages delivered to a subscription's push endpoint, but not yet acknowledged. | ||
| - name: num_undelivered_messages | ||
| type: long | ||
| description: Number of unacknowledged messages (a.k.a. backlog messages) in a subscription. | ||
| - name: oldest_unacked_message_age | ||
| type: long | ||
| description: Age (in seconds) of the oldest unacknowledged message (a.k.a. backlog message) in a subscription. | ||
| - name: pull_ack_message_operation_count | ||
| type: long | ||
| description: Cumulative count of acknowledge message operations, grouped by result. For a definition of message operations, see Cloud Pub/Sub metric subscription/mod_ack_deadline_message_operation_count. | ||
| - name: pull_ack_request_count | ||
| type: long | ||
| description: Cumulative count of acknowledge requests, grouped by result. | ||
| - name: pull_message_operation_count | ||
| type: long | ||
| description: Cumulative count of pull message operations, grouped by result. For a definition of message operations, see Cloud Pub/Sub metric subscription/mod_ack_deadline_message_operation_count. | ||
| - name: pull_request_count | ||
| type: long | ||
| description: Cumulative count of pull requests, grouped by result. | ||
| - name: push_request_count | ||
| type: long | ||
| description: Cumulative count of push attempts, grouped by result. Unlike pulls, the push server implementation does not batch user messages. So each request only contains one user message. The push server retries on errors, so a given user message can appear multiple times. | ||
| - name: push_request_latencies | ||
| type: long | ||
| description: Distribution of push request latencies (in microseconds), grouped by result. | ||
| - name: sent_message_count | ||
| type: long | ||
| description: Cumulative count of messages sent by Cloud Pub/Sub to subscriber clients, grouped by delivery type. | ||
| - name: streaming_pull_ack_message_operation_count | ||
| type: long | ||
| description: Cumulative count of StreamingPull acknowledge message operations, grouped by result. For a definition of message operations, see Cloud Pub/Sub metric subscription/mod_ack_deadline_message_operation_count. | ||
| - name: streaming_pull_ack_request_count | ||
| type: long | ||
| description: Cumulative count of streaming pull requests with non-empty acknowledge ids, grouped by result. | ||
| - name: streaming_pull_message_operation_count | ||
| type: long | ||
| description: Cumulative count of streaming pull message operations, grouped by result. For a definition of message operations, see Cloud Pub/Sub metric <code>subscription/mod_ack_deadline_message_operation_count | ||
| - name: streaming_pull_response_count | ||
| type: long | ||
| description: Cumulative count of streaming pull responses, grouped by result. | ||
| - name: topic | ||
| type: group | ||
| description: Topic related metrics | ||
| fields: | ||
| - name: streaming_pull_response_count | ||
| type: long | ||
| description: Cumulative count of streaming pull responses, grouped by result. | ||
| - name: send_message_operation_count | ||
| type: long | ||
| description: Cumulative count of publish message operations, grouped by result. For a definition of message operations, see Cloud Pub/Sub metric subscription/mod_ack_deadline_message_operation_count. | ||
| - name: send_request_count | ||
| type: long | ||
| description: Cumulative count of publish requests, grouped by result. |
Oops, something went wrong.