Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[AWS][EMR] Update metric type and set dimensions fields #6964

Merged
merged 2 commits into from
Jul 14, 2023
Merged

[AWS][EMR] Update metric type and set dimensions fields #6964

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
a0e15c3
Update metric type.
constanca-m Jul 14, 2023
f562e6c
Update changelog.
constanca-m Jul 14, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions packages/aws/changelog.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,9 @@
# newer versions go on top
- version: "1.46.6"
changes:
- description: Update metric type and set dimension fields for AWS EMR data stream.
type: enhancement
link: https://github.com/elastic/integrations/pull/6964
- version: "1.46.5"
changes:
- description: Fix metric type for API Gateway metric fields.
Expand Down
5 changes: 5 additions & 0 deletions packages/aws/data_stream/emr_metrics/fields/ecs.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@
name: cloud
- external: ecs
name: cloud.account.id
dimension: true
- external: ecs
name: cloud.account.name
- external: ecs
Expand All @@ -18,6 +19,7 @@
name: cloud.provider
- external: ecs
name: cloud.region
dimension: true
- external: ecs
name: ecs.version
- external: ecs
Expand Down Expand Up @@ -60,3 +62,6 @@
name: container.labels
- external: ecs
name: container.name
- name: agent.id
external: ecs
dimension: true
64 changes: 32 additions & 32 deletions packages/aws/data_stream/emr_metrics/fields/fields.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,15 +16,15 @@
- name: ContainerAllocated.sum
type: long
description: The number of resource containers allocated by the ResourceManager.
metric_type: counter
metric_type: gauge
- name: ContainerReserved.sum
type: long
description: The number of containers reserved.
metric_type: counter
metric_type: gauge
- name: ContainerPending.sum
type: long
description: The number of containers in the queue that have not yet been allocated.
metric_type: counter
metric_type: gauge
- name: ContainerPendingRatio.avg
type: long
description: The ratio of pending containers to containers allocated
Expand All @@ -33,31 +33,31 @@
- name: AppsCompleted.sum
type: long
description: The number of applications submitted to YARN that have completed.
metric_type: counter
metric_type: gauge
- name: AppsFailed.sum
type: long
description: The number of applications submitted to YARN that have failed to complete.
metric_type: counter
metric_type: gauge
- name: AppsKilled.sum
type: long
description: The number of applications submitted to YARN that have been killed.
metric_type: counter
metric_type: gauge
- name: AppsPending.sum
type: long
description: The number of applications submitted to YARN that are in a pending state.
metric_type: counter
metric_type: gauge
- name: AppsRunning.sum
type: long
description: The number of applications submitted to YARN that are running.
metric_type: counter
metric_type: gauge
- name: AppsSubmitted.sum
type: long
description: The number of applications submitted to YARN.
metric_type: counter
metric_type: gauge
- name: CoreNodesPending.sum
type: long
description: The number of core nodes waiting to be assigned.
metric_type: counter
metric_type: gauge
- name: LiveDataNodes.avg
type: double
description: The percentage of data nodes that are receiving work from Hadoop.
Expand All @@ -66,31 +66,31 @@
- name: MRTotalNodes.sum
type: long
description: The number of nodes presently available to MapReduce jobs.
metric_type: counter
metric_type: gauge
- name: MRActiveNodes.sum
type: long
description: The number of nodes presently running MapReduce tasks or jobs.
metric_type: counter
metric_type: gauge
- name: MRLostNodes.sum
type: long
description: The number of nodes allocated to MapReduce that have been marked in a LOST state.
metric_type: counter
metric_type: gauge
- name: MRUnhealthyNodes.sum
type: long
description: The number of nodes available to MapReduce jobs marked in an UNHEALTHY state.
metric_type: counter
metric_type: gauge
- name: MRDecommissionedNodes.sum
type: long
description: The number of nodes allocated to MapReduce applications that have been marked in a DECOMMISSIONED state.
metric_type: counter
metric_type: gauge
- name: MRRebootedNodes.sum
type: long
description: The number of nodes available to MapReduce that have been rebooted and marked in a REBOOTED state.
metric_type: counter
metric_type: gauge
- name: MultiMasterInstanceGroupNodesRunning.sum
type: long
description: The number of running master nodes.
metric_type: counter
metric_type: gauge
- name: MultiMasterInstanceGroupNodesRunningPercentage.avg
type: double
description: The percentage of master nodes that are running over the requested master node instance count.
Expand All @@ -99,16 +99,16 @@
- name: MultiMasterInstanceGroupNodesRequested.sum
type: long
description: The number of requested master nodes.
metric_type: counter
metric_type: gauge
- name: S3BytesWritten.sum
type: long
description: The number of bytes written to Amazon S3.
metric_type: counter
metric_type: gauge
unit: byte
- name: S3BytesRead.sum
type: long
description: The number of bytes read from Amazon S3.
metric_type: counter
metric_type: gauge
unit: byte
- name: HDFSUtilization.avg
type: double
Expand All @@ -118,12 +118,12 @@
- name: HDFSBytesRead.sum
type: long
description: The number of bytes read from HDFS.
metric_type: counter
metric_type: gauge
unit: byte
- name: HDFSBytesWritten.sum
type: long
description: The number of bytes written to HDFS.
metric_type: counter
metric_type: gauge
unit: byte
- name: MissingBlocks.max
type: long
Expand All @@ -136,21 +136,21 @@
- name: TotalLoad.sum
type: long
description: The total number of concurrent data transfers.
metric_type: counter
metric_type: gauge
- name: MemoryTotalMB.sum
type: long
description: The total amount of memory in the cluster.
metric_type: counter
metric_type: gauge
unit: byte
- name: MemoryReservedMB.sum
type: long
description: The amount of memory reserved.
metric_type: counter
metric_type: gauge
unit: byte
- name: MemoryAvailableMB.sum
type: long
description: The amount of memory available to be allocated.
metric_type: counter
metric_type: gauge
unit: byte
- name: YARNMemoryAvailablePercentage.avg
type: double
Expand All @@ -160,24 +160,24 @@
- name: MemoryAllocatedMB.sum
type: long
description: The amount of memory allocated to the cluster.
metric_type: counter
metric_type: gauge
unit: byte
- name: PendingDeletionBlocks.sum
type: long
description: The number of blocks marked for deletion.
metric_type: counter
metric_type: gauge
- name: UnderReplicatedBlocks.sum
type: long
description: The number of blocks that need to be replicated one or more times.
metric_type: counter
metric_type: gauge
- name: DfsPendingReplicationBlocks.sum
type: long
description: The status of block replication - blocks being replicated, age of replication requests, and unsuccessful replication requests.
metric_type: counter
metric_type: gauge
- name: CapacityRemainingGB.sum
type: long
description: The amount of remaining HDFS disk capacity.
metric_type: counter
metric_type: gauge
unit: byte
- name: TotalUnitsRequested.max
type: long
Expand Down Expand Up @@ -254,7 +254,7 @@
- name: TotalNotebookKernels.sum
type: long
description: The total number of running and idle notebook kernels on the cluster.
metric_type: counter
metric_type: gauge
- name: AutoTerminationIsClusterIdle.avg
type: long
description: Indicates whether the cluster is in use.
Expand Down
Loading