[Alerting] More telemetry for 8.0 based on Event Log data #115318

YulNaumenko · 2021-10-18T03:31:45Z

Summary

Resolves #60315

Added more telemetry based on the Event Log data. Answered the questions from the issue above:

What is the total count of rule executions? How many times did rules encounter execution failures for read? decrypt? unknown? license? What is the count by rule type of execution failures with an error reason of execute?
What is the total count of action executions? What is the count of action execution failures by connector type?
What is the average rule execution time by rule type?
What is the average connector execution time by connector type?

Actions telemetry:

"actions": {
  "count_total": 3,
  "count_by_type": {
    "server_log": 1,
    "email": 2
  },
  "count_active_total": 3,
  "count_active_by_type": {
    "server_log": 1,
    "email": 2
  },
  "count_active_alert_history_connectors": 0,
  "count_actions_executions": 324,
  "count_actions_executions_by_type": {
    "server_log": 160,
    "email": 164
  },
  "count_actions_executions_failured": 164,
  "count_actions_executions_failured_by_type": {
    "email": 164
  },
  "avg_execution_time": 191,
  "avg_execution_time_by_type": {
    "server_log": 1,
    "email": 376
  },
  "alert_history_connector_enabled": false
},

Rules telemetry:

"alerts": {
  "count_total": 2,
  "count_by_type": {
    "index_threshold": 2
  },
  "throttle_time": {
    "min": "0s",
    "avg": "0s",
    "max": "0s"
  },
  "schedule_time": {
    "min": "10s",
    "avg": "35s",
    "max": "60s"
  },
  "connectors_per_alert": {
    "min": 0,
    "avg": 1.5,
    "max": 0
  },
  "count_active_by_type": {
    "index_threshold": 2
  },
  "count_active_total": 2,
  "count_disabled_total": 0,
  "count_rules_executions": 774,
  "count_rules_executions_by_type": {
    "index_threshold": 774
  },
  "count_rules_executions_failured": 0,
  "count_rules_executions_failured_by_reason": {},
  "count_rules_executions_failured_by_reason_by_type": {},
  "avg_execution_time": 1409,
  "avg_execution_time_by_type": {
    "index_threshold": 1409
}

elasticmachine · 2021-10-18T03:31:48Z

Pinging @elastic/kibana-alerting-services (Team:Alerting Services)

pgayvallet · 2021-10-18T15:05:38Z

x-pack/plugins/telemetry_collection_xpack/schema/xpack_plugins.json

+        "count_actions_executions_failured_by_type": {
+          "properties": {
+            "DYNAMIC_KEY": {
+              "type": "long"
+            },


cc @Bamieh @TinaHeiligers that's a lot of additional fields, could you just check this seems fine to you?

We don't have a better alternatives for this sadly. we can handle mapping them via schema patterns on the telemetry cluster.

YulNaumenko · 2021-10-19T04:11:13Z

@elasticmachine merge upstream

YulNaumenko · 2021-10-19T15:39:21Z

@elasticmachine merge upstream

kibanamachine · 2021-10-19T17:56:57Z

💚 Build Succeeded

Metrics [docs]

Public APIs missing comments

Total count of every public API that lacks a comment. Target amount is 0. Run node scripts/build_api_docs --plugin [yourplugin] --stats comments for more detailed information.

id	before	after	diff
`eventLog`	80	81	+1

Unknown metric groups

API count

id	before	after	diff
`eventLog`	80	81	+1

History

💔 Build #162484 failed bb21d80
💔 Build #162468 failed 216bc96
💛 Build #162375 was flaky 91e9b09
💔 Build #162186 failed 30c9ae7
💔 Build #161851 failed 34eb761

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

cc @YulNaumenko

chrisronline

This is looking great so far! I did an initial pass and provided some feedback/questions

x-pack/plugins/actions/server/usage/actions_telemetry.ts

x-pack/plugins/actions/server/usage/actions_usage_collector.ts

x-pack/plugins/actions/server/usage/actions_telemetry.ts

YulNaumenko · 2021-10-25T19:49:50Z

@elasticmachine merge upstream

x-pack/plugins/actions/server/usage/actions_telemetry.ts

YulNaumenko · 2021-10-28T16:29:38Z

@elasticmachine merge upstream

x-pack/plugins/actions/server/usage/actions_telemetry.ts

YulNaumenko · 2021-11-01T14:52:08Z

@elasticmachine merge upstream

YulNaumenko · 2021-11-01T16:49:55Z

@elasticmachine merge upstream

chrisronline

LGTM! Great work!

pmuellr

LGTM with the addition of the date filtering on the event log queries

kibanamachine · 2021-11-02T17:21:58Z

💚 Build Succeeded

Metrics [docs]

Public APIs missing comments

Total count of every public API that lacks a comment. Target amount is 0. Run node scripts/build_api_docs --plugin [yourplugin] --stats comments for more detailed information.

id	before	after	diff
`eventLog`	80	81	+1

Unknown metric groups

API count

id	before	after	diff
`eventLog`	80	81	+1

History

💔 Build #3484 failed 5c6a53e
💔 Build #3480 failed 92f84df
💛 Build #3300 was flaky 1b90e4a
💔 Build #3246 failed e4cde26
💚 Build #2619 succeeded f43d3a9

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

cc @YulNaumenko

…5318) * [Alerting] More telemetry for 8.0 based on Event Log data * fixed event log index mapping * fixed typecheck * fixed tests * added avg aggs * set size to 0 * fixed due to comments * fixed telemetry schema * fixed query * removed test data * added tests * fixed test * fixed query * added exection detalization by day * fixed test * fixed for rules * fixed schema * fixed schema Co-authored-by: Kibana Machine <[email protected]>

…117191) * [Alerting] More telemetry for 8.0 based on Event Log data * fixed event log index mapping * fixed typecheck * fixed tests * added avg aggs * set size to 0 * fixed due to comments * fixed telemetry schema * fixed query * removed test data * added tests * fixed test * fixed query * added exection detalization by day * fixed test * fixed for rules * fixed schema * fixed schema Co-authored-by: Kibana Machine <[email protected]> Co-authored-by: Kibana Machine <[email protected]>

[Alerting] More telemetry for 8.0 based on Event Log data

b1683b3

YulNaumenko self-assigned this Oct 18, 2021

YulNaumenko requested review from a team as code owners October 18, 2021 03:31

fixed event log index mapping

34eb761

pgayvallet reviewed Oct 18, 2021

View reviewed changes

YulNaumenko added 2 commits October 18, 2021 09:06

fixed typecheck

30c9ae7

fixed tests

91e9b09

YulNaumenko removed the v7.16.0 label Oct 19, 2021

added avg aggs

216bc96

Merge branch 'master' into alerting-actions-more-8-0-telemetry

bb21d80

Merge branch 'master' into alerting-actions-more-8-0-telemetry

d657f25

chrisronline suggested changes Oct 20, 2021

View reviewed changes

set size to 0

6005cd4

YulNaumenko requested a review from chrisronline October 22, 2021 00:30

Merged upstream/master

76fad13

pgayvallet approved these changes Oct 25, 2021

View reviewed changes

Merge branch 'master' into alerting-actions-more-8-0-telemetry

ffde2fc

chrisronline reviewed Oct 27, 2021

View reviewed changes

x-pack/plugins/actions/server/usage/actions_telemetry.ts Outdated Show resolved Hide resolved

chrisronline reviewed Oct 27, 2021

View reviewed changes

x-pack/plugins/actions/server/usage/actions_telemetry.ts Outdated Show resolved Hide resolved

YulNaumenko requested review from chrisronline and pmuellr October 28, 2021 03:17

YulNaumenko and others added 5 commits October 27, 2021 20:56

fixed query

899a624

removed test data

26efb80

added tests

712d84c

fixed test

d486697

fixed query

04a5eea

Merge branch 'master' into alerting-actions-more-8-0-telemetry

f43d3a9

chrisronline reviewed Oct 29, 2021

View reviewed changes

x-pack/plugins/actions/server/usage/actions_telemetry.ts Show resolved Hide resolved

Merge branch 'main' into alerting-actions-more-8-0-telemetry

e4cde26

YulNaumenko requested a review from chrisronline November 1, 2021 14:54

Merge branch 'main' into alerting-actions-more-8-0-telemetry

1b90e4a

chrisronline approved these changes Nov 1, 2021

View reviewed changes

YulNaumenko added 4 commits November 1, 2021 22:23

added exection detalization by day

6920190

fixed test

92f84df

fixed for rules

20672e4

fixed schema

5c6a53e

pmuellr approved these changes Nov 2, 2021

View reviewed changes

fixed schema

04717dd

YulNaumenko merged commit 6e14338 into elastic:main Nov 2, 2021

YulNaumenko mentioned this pull request Nov 2, 2021

[8.0] [Alerting] More telemetry for 8.0 based on Event Log data (#115318) #117191

Merged

mikecote mentioned this pull request Mar 3, 2022

Removing task interval from timeout calculation #126707

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Alerting] More telemetry for 8.0 based on Event Log data #115318

[Alerting] More telemetry for 8.0 based on Event Log data #115318

YulNaumenko commented Oct 18, 2021 •

edited

Loading

elasticmachine commented Oct 18, 2021

pgayvallet Oct 18, 2021

Bamieh Oct 25, 2021

YulNaumenko commented Oct 19, 2021

YulNaumenko commented Oct 19, 2021

kibanamachine commented Oct 19, 2021

API count

chrisronline left a comment

YulNaumenko commented Oct 25, 2021

YulNaumenko commented Oct 28, 2021

YulNaumenko commented Nov 1, 2021

YulNaumenko commented Nov 1, 2021

chrisronline left a comment

pmuellr left a comment

kibanamachine commented Nov 2, 2021

API count

[Alerting] More telemetry for 8.0 based on Event Log data #115318

[Alerting] More telemetry for 8.0 based on Event Log data #115318

Conversation

YulNaumenko commented Oct 18, 2021 • edited Loading

Summary

elasticmachine commented Oct 18, 2021

pgayvallet Oct 18, 2021

Choose a reason for hiding this comment

Bamieh Oct 25, 2021

Choose a reason for hiding this comment

YulNaumenko commented Oct 19, 2021

YulNaumenko commented Oct 19, 2021

kibanamachine commented Oct 19, 2021

💚 Build Succeeded

Metrics [docs]

Public APIs missing comments

API count

History

chrisronline left a comment

Choose a reason for hiding this comment

YulNaumenko commented Oct 25, 2021

YulNaumenko commented Oct 28, 2021

YulNaumenko commented Nov 1, 2021

YulNaumenko commented Nov 1, 2021

chrisronline left a comment

Choose a reason for hiding this comment

pmuellr left a comment

Choose a reason for hiding this comment

kibanamachine commented Nov 2, 2021

💚 Build Succeeded

Metrics [docs]

Public APIs missing comments

API count

History

YulNaumenko commented Oct 18, 2021 •

edited

Loading