Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[DOC] Add new documentation alerting monitors and anomaly detectors #3449

Closed
wants to merge 244 commits into from
Closed
Show file tree
Hide file tree
Changes from 29 commits
Commits
Show all changes
244 commits
Select commit Hold shift + click to select a range
9dc0b92
Content development
vagimeli Mar 13, 2023
b412c24
Content development
vagimeli Mar 13, 2023
99f2369
Content development
vagimeli Mar 13, 2023
c3547cf
Content development
vagimeli Mar 14, 2023
759df32
Content development
vagimeli Mar 14, 2023
c49b277
Content planning
vagimeli Mar 21, 2023
6f43f64
Content planning
vagimeli Mar 21, 2023
aaeb974
Content planning
vagimeli Mar 21, 2023
2703be2
Content planning
vagimeli Mar 21, 2023
ac1caa8
Content planning
vagimeli Mar 22, 2023
1f686af
Content planning
vagimeli Mar 22, 2023
fd47625
Content planning
vagimeli Mar 22, 2023
9c55521
rename file
vagimeli Mar 22, 2023
e5f63d6
Content planning
vagimeli Mar 22, 2023
54eb605
Content planning
vagimeli Mar 22, 2023
f5d6649
Content planning
vagimeli Mar 22, 2023
11f2b13
Content planning
vagimeli Mar 23, 2023
a6ab992
Content planning
vagimeli Mar 23, 2023
883c41e
Content planning
vagimeli Mar 24, 2023
01608d9
Content planning
vagimeli Mar 24, 2023
15957d7
Content planning
vagimeli Mar 24, 2023
082850d
Content planning
vagimeli Mar 24, 2023
3e5d168
Content planning
vagimeli Mar 24, 2023
536fa48
Content planning
vagimeli Mar 27, 2023
86a1d6b
Content planning
vagimeli Mar 27, 2023
e050d74
Content planning
vagimeli Mar 27, 2023
8e2f8f0
Content planning
vagimeli Mar 27, 2023
9474493
Delete augmentation.md
vagimeli Mar 27, 2023
b4441bd
Delete managing-alerts.md
vagimeli Mar 27, 2023
c88cb20
Address tech review feedback
vagimeli May 1, 2023
081bf73
Address tech review feedback
vagimeli May 2, 2023
1326bf4
Address tech review feedback
vagimeli May 2, 2023
1caf110
Address tech review feedback
vagimeli May 3, 2023
f64c5b5
Writing
vagimeli May 3, 2023
9c86fc5
Fix link to data stream explanation (#3431)
FrcMoya Mar 16, 2023
9a8832b
Add documentation for Mutate strings (#2950)
carolxob Mar 16, 2023
2b2b535
Add reference to hybrid mmap extensions setting in k-NN (#3262)
kolchfa-aws Mar 16, 2023
52f698e
Remove GPU reference from approximate k-NN (#3263)
kolchfa-aws Mar 16, 2023
118c02c
Update put-mapping.md: add "dynamic" (#3384)
varfrog Mar 16, 2023
235b522
Fix table formatting for http_source doc. (#3451)
carolxob Mar 16, 2023
618633c
Fix table header in Bounded blocking doc. (#3450)
carolxob Mar 16, 2023
7f2662f
Add 1.3.9 version to version history (#3467)
kolchfa-aws Mar 16, 2023
ce5e1a1
add warning banner to querqy page (#3494)
ariamarble Mar 16, 2023
03af53c
Update example response with latest build (#3522)
Mar 17, 2023
d203359
Update vector size in Approximate KNN doc (#3515)
carolxob Mar 20, 2023
00c9e23
Remove typo from ML cluster settings (#3529)
Naarcha-AWS Mar 20, 2023
585ff04
Update OpenSearch Java Client documentation: update version, add Apac…
reta Mar 21, 2023
07ecce1
Change url for opensearch_security.openid.scope configuration setting…
LucaBlackDragon Mar 21, 2023
3661d16
Use npm registry instead of tgz file (#2754)
rupal-bq Mar 21, 2023
50d2118
updatetobreaking (#3547)
hdhalter Mar 23, 2023
c5b61e7
Update permissions.md (#3548)
monusingh-1 Mar 23, 2023
9648b2c
Add documentation to rename opensearch_dashboards predefined roles to…
cwillum Mar 23, 2023
c8e296e
Update to Breaking changes file (#3550)
hdhalter Mar 23, 2023
5530e93
Update prometheusmetrics.md (#3588)
nitincd Mar 28, 2023
6a28716
sqo-odbc brokenlinks (#3587)
hdhalter Mar 28, 2023
d9bf5dc
Add documentation for jwt_clock_skew_tolerance_seconds setting that r…
cwillum Mar 28, 2023
53bc239
Fix broken navigation for Notifications API (#3609)
Naarcha-AWS Mar 28, 2023
013d40c
Fix an error in building an index from a model (#3615)
kolchfa-aws Mar 29, 2023
0291b11
Remove LICENSE from possible endpint list (#2907)
Jakob3xD Mar 29, 2023
f81ea7a
Add Mutate event processor doc. (#3234)
carolxob Apr 3, 2023
09dd218
Add CI with link checker. (#3584)
dblock Apr 4, 2023
ed98bdc
Adding documentation for behavior of indices.recovery.max_bytes_per_s…
AmiStrn Apr 4, 2023
5718a7f
Refactor k-NN filter search (#3613)
kolchfa-aws Apr 5, 2023
fb5b1be
Add Security information to Security Analytics documentation (#3184)
cwillum Apr 5, 2023
e18b2b1
Add path to the script for the securityadmin.sh troubleshooting topic…
cwillum Apr 5, 2023
9df8c62
Link checker update: fail on PRs and run for all branches (#3690)
kolchfa-aws Apr 6, 2023
f6b6204
Specify versions in the link checker (#3696)
kolchfa-aws Apr 6, 2023
1637a6f
Add Parse JSON processor doc. (#3237)
carolxob Apr 6, 2023
979e2dd
Update MAINTAINERS.md (#3641)
Naarcha-AWS Apr 6, 2023
d7937c2
Update CODEOWNERS to reflect team (#3639)
Naarcha-AWS Apr 6, 2023
14e528f
Fixing brand and trademark links (#3647)
dtaivpp Apr 6, 2023
d8c83f4
Remove dynamic templates from a mapping example (#3698)
kolchfa-aws Apr 8, 2023
0323fc1
Updates Trace Analytics overview (#3727)
natebower Apr 10, 2023
102a757
Remove the list of branches for link checker (#3731)
kolchfa-aws Apr 10, 2023
d015a8c
Fix put rollup job api documentation (#3637)
tanqiuliu Apr 10, 2023
22f265c
Add documentation for Searchable Snapshots GA release (#3462)
ariamarble Apr 11, 2023
2787dc0
Add Trace peer forwarder doc. (#3238)
carolxob Apr 11, 2023
e9444aa
Update java client version (#3753)
VachaShah Apr 11, 2023
61c36d8
Add config parameters for traced models (#3456)
Naarcha-AWS Apr 12, 2023
c3d450a
Modify link checker to run once a week and not to run in forks (#3759)
kolchfa-aws Apr 12, 2023
45d7026
Add auto-truncation to pretrained models (#3756)
Naarcha-AWS Apr 13, 2023
83feb99
Use the modified dates of files in the sitemap (#3774)
AMoo-Miki Apr 14, 2023
fcf610f
Add additonal Snapshot Restore (#3783)
Naarcha-AWS Apr 14, 2023
9fbdb13
Add inline icons to formatting guide (#3785)
kolchfa-aws Apr 17, 2023
df50734
Add maximum allowed monitor number. (#3732)
carolxob Apr 17, 2023
140f933
Update link-checker.yml (#3787)
kolchfa-aws Apr 18, 2023
8db76fb
Add documentation for segment replication GA release (#3461)
ariamarble Apr 18, 2023
b570ab7
Add filtering data in maps doc. (#3640)
carolxob Apr 18, 2023
9494b62
Add documentation for wait_for_completion API (#3458)
ariamarble Apr 20, 2023
9c5ad62
Add Anomaly detector plugin doc. (#3569)
carolxob Apr 20, 2023
d33a893
Add E2E acknowledgements (#3811)
Naarcha-AWS Apr 20, 2023
7e85c9f
Add opensearch sink security (#3809)
lizsnyder Apr 20, 2023
5b20e59
Add list to map processor (#3806)
Naarcha-AWS Apr 20, 2023
c3988dd
Add documentation for IM UI index templates (#3459)
ariamarble Apr 20, 2023
ed6463f
Update Maintainer.md to match CODEOWNER (#3831)
ananzh Apr 20, 2023
144f668
Add documentation for dynamic configuration of tenancy in Dashboards …
cwillum Apr 21, 2023
6d281e8
Content updates for segrep - benchmarking and recommended settings (#…
ariamarble Apr 21, 2023
5469c23
Update baseline maintainer and CODEOWNER format (#3832)
ananzh Apr 24, 2023
8d7716a
Add editorial to maintainers and codeowners (#3840)
Naarcha-AWS Apr 24, 2023
e6e20f9
Add new settings for SAML and OIDC that allow for cookie splitting (#…
cwillum Apr 24, 2023
36f4227
Add Event Analytics visualizations in Dashboards feature (#3752)
vagimeli Apr 24, 2023
03b91c6
Add examples to Key value page (#3105)
carolxob Apr 25, 2023
de32efd
Add max_retries option (#3835)
lizsnyder Apr 25, 2023
6dbff1f
Updates the Data Prepper S3 source documentation (#3813)
dlvenable Apr 25, 2023
4bd885d
[DOC] Add multidata source Dev Tools integration (#3653)
vagimeli Apr 25, 2023
cff67c1
Add OTel metrics raw processor doc to doc website. (#3235)
carolxob Apr 25, 2023
0555c63
Add OTel logs source doc to doc website. (#3608)
carolxob Apr 25, 2023
0252082
Add OTel trace group processor to doc website. (#3236)
carolxob Apr 25, 2023
a070189
Add DLQ to OpenSearch sink configuration options. (#3743)
carolxob Apr 26, 2023
7772ace
readmeupdates (#3881)
hdhalter Apr 26, 2023
bc3d644
Updates to repo (#3882)
Naarcha-AWS Apr 27, 2023
5500edf
[DOC] Create Dashboard list extensions documentation (#3443)
vagimeli Apr 27, 2023
2486b2a
Add metrics correlation algorithm (#3877)
Naarcha-AWS Apr 27, 2023
e5cd708
Add DLQ and aggregate processor edits. Baseline processor names (#3880)
Naarcha-AWS Apr 27, 2023
9ceab84
Updates the images used for Data Prepper use-cases to use images with…
dlvenable Apr 27, 2023
fec44e8
Adds general S3 use-case documentation. #3833 (#3842)
dlvenable Apr 27, 2023
4b5cc82
Fix typo on processor page. (#3903)
Naarcha-AWS Apr 27, 2023
d2d2267
Add ML fault tolerance (#3803)
Naarcha-AWS May 1, 2023
d60f6b4
Fix REST method names (#3789)
lukas-vlcek May 1, 2023
c08707d
Fix spelling error on upgrade page (#3910)
Naarcha-AWS May 2, 2023
3eb7a5b
Update segment replication backpressure (#3839)
kolchfa-aws May 2, 2023
4fc5852
Add Search Relevance API documentation (#3770)
kolchfa-aws May 2, 2023
5b37f7d
Add primary shard balancing documentation (#3769)
kolchfa-aws May 2, 2023
93aa981
Add documentation for Shard Hotspot Identification RCA (#3741)
ariamarble May 2, 2023
8ade18a
Add flat object field type (#3714)
kolchfa-aws May 2, 2023
80460f0
Add CAT segment replication API (#2937)
kolchfa-aws May 2, 2023
f515759
Add Maps Stats API documentation (#3804)
kolchfa-aws May 2, 2023
8b8dec0
Remove precision from TERMS and capitalize Logstash (#3920)
kolchfa-aws May 2, 2023
d32972b
Add new ISM errors (#3924)
Naarcha-AWS May 2, 2023
e33c867
Add correlation engine to Security Analytics documentation (#3814)
cwillum May 2, 2023
1816607
Adds per document monitor enhancements for 2.7 (#3925)
Naarcha-AWS May 2, 2023
2eb69ce
Add 2.7 version PR (#3917)
kolchfa-aws May 2, 2023
91227c8
Add version history 2.7 (#3918)
kolchfa-aws May 2, 2023
c13dad8
Add 2.7 documentation release notes (#3919)
kolchfa-aws May 2, 2023
5d9221e
Remove a known limitation from searchable snapshot (#3942)
kotwanikunal May 2, 2023
656af40
Remove experimental note from multi data source feature (#3948)
hdhalter May 3, 2023
4f16c52
fix table headings (#3943)
hdhalter May 3, 2023
6593c9e
Add style workflow (#3909)
kolchfa-aws May 3, 2023
c8bdf30
Writing
vagimeli May 4, 2023
e6b5e96
Writing cont.
vagimeli May 4, 2023
3732561
Revert "Add style workflow (#3909)"
vagimeli May 4, 2023
0580344
Revert "fix table headings (#3943)"
vagimeli May 4, 2023
204a0ef
Revert "Remove experimental note from multi data source feature (#3948)"
vagimeli May 4, 2023
db5d5d7
Revert "Remove a known limitation from searchable snapshot (#3942)"
vagimeli May 4, 2023
28c2b42
Revert "Add 2.7 documentation release notes (#3919)"
vagimeli May 4, 2023
185b4c7
Revert "Add version history 2.7 (#3918)"
vagimeli May 4, 2023
e7c2c7a
Revert "Add 2.7 version PR (#3917)"
vagimeli May 4, 2023
8691d45
Revert "Adds per document monitor enhancements for 2.7 (#3925)"
vagimeli May 4, 2023
ea85bf4
Revert "Add correlation engine to Security Analytics documentation (#…
vagimeli May 4, 2023
784a48e
Revert "Add new ISM errors (#3924)"
vagimeli May 4, 2023
b07aa53
Revert "Remove precision from TERMS and capitalize Logstash (#3920)"
vagimeli May 4, 2023
8389844
Revert "Add Maps Stats API documentation (#3804)"
vagimeli May 4, 2023
28bc9de
Revert "Add CAT segment replication API (#2937)"
vagimeli May 4, 2023
5ead78c
Revert "Add flat object field type (#3714)"
vagimeli May 4, 2023
92cb886
Revert "Add documentation for Shard Hotspot Identification RCA (#3741)"
vagimeli May 4, 2023
5f696bd
Revert "Add primary shard balancing documentation (#3769)"
vagimeli May 4, 2023
2873584
Revert "Add Search Relevance API documentation (#3770)"
vagimeli May 4, 2023
8d7bd52
Revert "Update segment replication backpressure (#3839)"
vagimeli May 4, 2023
5e15c8d
Revert "Fix spelling error on upgrade page (#3910)"
vagimeli May 4, 2023
9d07b52
Revert "Fix REST method names (#3789)"
vagimeli May 4, 2023
c9ffb23
Revert "Add ML fault tolerance (#3803)"
vagimeli May 4, 2023
cf8865a
Revert "Fix typo on processor page. (#3903)"
vagimeli May 4, 2023
6c91683
Revert "Adds general S3 use-case documentation. #3833 (#3842)"
vagimeli May 4, 2023
ae53b7d
Revert "Updates the images used for Data Prepper use-cases to use ima…
vagimeli May 4, 2023
755b798
Revert "Add DLQ and aggregate processor edits. Baseline processor nam…
vagimeli May 4, 2023
212e0f5
Revert "Add metrics correlation algorithm (#3877)"
vagimeli May 4, 2023
0cfe8e5
Revert "[DOC] Create Dashboard list extensions documentation (#3443)"
vagimeli May 4, 2023
e550764
Revert "Updates to repo (#3882)"
vagimeli May 4, 2023
4c80de8
Revert "readmeupdates (#3881)"
vagimeli May 4, 2023
1962d2f
Revert "Add DLQ to OpenSearch sink configuration options. (#3743)"
vagimeli May 4, 2023
7f327e1
Revert "Add OTel trace group processor to doc website. (#3236)"
vagimeli May 4, 2023
e506882
Revert "Add OTel logs source doc to doc website. (#3608)"
vagimeli May 4, 2023
0542d71
Revert "Add OTel metrics raw processor doc to doc website. (#3235)"
vagimeli May 4, 2023
28218cf
Revert "[DOC] Add multidata source Dev Tools integration (#3653)"
vagimeli May 4, 2023
6e057d8
Revert "Updates the Data Prepper S3 source documentation (#3813)"
vagimeli May 4, 2023
47e8b4b
Revert "Add max_retries option (#3835)"
vagimeli May 4, 2023
fa285f7
Revert "Add examples to Key value page (#3105)"
vagimeli May 4, 2023
99cec6d
Revert "Add Event Analytics visualizations in Dashboards feature (#37…
vagimeli May 4, 2023
6ac3a6b
Revert "Add new settings for SAML and OIDC that allow for cookie spli…
vagimeli May 4, 2023
16141fb
Revert "Add editorial to maintainers and codeowners (#3840)"
vagimeli May 4, 2023
ad634b1
Revert "Update baseline maintainer and CODEOWNER format (#3832)"
vagimeli May 4, 2023
1775234
Revert "Content updates for segrep - benchmarking and recommended set…
vagimeli May 4, 2023
9cd5947
Revert "Add documentation for dynamic configuration of tenancy in Das…
vagimeli May 4, 2023
0481e2f
Revert "Update Maintainer.md to match CODEOWNER (#3831)"
vagimeli May 4, 2023
4af503b
Revert "Add documentation for IM UI index templates (#3459)"
vagimeli May 4, 2023
41ec6b7
Revert "Add list to map processor (#3806)"
vagimeli May 4, 2023
c2f5121
Revert "Add opensearch sink security (#3809)"
vagimeli May 4, 2023
c6d06a3
Revert "Add E2E acknowledgements (#3811)"
vagimeli May 4, 2023
4722568
Revert "Add Anomaly detector plugin doc. (#3569)"
vagimeli May 4, 2023
fa38b1c
Revert "Add documentation for wait_for_completion API (#3458)"
vagimeli May 4, 2023
c4001a6
Revert "Fix link to data stream explanation (#3431)"
vagimeli May 4, 2023
e15e513
Revert "Add documentation for Mutate strings (#2950)"
vagimeli May 4, 2023
a2af701
Revert "Add reference to hybrid mmap extensions setting in k-NN (#3262)"
vagimeli May 4, 2023
0f2b019
Writing
vagimeli May 4, 2023
cf3f8ae
Update approximate-knn.md
vagimeli May 4, 2023
f097a24
Merge branch 'anywhere-alert' of https://github.com/opensearch-projec…
vagimeli May 4, 2023
b6b3bd4
Revert "Update approximate-knn.md"
vagimeli May 4, 2023
df15fa5
Update index.md
vagimeli May 4, 2023
3cd0034
Update pretrained-models.md
vagimeli May 4, 2023
28eb335
Revert "Update pretrained-models.md"
vagimeli May 4, 2023
5efd935
Revert "Update index.md"
vagimeli May 4, 2023
05eef65
Revert "Revert "Add reference to hybrid mmap extensions setting in k-…
vagimeli May 4, 2023
1eb7432
Revert issues
vagimeli May 4, 2023
fa753ca
Revert "sqo-odbc brokenlinks (#3587)"
vagimeli May 4, 2023
9497a3c
Revert "Remove typo from ML cluster settings (#3529)"
vagimeli May 4, 2023
a39b2bc
Revert "Update example response with latest build (#3522)"
vagimeli May 4, 2023
748262f
Revert "Update put-mapping.md: add "dynamic" (#3384)"
vagimeli May 4, 2023
5adad22
Revert "Fix table formatting for http_source doc. (#3451)"
vagimeli May 4, 2023
74cb606
Revert "Fix table header in Bounded blocking doc. (#3450)"
vagimeli May 4, 2023
2485483
Revert "Add 1.3.9 version to version history (#3467)"
vagimeli May 4, 2023
0c05da9
Revert "add warning banner to querqy page (#3494)"
vagimeli May 4, 2023
3494729
Revert "Use npm registry instead of tgz file (#2754)"
vagimeli May 4, 2023
7973120
Revert "Update permissions.md (#3548)"
vagimeli May 4, 2023
e757159
Revert "Update prometheusmetrics.md (#3588)"
vagimeli May 4, 2023
e9d3676
Revert "Fix broken navigation for Notifications API (#3609)"
vagimeli May 4, 2023
40aa6ad
Revert "Add documentation for jwt_clock_skew_tolerance_seconds settin…
vagimeli May 4, 2023
86808b6
Revert "Change url for opensearch_security.openid.scope configuration…
vagimeli May 4, 2023
ce9d213
Revert "Adding documentation for behavior of indices.recovery.max_byt…
vagimeli May 4, 2023
2c05d5c
Revert "Add Security information to Security Analytics documentation …
vagimeli May 4, 2023
679b622
Revert "Add Trace peer forwarder doc. (#3238)"
vagimeli May 4, 2023
5916a53
Revert "Fix put rollup job api documentation (#3637)"
vagimeli May 4, 2023
251d78b
Revert "Add config parameters for traced models (#3456)"
vagimeli May 4, 2023
af5d6de
Revert "Add auto-truncation to pretrained models (#3756)"
vagimeli May 4, 2023
159741e
Revert "Use the modified dates of files in the sitemap (#3774)"
vagimeli May 4, 2023
4d43745
Revert "Add inline icons to formatting guide (#3785)"
vagimeli May 4, 2023
6d32e1f
Revert "Add filtering data in maps doc. (#3640)"
vagimeli May 4, 2023
fb64054
Revert "Update MAINTAINERS.md (#3641)"
vagimeli May 4, 2023
5461638
Revert "Fixing brand and trademark links (#3647)"
vagimeli May 4, 2023
c132adf
Revert "Update CODEOWNERS to reflect team (#3639)"
vagimeli May 4, 2023
74f7701
Revert "Add Parse JSON processor doc. (#3237)"
vagimeli May 4, 2023
96b1829
Revert "Add reference to hybrid mmap extensions setting in k-NN (#3262)"
vagimeli May 4, 2023
b9a4b51
Revert "Add documentation to rename opensearch_dashboards predefined …
vagimeli May 4, 2023
0167fed
Revert "Update to Breaking changes file (#3550)"
vagimeli May 4, 2023
84d5d91
Revert "Remove LICENSE from possible endpint list (#2907)"
vagimeli May 4, 2023
6ee59cd
Revert "Add Mutate event processor doc. (#3234)"
vagimeli May 4, 2023
057db61
Revert "Add path to the script for the securityadmin.sh troubleshooti…
vagimeli May 4, 2023
897e317
Revert "Remove dynamic templates from a mapping example (#3698)"
vagimeli May 4, 2023
a1ef7e1
Revert "Updates Trace Analytics overview (#3727)"
vagimeli May 4, 2023
d8df9a7
Revert "Add documentation for Searchable Snapshots GA release (#3462)"
vagimeli May 5, 2023
17c896b
Revert "Update java client version (#3753)"
vagimeli May 5, 2023
c8a365e
Revert "Add additonal Snapshot Restore (#3783)"
vagimeli May 5, 2023
3ca0834
Revert "Add maximum allowed monitor number. (#3732)"
vagimeli May 5, 2023
e7f622d
Revert "Update link-checker.yml (#3787)"
vagimeli May 5, 2023
97dcb95
Revert "Add documentation for segment replication GA release (#3461)"
vagimeli May 5, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
158 changes: 158 additions & 0 deletions _dashboards/discover/visualizing-ad.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,158 @@
---
layout: default
title: Creating and managing alerts and anomalies using data visualizations
parent: Exploring data
vagimeli marked this conversation as resolved.
Show resolved Hide resolved
nav_order: 50
---

# Creating and managing alerts and anomalies using data visualizations
Introduced 2.8
{: .label .label-purple }

OpenSearch Dashboards' alerting and anomaly detection features allow you to localize and address errors, fraud, or potential issues in your system before they become serious problems and to improve the accuracy of your data and analytics.

Alerts and anomalies are closely related, but they serve different purposes. Alerts inform you of issues that require immediate attention, while anomalies help you detect issues that may not yet have triggered an alert. By using both alerts and anomalies in your monitoring system, you can proactively manage and troubleshoot your system.

In this tutorial, you will learn to perform anomaly detection using the **Discover** application and line chart visualizations with OpenSearch sample data. At the end of this tutorial, you should have a good idea of how to use the Discover application and visualizations to monitor your own data. Common use cases for alerting and anomaly detection include network behavior, application performance, and web application security.
vagimeli marked this conversation as resolved.
Show resolved Hide resolved

The following video provides a quick overview of the steps performed in this tutorial:

<insert demo from SME>


## Getting started

The following is useful terminology to understand before getting started with this tutorial:

- _Anomaly detection_ is a technique used in data analysis to identify patterns or data points that deviate from the norm or expected behavior. It can be performed in real time, near real time, or on a scheduled basis.
- _Alert_ refers to an event associated with a trigger. When an alert is created, the trigger performs actions, which can include sending a notification..
vagimeli marked this conversation as resolved.
Show resolved Hide resolved
- _Monitor_ refers to jobs that run on a defined schedule and query OpenSearch indexes.
- _Trigger_ refers to conditions that generate alerts.
- _Action_ refers to the information that you want the monitor to send out after being triggered. Actions have a destination, a message subject, and a message body.
- _Destination_ refers to a reusable location for an action. Supported locations are Slack, email, or custom webhook.
vagimeli marked this conversation as resolved.
Show resolved Hide resolved

For an overview of the OpenSearch Alerting and Anomaly Detection plugins, see the OpenSearch documentation for [Alerting]({{site.url}}{{site.baseurl}}/observing-your-data/alerting/index/) and [Anomaly Detection]({{site.url}}{{site.baseurl}}/observing-your-data/ad/index/).
{: .note}

## Prerequisites
vagimeli marked this conversation as resolved.
Show resolved Hide resolved

This tutorial has the following prerequisites:

- You must be running OpenSearch Dashboards before proceeding with the tutorial.
- You must be connected to your local OpenSearch Dashboards environment or `https://localhost:5601`. The username and password are `admin`.
- You have basic familiarity with <prereq?>
vagimeli marked this conversation as resolved.
Show resolved Hide resolved
- <Other prereq?>

## Configuring admin settings

Access to alerting and anomaly detection using visualizations is controlled by OpenSearch and OpenSearch Dashboards privileges. **Stack Management** is the place where you manage advanced settings. The setting is enabled by default and appears as a feature in the **Stack Management > Advanced Settings > Visualizations** window as shown below. If the setting is disabled, it does not appear in this window. The following image shows the enabled settings in the Advanced Settings window.
vagimeli marked this conversation as resolved.
Show resolved Hide resolved

<insert UI>

If you are an administrator, you can disable the settings at the cluster level through the `dashboards.yml` configuration file, for example:

```bash
<insert code sample>
```

## Creating alerting monitors

By default, when you begin to create the alert monitor workflow with the Discover application, you are presented with a menu-driven interface. This interface provides you with a range of options displayed in full screen, pop up, pull down, or drop down. In the interface, you define the metrics to monitor, set thresholds, customize triggers that automate workflows, and generate actions when conditions are met.
vagimeli marked this conversation as resolved.
Show resolved Hide resolved

Alerts and anomalies are visualized in line charts on a dashboard. To create an alerting monitor using the Discover application and sample data and to visualize the alerts on a dashboard, follow these steps:

1. From the OpenSearch Dashboards main menu, select **Discover**.
vagimeli marked this conversation as resolved.
Show resolved Hide resolved
1. From **<window>**, select the **<name>** sample dataset. You will see a pre-populated dashboard with line chart visualizations.
1. Select the ellipsis icon from the **<name>** visualization pane, and then from the **Options** pop-up window, choose **Alerting** > **Add alerting monitor**.
1. Select **Create new monitor**.
1. From the **Add alerting monitor** window, define the metrics and set the thresholds under **Monitor details** and **Triggers**.
1. Under **Monitor details**, select <option> from the dropdown menu.
1. Under **Triggers**, select <option> from the dropdown menu, and then specify a name and severity level for the trigger.
1. _Optional_: In **Notifications**, select <option> from the dropdown menu, and then customize the notification message and select the Notification channel.
1. Choose **Create monitor**.

Once you have created a new monitor, the monitor is added to the visualization, as shown in the following image.

<insert UI>

## Adding an associated monitor
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.


You can add existing monitors to a visualization using the Discover application instead of Stack Management, giving you a single user interface where can add, view, and edit monitors and monitor details. Continuing with the visualization and dashboard in the preceding tutorial, follow these steps to associate an existing monitor to a visualization:
vagimeli marked this conversation as resolved.
Show resolved Hide resolved

1. From the dashboard, select the ellipsis icon on the **<name>**, then **Alerting**.
1. Select **Associate existing monitor**.
1. From the **Select monitor to associate** dropdown menu, view the list of existing monitors and then select the desired monitor. In this example, select **<name>**. Note that basic information about the monitor is summarized in the window. To view more comprehensive details, select **View monitor page**, which opens the monitor details from the Alerting plugin page.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We may want to add a note that only eligible monitors will be visible here

cc @lezzago

1. Verify you have selected the appropriate monitor, and then select **Associate monitor**.

An existing monitor is now associated to the visualization, as shown in the following image:

<insert UI>

## Exploring alerting monitor details

Once you have created or associated alerting monitors, you can verify the monitor is generating the alerts and explore the alert details by following these steps:

1. From the dashboard, view the <**name**> visualization. Alerts are indicated with a red triangle.
vagimeli marked this conversation as resolved.
Show resolved Hide resolved
1. Hover over a triangle to view the alert count details. To investigate the alert details, select a triangle to activate the pop-up window containing the monitor details. Alternatively, select the ellipsis icon in the visualization pane and choose **View events** to view those details.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

pop-up -> flyout may be better wording? And within the view events flyout, I probably wouldn't call them "monitor details" - rather, its a more fine-grained breakdown of all of the events for each plugin resource (monitor). To really see more details about a particular monitor, users can click on the monitor name within this flyout to get full details on the respective plugin details page.

1. Select the ellipsis icon, then **Alerting** > **Associated monitors**.
1. From the Associated monitors window, view the monitor list and then choose an alerting monitor, for example, <**CPU Usage Across World**>. Monitor details such as history, alerts, and associated visualizations, are shown from within the visualization pane.
1. _Optional_: From the Associated monitors window, unlink a monitor from the visualization by selecting the link icon under **Actions**. This unlinks the monitor from the visualization only; it does not delete the monitor itself.
1. _Optional_: From the Associated monitors window, edit the monitor's metrics, for example, the threshold for CPU usage.

Monitor details are viewable from the dashboard and visualization pane, as shown in the following image:

<insert UI>

## Creating anomaly detectors

Anomaly detectors identify unusual patterns or outliers in a dataset. Once you have defined your problem and prepared your data, you can load it into OpenSearch for indexing. In the following steps, you will continue using the sample dataset that you used in the preceding steps.

To create an anomaly detector using the Discover application and OpenSearch sample data and then visualize the anomalies on a dashboard, follow these steps:

1. From the OpenSearch Dashboards main menu, select **Discover**.
vagimeli marked this conversation as resolved.
Show resolved Hide resolved
1. From the <name> window, select the **<name>** sample dataset. You will see a pre-populated dashboard with line chart visualizations.
1. Select the ellipsis icon from the **<name>** visualization pane, and then from the **Options** pop-up window, choose **Anomaly Detection** > **Add anomaly detector**.
1. Select **Create new detector**.
1. From the Add anomaly detector window, define the metrics and set thresholds under **Detector details** and **Model features**. To view the visualization within this window, toggle the **Show visualization** button.
Copy link
Member

@ohltyler ohltyler Apr 21, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"configure/define the features" may be better wording than "metrics" here. I think thresholds can be removed.

The high-level things the user can set in the creation flow is

  • features (which are the AD-term for metric, essentially)
  • filters (to filter out unwanted data)

Other settings, like timestamp/interval/name/result index, etc. should have reasonable defaults set. And assuming this tutorial is requiring using the sample data, we can confirm that the rest of the detector settings can be pre-populated correctly.

1. _Optional_: Under **Detector details**, select <option> from the dropdown menu to configure the detector details.
1. _Optional_: Under **Model features**, select <option> from the dropdown menu to configure the evaluation metrics. You can add and delete model features as best suited your use case and data, but you are limited to five model features for that detector.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this can be simplified to just "configure the model features". Prefer to avoid metrics wording since that won't be present in the UI for this flow.

1. Select **Create detector**.

Once you have created a new detector, the detector is added to the visualization, as shown in the following image.

<insert UI>

### Creating an alerting monitor from the detector

You can set up an alerting monitor based on your detector in one flow from the visualization. To set up an alerting monitor based on the detector created in the preceding steps, follow these steps:

1. In the notification pop-up window that appears after you have created the detector, select **Set up alerts**.
2. Follow the steps described in [Creating alerting monitors](#creating-alerting-monitors).

You now have an alerting monitor and anomaly detector associated with the visualization.

<insert UI>

## Adding associated anomaly detectors
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"Adding existing anomaly detectors" more appropriate?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

or "Associating existing anomaly detectors"


You can add existing detectors to a visualization using the Discover application instead of Stack Management, giving you a single user interface where can add, view, and edit monitors and monitor details. Continuing with the visualization and dashboard in the preceding tutorial, follow these steps to associate an anomaly detector to a visualization:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.


1. From the dashboard, select the ellipsis icon on the **<name>**, then **Anomaly Detection**.
2. Select **Associate a detector**.
3. From the **Select detector to associate** dropdown menu, view the list of existing monitors and then select the desired monitor. In this example, select **<name>**. Note that basic information about the monitor is summarized in the window. To view more comprehensive details, select **View detector page**, which opens the detector details from the Anomaly Detection plugin page.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

monitor -> detector

4. Verify you have selected the appropriate detector, and then select **Associate detector**.

An existing detector is now associated to the visualization, as shown in the following image:

<insert UI>

## Refreshing the visualization

Depending on the threshold settings, the visualization refreshes at the set interval. To manually refresh the visualization to display real-time alerts and anomalies, select **Refresh** on the dashboard window.

## Considerations and limitations
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can also mention that by default, only 10 plugin resources per visualization is supported. This can be configured and changed in the advanced settings.

I mentioned it in the above comments a bit too, but we can provide some more details on

  • the visualization eligibility requirements to support the feature
  • the eligibility of alerting monitors that can be associated
  • the eligibility of anomaly detectors that can be associated.

For anomaly detectors, the limitation is that detectors with category fields (we call them high-cardinality detectors') are not eligible.

cc @lezzago to help provide details on the others.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For the Alerting plugin, we only allow creating simple Query Level monitors. Also we can reiterate the visualization eligibility requirements.


<SME input needed>

- Alerting and anomaly detection visualizations are only supported for line charts containing time series data.
-
Binary file added images/dashboards/alerting-dashboard.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.