-
|
Why the alerts are not displaying the summary table content and are instead sending each alert separately? Using Docker image jertel/elastalert2:2.19.0. ElasticSearch indexPOST applog/_doc
{
"appname": "cms",
"level": "error",
"message": "i am cms 1",
"extra": "abc",
"@timestamp": "2024-09-07T12:37:12.000+08:00"
}
POST applog/_doc
{
"appname": "cms",
"level": "error",
"message": "i am cms 3",
"extra": "abc",
"@timestamp": "2024-09-07T12:37:14.000+08:00"
}Rules configurationname: alert by level field
type: blacklist
index: applog
compare_key: level
blacklist:
- error
- err
- fatal
aggregation:
minutes: 1
aggregation_key:
- appname.keyword
summary_table_fields:
- appname.keyword
- count
- message
alert:
- debugOutput debug logsKey log fragments ...
elastalert-elastalert2-1 | INFO:elastalert:New aggregation for alert by level field, aggregation_key: cms. next alert at 2024-09-07 04:39:04.332983+00:00.
elastalert-elastalert2-1 | INFO:elastalert:Adding alert for alert by level field to aggregation(id: 7GfGypEB5rLNh48YYVpW, aggregation_key: cms), next alert at 2024-09-07 04:39:04.332983+00:00
...
elastalert-elastalert2-1 | INFO:elastalert:alert by level field
elastalert-elastalert2-1 |
elastalert-elastalert2-1 | @timestamp: 2024-09-07T12:37:12+08:00
elastalert-elastalert2-1 | _id: 6mfGypEB5rLNh48YUlre
elastalert-elastalert2-1 | _index: applog
elastalert-elastalert2-1 | appname: cms
elastalert-elastalert2-1 | appname.keyword: cms
elastalert-elastalert2-1 | extra: abc
elastalert-elastalert2-1 | level: error
elastalert-elastalert2-1 | message: i am cms 1
elastalert-elastalert2-1 | num_hits: 2
elastalert-elastalert2-1 | num_matches: 2
elastalert-elastalert2-1 |
elastalert-elastalert2-1 | INFO:elastalert:Alert for alert by level field at 2024-09-07T12:37:14+08:00:
elastalert-elastalert2-1 | INFO:elastalert:alert by level field
elastalert-elastalert2-1 |
elastalert-elastalert2-1 | @timestamp: 2024-09-07T12:37:14+08:00
elastalert-elastalert2-1 | _id: 62fGypEB5rLNh48YUlr-
elastalert-elastalert2-1 | _index: applog
elastalert-elastalert2-1 | appname: cms
elastalert-elastalert2-1 | appname.keyword: cms
elastalert-elastalert2-1 | extra: abc
elastalert-elastalert2-1 | level: error
elastalert-elastalert2-1 | message: i am cms 3
elastalert-elastalert2-1 | num_hits: 2
elastalert-elastalert2-1 | num_matches: 2
elastalert-elastalert2-1 | The full log |
Beta Was this translation helpful? Give feedback.
Answered by
jertel
Sep 23, 2024
Replies: 3 comments 3 replies
-
|
Per the docs, aggregation_key isn't supported by the Blacklist or Whitelist rule types: |
Beta Was this translation helpful? Give feedback.
3 replies
-
|
I try to use the any rule type，but the debug output is the same as above. |
Beta Was this translation helpful? Give feedback.
-
|
I recommend enabling debug logging to watch the queries and the results coming back to ElastAlert 2. Also, there are two info (non-debug) log lines that mention when an event is aggregated:
Assuming there are no errors, if you aren't seeing those log lines then it means the rule does not appear to have aggregation setup correctly. That would happen if the wrong rule file is being loaded or the rule file failed to properly load (there would be errors at startup). |
Beta Was this translation helpful? Give feedback.
-
|
Add some data to ES This is my rule config any.yaml name: test
type: any
index: test
aggregation:
minutes: 1
aggregation_key: 'my_data.username'
summary_table_fields:
- my_data.username
- my_data.event_type
alert:
- debugMain config elastalert.yaml rules_folder: rules
run_every:
seconds: 10
buffer_time:
minutes: 15
es_host: 172.31.96.151
es_port: 19201
use_ssl: True
verify_certs: False
es_username: elastic
es_password: mypassword
ssl_show_warn: False
writeback_index: elastalert_status
# is the retry window for failed alerts.
alert_time_limit:
minutes: 15 |
Beta Was this translation helpful? Give feedback.
-
|
It's not clear from the logs what's going on with your test. In one log file it's showing that all three events are being treated as a separate aggregation, even though two of them use "alice" as the username. It's unknown if there's a username vs username.keyword issue, or something is off in the raw event data itself (trailing spaces, etc). Also, the debug alerter doesn't support rendering summary tables. |
Beta Was this translation helpful? Give feedback.
0 replies
Answer selected by
aisuhua
-
|
@jertel The problem has been solved, thank you very much! Below is my rule code name: test
type: any
index: test
aggregation:
minutes: 1
aggregation_key: 'my_data.username.keyword'
summary_table_fields:
- my_data.username.keyword
- my_data.event_type.keyword
alert:
- email
email: [email protected]
smtp_host: smtp.qq.com
smtp_port: 465
smtp_ssl: true
smtp_auth_file: /opt/elastalert/rules/email_auth
from_addr: [email protected]The email i had received:
|
Beta Was this translation helpful? Give feedback.
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
It's not clear from the logs what's going on with your test. In one log file it's showing that all three events are being treated as a separate aggregation, even though two of them use "alice" as the username. It's unknown if there's a username vs username.keyword issue, or something is off in the raw event data itself (trailing spaces, etc).
Also, the debug alerter doesn't support rendering summary tables.