Skip to content

Commit

Permalink
Fix ml job because of missing Datafeed in the logs (elastic#9718)
Browse files Browse the repository at this point in the history
The log output from ML changed. Because of this the check on the Beats side has to be adjusted. The old check was kept in place for backward compatiblity.

```
2018-12-20T14:11:14.356ZERRORinstance/beat.go:906Exiting: 2 errors: Error setting up ML for apache2: 5 errors: [status_exception] A datafeed [datafeed-filebeat-apache2-access-visitor_rate] already exists for job [filebeat-apache2-access-visitor_rate]; [status_exception] A datafeed [datafeed-filebeat-apache2-access-response_code] already exists for job [filebeat-apache2-access-response_code]; [status_exception] A datafeed [datafeed-filebeat-apache2-access-remote_ip_url_count] already exists for job [filebeat-apache2-access-remote_ip_url_count]; [status_exception] A datafeed [datafeed-filebeat-apache2-access-low_request_rate] already exists for job [filebeat-apache2-access-low_request_rate]; [status_exception] A datafeed [datafeed-filebeat-apache2-access-remote_ip_request_rate] already exists for job [filebeat-apache2-access-remote_ip_request_rate]; Error setting up ML for nginx: 5 errors: [status_exception] A datafeed [datafeed-filebeat-nginx-access-visitor_rate] already exists for job [filebeat-nginx-access-visitor_rate]; [status_exception] A datafeed [datafeed-filebeat-nginx-access-remote_ip_url_count] already exists for job [filebeat-nginx-access-remote_ip_url_count]; [status_exception] A datafeed [datafeed-filebeat-nginx-access-response_code] already exists for job [filebeat-nginx-access-response_code]; [status_exception] A datafeed [datafeed-filebeat-nginx-access-remote_ip_request_rate] already exists for job [filebeat-nginx-access-remote_ip_request_rate]; [status_exception] A datafeed [datafeed-filebeat-nginx-access-low_request_rate] already exists for job [filebeat-nginx-access-low_request_rate]
```
  • Loading branch information
ruflin authored Dec 20, 2018
1 parent 3866b5a commit 837929e
Show file tree
Hide file tree
Showing 2 changed files with 4 additions and 1 deletion.
3 changes: 3 additions & 0 deletions filebeat/tests/system/test_ml.py
Original file line number Diff line number Diff line change
Expand Up @@ -93,6 +93,9 @@ def _run_ml_test(self, setup_flag, modules_flag):
"-c", cfgfile
]

# Skipping dashboard loading to speed up tests, unfortunately only works for setup and not --setup
cmd += ["-E", "setup.dashboards.enabled=false"]

if setup_flag:
cmd += ["--setup"]
else:
Expand Down
2 changes: 1 addition & 1 deletion libbeat/ml-importer/importer.go
Original file line number Diff line number Diff line change
Expand Up @@ -229,7 +229,7 @@ func checkResponse(r []byte) error {

for _, feed := range resp.Datafeeds {
if !feed.Success {
if strings.HasPrefix(feed.Error.Msg, "[resource_already_exists_exception]") {
if strings.HasPrefix(feed.Error.Msg, "[status_exception] A datafeed") || strings.HasPrefix(feed.Error.Msg, "[resource_already_exists_exception]") {
logp.Debug("machine-learning", "Datafeed already exists: %s, error: %s", feed.ID, feed.Error.Msg)
continue
}
Expand Down

0 comments on commit 837929e

Please sign in to comment.