-
Notifications
You must be signed in to change notification settings - Fork 8.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[ML] AIOps Log Rate Analysis: Support for text fields. #167467
Labels
enhancement
New value added to drive a business result
Feature:ML/AIOps
ML AIOps features: Change Point Detection, Log Pattern Analysis, Log Rate Analysis
:ml
v8.11.0
Comments
walterra
added
enhancement
New value added to drive a business result
:ml
Feature:ML/AIOps
ML AIOps features: Change Point Detection, Log Pattern Analysis, Log Rate Analysis
v8.11.0
labels
Sep 28, 2023
Pinging @elastic/ml-ui (:ml) |
walterra
changed the title
[ML] AIOPs Log Rate Analysis: Support for text fields.
[ML] AIOps Log Rate Analysis: Support for text fields.
Sep 28, 2023
This was referenced Sep 28, 2023
Merged
walterra
added a commit
that referenced
this issue
Sep 28, 2023
…ailable via `server` (#167465) Follow up to #167237. Part of #167467. We plan to reuse some of the queries log pattern analysis does to use via log rate analysis too. Log pattern analysis mostly does queries from the client side, late rate analysis has its own API endpoint and does ES queries via Kibana server. In preparation for the use via log rate analysis, this moves the code we need to have available server side for log rate analysis to the `common` area of the plugin so it can be used both on server/client.
walterra
added a commit
that referenced
this issue
Oct 4, 2023
Part of #167467. Adds support for text fields in log pattern analysis. Text fields will be analysed using log categorization, similar to log pattern analysis. Significant log patterns will be identified using the `chi2test` package, similar to how we detect data drifts.
Moved follow up tasks to the meta issue here #168459. Closing as this covers the work done for |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Labels
enhancement
New value added to drive a business result
Feature:ML/AIOps
ML AIOps features: Change Point Detection, Log Pattern Analysis, Log Rate Analysis
:ml
v8.11.0
With the recent work done for the data drift view, we now have utilities in place to calculate p-values for any kind of data to be compared. We can make use of that to add support for identifying significant log categories. Within log rate analysis, we could categorize logs for the baseline and deviation time range and then use
chi2test
to identify significant changes. The results could then be part of the analysis results table. The trickier part will be to make this part of the grouping as the log categories cannot be used within thefrequent_item_set aggregation
. So this would have to be something on top, like augmenting the keyword based groups with matching log categories.Tasks
The text was updated successfully, but these errors were encountered: