Use ML for clustering errors

[gbittencourt]

Try to use text mining algorithms to retrieve useful information from log. With the retrieved information, automatically clusterise errors and automatically create meaningful issues.

The idea here is to reduce/streamline the job of creating the issues by users.

For the text mining itself it might be necessary to try different algorithms and/or to clean the data (analysing only part of the log). Trying different approaches may be necessary, it all will depend on how easy or difficult it will be to retrieve reliable results.

[aliceinwire]

Is possible to use the command kci-dev results for getting KernelCI jobs json results and logs
https://kernelci.github.io/kci-dev/results/

if you have any problem or suggestion feel free to send a PR or open a issue

[pawiecz]

This sounds similar to what Rado Vrbovsky worked on and presented at DevConf.cz 2021:
https://devconfcz2021.sched.com/event/gmMq + https://www.youtube.com/watch?v=vF61Outfbow

You might also find interesting work by Ondrej Klinovský on detecting anomalies in Linux kernel logs:
https://is.muni.cz/th/ug8ef/detect-anomalies-in-linux-kernel-logs.pdf

I wasn't able to locate source code for these efforts (I'm not even sure if that has been published).

[pawiecz]

Also this tool might be helpful (e.g. for results verification): https://gitlab.collabora.com/rcn/logspec

logspec is a context-sensitive parser for output logs, specifically targeted at test output logs but usable on any kind of log.