You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I've been having issues with the reporting functionality in Thruk. See shinken-solutions/shinken#1566 for my original bug report. After digging I found it's actually a small problem in this module that's causing the problem. Naparuba's statement that getting logs via the livestatus module will blow up if you have a very large number of logs is very true, however currently this module gets all logs instead of the relevant logs when using the reporting functionality in Thruk, which makes that more or less useless.
Details:
This is the relevant part of the livestatus command that's being issued by Thruk for an "SLA Hostgroup" report:
The 2 'message' filter attributes are gone and replaced by these dummy 'time' entries. These are always true and cause the filter to get all logs. This causes my "report from hell" to request 7 GB of logs and then churns the broker to death in about 5 minutes by eating all memory and eating 100% cpu. And that's just our test environment: our production environment has 10 times as much log entries...
So I added the 'message' attribute to the 'good_attributes' and 'string_attributes' lists in the make_mongo_filter function, et voila, now this is the result:
That's more like it! After this fix the report completes in about a minute and does not consume huge amounts of memory or cpu.
Here's the pull request: #13
I hope you'll merge it.
PS: there's something wrong with the Travis integration, since that fails on PRs for this module on stuff that wasn't even touched. See also my other PR (#11) that updates this module to be MongoDB 3 compatible :)
The text was updated successfully, but these errors were encountered:
Hi,
I've been having issues with the reporting functionality in Thruk. See shinken-solutions/shinken#1566 for my original bug report. After digging I found it's actually a small problem in this module that's causing the problem. Naparuba's statement that getting logs via the livestatus module will blow up if you have a very large number of logs is very true, however currently this module gets all logs instead of the relevant logs when using the reporting functionality in Thruk, which makes that more or less useless.
Details:
This is the relevant part of the livestatus command that's being issued by Thruk for an "SLA Hostgroup" report:
The make_mongo_filter code translates that into (nice piece of coding btw, changing the postfix livestatus protocol into the prefix mongo filter!):
The 2 'message' filter attributes are gone and replaced by these dummy 'time' entries. These are always true and cause the filter to get all logs. This causes my "report from hell" to request 7 GB of logs and then churns the broker to death in about 5 minutes by eating all memory and eating 100% cpu. And that's just our test environment: our production environment has 10 times as much log entries...
So I added the 'message' attribute to the 'good_attributes' and 'string_attributes' lists in the make_mongo_filter function, et voila, now this is the result:
That's more like it! After this fix the report completes in about a minute and does not consume huge amounts of memory or cpu.
Here's the pull request: #13
I hope you'll merge it.
PS: there's something wrong with the Travis integration, since that fails on PRs for this module on stuff that wasn't even touched. See also my other PR (#11) that updates this module to be MongoDB 3 compatible :)
The text was updated successfully, but these errors were encountered: