Redundant conditions?

[rmccu]

NebulaLogger/nebula-logger/core/main/log-management/classes/LogEntryEventHandler.cls

Line 418 in 3b04138

WHERE CreatedDate >= :fourHoursAgo AND CreatedDate = TODAY AND ApiReleaseNumber__c != NULL

Isn't checking if CreatedDate >= 4 hours ago redundant to checking it CreatedDate is today? The only gap I can see is if it is 1am, this will only pull the records created from midnight to 1am instead of the last 4 hours. Is there value in that?

Doesn't this mean there is a gap in the setStatusApiDetails() method where if that is not called at the end of the day, there are records that might miss getting their API release number & version getting set?

[jongpie]

Hi @rmccu - the gap that you're referring to is exactly the goal of this. My intent is for the code to make the callout at least once per 24 hours to ensure that the API details (retrieved via the callout) are accurate. At one point, the code only used WHERE CreatedDate >= :fourHoursAgo, which caused problems in some larger orgs with ongoing logging. In these orgs, there were scheduled jobs running hourly, so there was always 1+ Log__c created within the last 4 hours. As a result, LogEntryEventHandler would continue to use a queried Log__c record - the callout wasn't executed for a while (a few weeks, if I remember correctly), resulting in stale data. By changing the query to WHERE CreatedDate >= :fourHoursAgo AND CreatedDate = TODAY, it provides a way to have the code execute the callout once per day.

[rmccu]

I think putting this in a comment before the query would be extremely helpful. I don't know if it is in the newer version of the code, as has been identified, I was looking at a version of the code that was several months old.

[rmccu]

Would it make more sense to store this in platform cache? It seems like it would avoid the DB hit? I am still not sure how efficient this query will be. In our case, we are creating ~2M logs/day. I need to run this through an explain plan and see...

[rmccu]

I ran the SOQL and it looks like it is using the CreatedDate index. Even though this is using an index, is the query expensive enough to consider platform cache being that this needs to be retrieved for every Log Entry insert?

[jongpie]

@rmccu not all orgs have platform cache available, so I'm avoiding using it in Nebula Logger. Since this code is running asynchronously & it's already using an index for CreatedDate, I'm not sure I'd consider it expensive enough (yet) to try to change it. I'm also not sure what other options could be used to make it more performant. One possible option is that I could reduce the current 4 hour window to instead by 1 or 2 hours - it would result in ewer records returned by the query index, but more callouts per day, so there's a tradeoff with this idea.

As far as your specific use case, 2M logs/day is quite a lot of data - are you finding that much data to actually be useful? Personally, I'd find that volume to be too much noise, but I know a couple of other orgs have similar volumes of data, so I'd love to hear more about how you/your team are leveraging that much logging data.

[rmccu]

@jongpie Fair point on the lack of platform cache. Maybe an optional flag? While I get that this is running async, so there is less earnestness in ensuring it is performant, it still consumes system resources, which affect overall system responsiveness. While this query is using an index, I am still curious how big of a DB impact it has.

We have 60k users (with more being brought on board every few weeks). Some of this logging was added due to a web-mashup integration we have with another system where we are using the JavaScript postMessage() method and listeners so that the 2 windows can speak to one another. This integration has been problematic and in order to prove the source of the issue was not in Salesforce, we had to add a lot of logging of each step of the integration so that we could show that the 3rd party system was too unstable and not receiving all of our messages.

There are additional entries coming from our integrations to the OpenCTI adapter, and we have some integrations that run when we "pop" our C360 experience and try to determine if the customer information has been populated from the back-end systems.

[jongpie]

Yeah, I can look into possibly adding an optional flag to control if platform cache is used - there are several orgs like yours that would benefit from using platform cache, so I'll do some more research to see how complex it would be to try to (optionally) leverage it. I'm actively working on some other optimizations for #311, so I'll include platform cache in the scope of that work.

And thanks for sharing some details about your use case! It's helpful to hear about other people's use cases for Nebula Logger, especially large data volumes like yours.