Default Logger Settings values change the results of unit tests in sandboxes vs. production

[arbokrad]

The behavior of Logger.getUserSettings() changes between sandbox and production environments, and can cause tests to pass in one and fail in the other unexpectedly if the set log levels are different.

Example:

@IsTest
static void noLogLevelSetByTestOrOrg_PassesSandboxButNotProd() {

	Logger.info('Hello World');
	
	// query for it - works in sandbox but not prod
	List<LogEntry__c> entries = ...;
	Assert.areEqual(1, entries.size());
}

In the above example, no log level is set by the test or using org defaults, so Logger.getUserSettings() will default to picking up FINEST as the log level in sandboxes and scratch orgs - all appears fine because the defaulting code has a specific exception made for sandboxes.

However if Production has an org default set higher than this, the test will fail.

@IsTest
static void logLevelSetByTest_PassesBoth() {
	Logger.getUserSettings().LoggingLevel__c = System.LoggingLevel.INFO.name();

	Logger.info('Hello World');
	
	// query for it - works in all envs
	List<LogEntry__c> entries = ...;
	Assert.areEqual(1, entries.size());
}

...the fix is straightforward. The logging level specified inside the test keeps the behavior consistent between environments. I think the problem here is that it changes behavior in a way we didn't expect. I'd almost rather have it fail the same way throughout than fall back to something for sandboxes only? Worth some debate at least.