Proposal for a logging module

[DinoBektesevic]

Issue

Logging in Python is a singleton that expects a logger to be declared on a module level. Each Logger has a unique name, and is retrieved via getLogger with the corresponding name. The recommended way to uniquely achieve this Python is to register a logger on a module level, using module's name:

logger = logging.getlogger(__name__)

Occasionally we have to produce a log on the C++ side. Primary use-case are logs produced by StackSearch - a C++ class used to manage processing, but generally not by the user. Users are expected to use the SearchRunner class instead. SearchRunner produces logs of the processing progress, Python-side, using print and the StackSearch on the C++ side using std::cout. This is done so that the print-outs appear as if they are coming from the same place. If we replaced the prints Python side with logging functionality, the outputs from the two objects would diverge and the C++ side output would not follow the Python logging output configuration anymore.

Potential solutions

To restore this behavior we would need to call the Python-side logger from C++. For that, we need to (somehow) pass a reference to the logger into the "StackSearch" class. Several options exist:

1. pass the reference at instantiation time
2. wrap the Python-side logger in a singleton from the C++ side and keep a lookup table of logger references resolved with the same name-matching the Python side

The second approach is implemented in this proposal. I don't see many (any) benefits of the first approach, we would have to wrap all uses of the logger in a class anyhow (which is basically what 2 is minus the Py-wrapper logger) or use pre-processor directives to call either some output or the Python logger.

How it works

The Logging class is a singleton that keeps a reference to all created Loggers. The Loggers define the log format and IO method (stdout, file etc.). Logging keeps references to Loggers in a registry. This registry is exposed via the getLogger method, which doubles as a factory function for Loggers. This is modeled after Python's logging module. When getLogger is called from Python (via the pybind11 bindings) it creates a new Python-side Logger object and registers its reference. When called C++ side it creates a C++-side Logger and registers its reference. Accessing a getLogger using a name that was already registered - it returns the reference from the registry.

This allows us to access Python-side loggers directly from C++ by invoking

Logging.getLogger("module_name")

from C++. On the example of SearchRunner, placing:

Logging.getLogger("kbmod.run_search")

somewhere in the search_runner.cpp would return:
a) the Python Logger object for the module, assuming search module is being called from Python, because the loggers are defined as module-global variables; i.e. guaranteed to occur before any C++ code executes.
b) or the default std::cout logger named "kbmod.run_search" that is basically acting as a more glorified print formatted that can prune and not-emit certain "levels" of messages.

The obvious pitfall is the case when a user does not route through this Logging, and instead registers a Python-side Logger via its logging module. Because these Python-side Loggers are not registered in the Logging's registry the KBMOD Logging will default to using the C++ std::out logger. This can lead to differing output formats if the Python Logger in question is then re-configured (as the default behavior of the internal KBMOD Logger was made to match that of default Python Logger).