Allow filtering of properties and suppression of logging

[benkehoe]

The idea proposed here roughly corresponds to the concept of "logging levels", but aims to be separate from a strict logging levels paradigm. Based on a discussion with @jaredcnance.

I want the ability to add large numbers of properties to the MetricsLogger in my code, far more than would be economical to output for execution in production. Then, based on some criteria, like sampling, an error occurring, some sort of "debug" flag, etc., I want to filter down the properties that get flushed. Additionally, I may want to suppress the output entirely.

Proposed solution

1. Methods on MetricsLogger like set_property() and put_metric() will add an optional keyword argument called tag that takes a string or list of strings.
2. MetricsLogger will add two properties, pre_filter and filter, that can be set to callables.
3. pre_filter is called when a method is called with the tag keyword argument. It takes as input the context object, the tags, and the inputs to the method (e.g., property name and value), and returns a boolean indicating whether to actually perform the action or to silently ignore the input
   For convenience, pre_filter can be set to a list of tags, equivalent to a callable that matches the given tags against any in the list.
4. filter is called when the output is being collated. It takes as input the context object and any uncaught error that the metric scope (if there is one) has handled. It returns a boolean or a list of tags. If it returns a boolean, and the value is False, the logging is suppressed. If the boolean is True, all properties/metrics will be included in the output. If it returns a list of tags, the output is filtered to include untagged properties/metrics and any tagged properties/metrics that have at least one tag matching the list.
   For convenience, filter can be set to a boolean or a list of tags, equivalent to a callable that returns that value.
5. If no pre_filter is set, tagged properties/metrics will not be dropped.
6. If no filter is set, all tagged properties/metrics will be included in the output.

The reason to call it filter rather than post_filter is to encourage its use over pre_filter, which should generally only be used if memory/GC usage is a concern. The delayed action of filter allows decisions on properties to be made with context that may come in later (for example, increased logging verbosity in case of an error).

Use cases

1. "Debug" logging

To simply have a set of verbose properties, I might do the following:

metrics.filter = ["debug"] if os.environ.get("DEBUG", "false") in ["1", "true"] else []
# ...
metrics.set_property("some_debug_property", some_debug_value)

2. Verbose logging in case of an error

global is_error = False
metrics.filter = lambda context, uncaught_err: ["only_on_error"] if is_error else []

try:
    metrics.set_property("verbose_but_relevant_for_an_error", value, tags=["only_on_error"])
    # ...
except:
    is_error = True

3. Sampling

global sampled = bool(random.random < float(os.environ["SAMPLING_RATE"]))
metrics.filter = sampled

4. Combined uses

global is_error = False
global sampled = bool(random.random < float(os.environ["SAMPLING_RATE"]))
def filter(context, uncaught_err):
    tags = []
    if is_error:
        tags.append("on_error")
    if sampled:
        tags.append("on_sample")
    debug = os.environ.get("DEBUG", "false") in ["1", "true"]
    if verbose:
        tags.append("debug")
    return tags

metrics.filter = filter

metrics.set_property("foo", "foo value", tags=["debug", "on_error"])

metrics.set_property("bar", "bar value", tags=["on_sample", "on_error"])

Questions

• Should a property/metric have multiple tags or only a single one? I can see value in multiple tags but maybe it's too much complexity.