Separate data-logging from process-logging

[blueedgenick]

There seems to be an increasing number of instances where we are discussing how to handle and report (or not) various exceptions caused by a legal query against possibly-illegal data. For example:

• requesting an array element which is out-of-bounds for a particular record
• requesting a map element which doesn't exist
• deserialization exceptions

For all such cases I believe the behavior in terms of, e.g. returning NULL for an impossible-to-reach element is correct.

The question arises about how/when we should notify teh user or operator that such a situation has occurred. Conceptually I class all of these as "data errors" and propose that they be directed to a dedicated logger for this purpose, perhaps creating a ksql-data-errors.log or similar. This helps to avoid cluttering the actual server log - which is monitored by an operator for what i would call "process problems" (jvm issues, real exceptions, etc.) - with problesm which may be handled by a different team and with less urgency as they are only "fixable" by changing data or altering queries which are, from a semantic perspective, already "correct".