Store log analysis output, or stop creating it

[anjackson]

The log analysis task outputs dead seeds and statistics as well as documents. The stats look like this:

BY_DAY_HOST_SOURCE,2021-11-05T00:00:00Z,0.gravatar.com,tid:40476:http://guthan.wordpress.com {"lines": 18, "status_code:200": 18, "content_type:image/png": 17, "hop:E": 18, "sum:content_length": 18357, "ip:192.0.73.2": 18, "content_type:image/jpeg": 1}

It would be great to build on this to store these high-level stats somewhere (e.g. most fields as columns with things like content types as GIN JSONB in PostgreSQL), OR stop doing this and create new reporting processes that run from the logs.