[feat] Support different log formats, zwavejs & zwavejs2mqtt

[LordMike]

Is your feature request related to a problem? Please describe.
When troubleshooting, I find it useful to view logs across multiple platforms: mqtt, zwavejs2mqtt and zwavejs. I've made some initial pushes at ingesting all these logs into an ELK stack, so I have one coherent timeline.

My issue is then that to parse the zwavejs logs, I need to do a lot of string parsing and manipulation, but I was told that it is actually possible to replace the log driver with something else, including a JSON one.

Describe the solution you'd like
The ability to log both zwavejs and zwavejs2mqtt logs in a Json format, so that I easily can ingest them within my ELK stack. An alternate option would be to not write to disk, but send it directly to some server. I wouldn't recommend supporting ElasticSearch directly, but sending JSON to a TCP socket could be done with Logbeat/Logstash, which then forwards to ES.

Additionally, I imagine that reporting bugs here would be way more tricky with different formats - especially for zwavejs where a lot of time has gone into making a pretty printed log output that can be read by AlCalzone. To keep supporting that, maybe write logs in multiple places, as an option?

So what I might really be asking for, is an option to send logs elsewhere, in addition to the files being made right now.

What I hope to have in the end, is an ELK stack where I can put in "node 77" and then view all logs, across all software, for node 77. Each log entry will be noted with severities, such that I can easily find warnings or errors (timeouts, drops, ..), while troubleshooting.. :)

Describe alternatives you've considered
N/A

Additional context
Here is my filebeat config, which shows some of the issues I've had with zwavejs logs parsing. It also doesn't ingest zwavejs2mqtt yet, as I didn't have time to make that. I went with MQTT logging, which has its own drawback, as with this simple topic filter, I also see driver updates and client statuses etc.. and not just whatever I'm interested in, such as writes and reads.

MQTT of course does not know anything about node Id's, so the logs from there will always be lacking that attribute. Logs from zwavejs2mqtt would be better, as I would both get the message that was received from MQTT, and potentially the node Id the message was for, as Z2M can translate that.

# cat /etc/filebeat/filebeat.yml
filebeat.inputs:

# Zwave logs
- type: filestream
  fields:
    source: zwavejs
  paths:
    - /mnt/systems/home-assistant/zwavejs2mqtt/*.log
  parsers:
    - multiline.type: pattern
      multiline.pattern: '^2'
      multiline.negate: true
      multiline.match: after
  processors:
    - if:
        contains:
          message: ' [Node '
      then:
        - dissect:
            tokenizer: '%{timestamp} %{component} %{direction} [Node %{nodeid|integer}] %{message}'
            overwrite_keys: true
            target_prefix: ''
            trim_values: left
      else:
        - dissect:
            tokenizer: '%{timestamp} %{component} %{direction} %{message}'
            overwrite_keys: true
            target_prefix: ''
            trim_values: left
    - timestamp:
        field: timestamp
        layouts:
          - '2006-01-02T15:04:05.999Z'
        test:
          - '2021-12-27T18:47:54.103Z'
    - drop_fields:
        fields: [timestamp]

# Zwave commands
- type: mqtt
  fields:
    source: zwave-mqtt
  hosts: ['tcp://mqtt:1883']
  topics:
    - zwavejs2mqtt/+/#
  processors:
    - copy_fields:
        fields:
          - from: mqtt.topic
            to: component

setup.template.settings:
  index.number_of_shards: 1
  #index.codec: best_compression
  #_source.enabled: false

#setup.dashboards.enabled: true

setup.kibana:
  host: "elasticsearch:5601"

output.elasticsearch:
  hosts: ["kibana:9200"]

#output.console:
#  pretty: true

[robertsLando]

@LordMike this could be done using a custom plugin on my side: https://zwave-js.github.io/zwavejs2mqtt/#/guide/plugins?id=developing-custom-plugins

[LordMike]

Iirc @AlCalzone said something about json logs had to be compiled in to work, and that by default it wasn't - I assume this has to be in place before I can invoke it?

[AlCalzone]

Further reading:
zwave-js/node-zwave-js#3019
zwave-js/node-zwave-js#2142

and the available log transports are here:
https://github.com/zwave-js/log-transports/tree/master/packages

If anyone feels compelled to turn this into a coherent documentation for https://github.com/zwave-js/node-zwave-js, please go ahead.