Add default "newline detection" recombine rule for docker

[matthewmodestino]

As a OTel native logging user, I would like there to be a default recombine rule for docker logs, similar to what we ship for cri-o and containerd, where we use "newline" detection to break events.

Docker JSON-file driver will split long events when printing to disk and will insert a \n at the end of the complete log entries.

We should be able to use a generic is_last_entry rule in the recombine operator to detect new lines and combine the previous events seen.

[matthewmodestino]

I have managed to get a proof of concept of this working in GKE with their Ubuntu Docker nodes, using recombine rule added in extraOperators section. Ideally this would be inserted earlier in the pipeline to ensure multlineConfigs don't interfere and so we can use $$attributes["file.path"] default setting, which didn't work for me lower in the pipeline.

    extraOperators:
      - combine_field: log
        id: docker-recombine
        is_last_entry: ($$.log) endsWith "\n"
        type: recombine
        source_identifier: $$resource["com.splunk.source"]
        combine_with: ""

filelog receiver config ended up like this in my configMap:

  receivers:
      filelog:
        encoding: utf-8
        exclude: null
        fingerprint_size: 1kb
        force_flush_period: "0"
        include:
        - /var/log/pods/*/*/*.log
        include_file_name: false
        include_file_path: true
        max_concurrent_files: 1024
        max_log_size: 1MiB
        operators:
        - id: parser-docker
          timestamp:
            layout: '%Y-%m-%dT%H:%M:%S.%LZ'
            parse_from: time
          type: json_parser
        - id: filename
          resource:
            com.splunk.source: EXPR($$attributes["file.path"])
          type: metadata
        - id: extract_metadata_from_filepath
          parse_from: $$attributes["file.path"]
          regex: ^\/var\/log\/pods\/(?P<namespace>[^_]+)_(?P<pod_name>[^_]+)_(?P<uid>[^\/]+)\/(?P<container_name>[^\._]+)\/(?P<restart_count>\d+)\.log$
          type: regex_parser
        - attributes:
            log.iostream: EXPR($$.stream)
          resource:
            com.splunk.sourcetype: EXPR("kube:container:"+$$.container_name)
            k8s.container.name: EXPR($$.container_name)
            k8s.container.restart_count: EXPR($$.restart_count)
            k8s.namespace.name: EXPR($$.namespace)
            k8s.pod.name: EXPR($$.pod_name)
            k8s.pod.uid: EXPR($$.uid)
          type: metadata
        - combine_field: log
          combine_with: ""
          id: docker-recombine
          is_last_entry: ($$.log) endsWith "\n"
          source_identifier: $$resource["com.splunk.source"]
          type: recombine
        - id: clean-up-log-record
          ops:
          - move:
              from: log
              to: $$
          type: restructure