Nginx ingress_controller pipeline can not parse message if a second upstream fails to return response

[leweafan]

Describe the enhancement:

Nginx ingress_controller pipeline can not parse message if a second upstream fails to return response.
According to test.log this case is missing.
Grok patten should be enhanced.
Current pattern definitions:

UPSTREAM_RESPONSE_TIME_LIST: (?:%{NUMBER})("?,?\s*(?:%{NUMBER}))*
UPSTREAM_RESPONSE_STATUS_CODE_LIST: (?:%{NUMBER})("?,?\s*(?:%{NUMBER}))*

Not match for

0.100, - 
200, -

Describe a specific use case for the enhancement or feature:

192.168.64.14 - - [07/Feb/2020:12:02:42 +0000] "GET /test/test.mp4 HTTP/1.1" 200 61 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.14; rv:72.0) Gecko/20100101 Firefox/72.0" 348 0.001 [default-web2-8080] [] 172.17.0.6:8080, 172.17.0.7:8080 61, - 0.100, - 200, - 835136ae24486dbb4156dcbe21f5d402

Reproduce steps

POST /_ingest/pipeline/filebeat-7.17.1-nginx-ingress_controller-pipeline/_simulate
{
  "docs": [
    {
      "_source": {
        "@timestamp": "2022-08-27T09:24:26.000Z",
        "message": "192.168.64.14 - - [07/Feb/2020:12:02:42 +0000] \"GET /test/test.mp4 HTTP/1.1\" 200 61 \"-\" \"Mozilla/5.0 (Macintosh; Intel Mac OS X 10.14; rv:72.0) Gecko/20100101 Firefox/72.0\" 348 0.001 [default-web2-8080] [] 172.17.0.6:8080, 172.17.0.7:8080 - 0.100, - 200, - 835136ae24486dbb4156dcbe21f5d402"
      }
    }
  ]
}

{
  "docs": [
    {
      "doc": {
        "_index": "_index",
        "_id": "_id",
        "_version": "-3",
        "_source": {
          "event": {
            "ingested": "2023-02-26T15:14:59.047075786Z",
            "original": """192.168.64.14 - - [07/Feb/2020:12:02:42 +0000] "GET /test/test.mp4 HTTP/1.1" 200 61 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.14; rv:72.0) Gecko/20100101 Firefox/72.0" 348 0.001 [default-web2-8080] [] 172.17.0.6:8080, 172.17.0.7:8080 - 0.100, - 200, - 835136ae24486dbb4156dcbe21f5d402"""
          },
          "error": {
            "message": """Provided Grok expressions do not match field value: [192.168.64.14 - - [07/Feb/2020:12:02:42 +0000] \"GET /test/test.mp4 HTTP/1.1\" 200 61 \"-\" \"Mozilla/5.0 (Macintosh; Intel Mac OS X 10.14; rv:72.0) Gecko/20100101 Firefox/72.0\" 348 0.001 [default-web2-8080] [] 172.17.0.6:8080, 172.17.0.7:8080 - 0.100, - 200, - 835136ae24486dbb4156dcbe21f5d402]"""
          },
          "@timestamp": "2022-08-27T09:24:26.000Z"
        },
        "_ingest": {
          "timestamp": "2023-02-26T15:14:59.047075786Z"
        }
      }
    }
  ]
}

In case second upstream available:

POST /_ingest/pipeline/filebeat-8.6.1-nginx-ingress_controller-pipeline/_simulate
{
  "docs": [
    {
      "_source": {
        "@timestamp": "2022-08-27T09:24:26.000Z",
        "message": "192.168.64.14 - - [07/Feb/2020:12:02:42 +0000] \"GET /test/test.mp4 HTTP/1.1\" 200 61 \"-\" \"Mozilla/5.0 (Macintosh; Intel Mac OS X 10.14; rv:72.0) Gecko/20100101 Firefox/72.0\" 348 0.001 [default-web2-8080] [] 172.17.0.6:8080, 172.17.0.7:8080 - 0.100, 0.099 200, 401 835136ae24486dbb4156dcbe21f5d402"
      }
    }
  ]
}

{
  "docs": [
    {
      "doc": {
        "_index": "_index",
        "_id": "_id",
        "_version": "-3",
        "_source": {
          "@timestamp": "2020-02-07T12:02:42.000Z",
          "nginx": {
            "ingress_controller": {
              "upstream": {
                "alternative_name": "",
                "port": 8080,
                "response": {
                  "status_code": 401,
                  "time_list": [
                    "0.100",
                    "0.099"
                  ],
                  "status_code_list": [
                    "200",
                    "401"
                  ],
                  "time": 0.199
                },
                "ip": "172.17.0.7",
                "name": "default-web2-8080"
              },
              "upstream_address_list": [
                "172.17.0.6:8080",
                "172.17.0.7:8080"
              ],
              "http": {
                "request": {
                  "length": 348,
                  "id": "835136ae24486dbb4156dcbe21f5d402",
                  "time": 0.001
                }
              },
              "remote_ip_list": [
                "192.168.64.14"
              ]
            }
          },
          "related": {
            "ip": [
              "192.168.64.14"
            ]
          },
          "http": {
            "request": {
              "method": "GET",
              "id": "835136ae24486dbb4156dcbe21f5d402"
            },
            "response": {
              "status_code": 200,
              "body": {
                "bytes": 61
              }
            },
            "version": "1.1"
          },
          "source": {
            "address": "192.168.64.14",
            "ip": "192.168.64.14"
          },
          "event": {
            "ingested": "2023-03-09T13:58:26.055764844Z",
            "original": """192.168.64.14 - - [07/Feb/2020:12:02:42 +0000] "GET /test/test.mp4 HTTP/1.1" 200 61 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.14; rv:72.0) Gecko/20100101 Firefox/72.0" 348 0.001 [default-web2-8080] [] 172.17.0.6:8080, 172.17.0.7:8080 - 0.100, 0.099 200, 401 835136ae24486dbb4156dcbe21f5d402""",
            "created": "2022-08-27T09:24:26.000Z",
            "kind": "event",
            "category": [
              "web"
            ],
            "type": [
              "info"
            ],
            "outcome": "success"
          },
          "user_agent": {
            "original": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.14; rv:72.0) Gecko/20100101 Firefox/72.0",
            "os": {
              "name": "Mac OS X",
              "version": "10.14",
              "full": "Mac OS X 10.14"
            },
            "name": "Firefox",
            "device": {
              "name": "Mac"
            },
            "version": "72.0."
          },
          "url": {
            "path": "/test/test.mp4",
            "extension": "mp4",
            "original": "/test/test.mp4"
          },
          "tags": [
            "_geoip_database_unavailable_GeoLite2-City.mmdb",
            "_geoip_database_unavailable_GeoLite2-ASN.mmdb"
          ]
        },
        "_ingest": {
          "timestamp": "2023-03-09T13:58:26.055764844Z"
        }
      }
    }
  ]
}

We can see that in case second upstream available message parsing is sucessfull nginx.ingress_controller.upstream.time_list and nginx.ingress_controller.upstream.status_code_list have two values.

[elasticmachine]

Pinging @elastic/integrations (Team:Integrations)