Validate index name in data stream mode of reroute processor

[gbanasiak]

Version: 8.15.0

Description:
Originally noted in #111828 (comment).

Reroute processor does not seem to validate index name format convention when running in data stream mode which leads to unexpected results.

PUT _ingest/pipeline/test-reroute
{
  "processors": [
    {
      "reroute": {
        "dataset": [
          "{{data_stream.dataset}}"
        ],
        "namespace": [
          "test"
        ]
      }
    }
  ]
}

POST _ingest/pipeline/test-reroute/_simulate
{
  "docs": [
    {
      "_index": ".ds-logs-system.auth-default-2023.01.04-000058", <--- incorrect, should be data stream name, i.e. "logs-system.auth-default"
      "_source": {
        "data_stream": {
          "namespace": "default",
          "type": "logs",
          "dataset": "system.auth"
        }
      }
    }
  ]
}

Result:

{
  "docs": [
    {
      "doc": {
        "_index": ".ds-system.auth-test",
        "_version": "-3",
        "_id": "_id",
        "_source": {
          "data_stream": {
            "namespace": "test",
            "type": ".ds", <--- type corruption
            "dataset": "system.auth"
          }
        },
        "_ingest": {
          "timestamp": "2024-09-05T08:47:47.6544947Z"
        }
      }
    }
  ]
}

Expected result:
Reroute processor should raise error/exception when index name does not meet data stream naming convention when running in data stream mode.

[elasticsearchmachine]

Pinging @elastic/es-data-management (Team:Data Management)

[kanoshiou]

Hi, @gbanasiak.

May I take on this issue?

I'm considering raising an exception in the execute method of RerouteProcessor by checking if the number of dashes equals two. However, I'm not sure if this aligns with your expectations. Could you please provide some guidance or let me know if there are any specific requirements I should be aware of?