Final Pipeline is not applied when routing using a pipeline

[m-adams]

Elasticsearch version (bin/elasticsearch --version): 7.11.1 , 7.12.0

Plugins installed: []

JVM version (java -version): Tested on ESS

OS version (uname -a if on a Unix-like system): Tested on ESS

Description of the problem including expected versus actual behavior:

When routing to the correct index by using a pipeline e.g. using the date, the index template is applied but the documents seem to not go through the specified final pipeline.
This was reported by a user who had this successfully working on 7.8

The expected behaviour is that after routing, all index template settings are applied including the final pipeline

Steps to reproduce:

Dev tools

`DELETE test-*

PUT _ingest/pipeline/test-pipeline
{
  "processors": [
    {
      "set": {
        "field": "pipeline",
        "value": "completed"
      }
    }
  ]
}

POST _index_template/test-template
{
  "index_patterns": [
    "test-*"
  ],
  "template": {
                  "settings": {
                    "final_pipeline": "test-pipeline",
                    "number_of_shards": 5
                  }
  }
}

POST /test-1/_doc
{
  "time-field": "2021-02-10T12:02:01.789Z",
  "CONTENT": "blah"
}

GET /test-1/_search

This works, the final pipeline was applied
Now lets route the document using a pipeline

PUT /_ingest/pipeline/routing-test-pipeline
{
  "description": "Time series DAY pipeline",
  "processors": [
     {
         "date_index_name": {
             "field": "time-field",
             "index_name_prefix": "test-",
             "date_rounding": "d",
             "index_name_format": "yyyy-MM-dd"
         }
     }
 ]
}

POST /test-1/_doc?pipeline=routing-test-pipeline
{
  "time-field": "2021-02-10T12:02:01.789Z",
  "CONTENT": "blah"
}

GET /test-2021-02-10/_search

Expected Behaviour:
Documents in the index test-2021-02-10 will have the field "pipeline" : "completed" included.

Observed Behaviour:
The field is not present. E.g:

     {
        "_index" : "test-2021-02-10",
        "_type" : "_doc",
        "_id" : "Aju4pngBFgDNhff2pq3R",
        "_score" : 1.0,
        "_source" : {
          "CONTENT" : "blah",
          "time-field" : "2021-02-10T12:02:01.789Z"
        }
      },

Note that GET test-2021-02-10 shows that the final_pipeline is still in the index settings:

{
  "test-2021-02-10" : {
    "aliases" : { },
    "mappings" : {
      "properties" : {
        "CONTENT" : {
          "type" : "text",
          "fields" : {
            "keyword" : {
              "type" : "keyword",
              "ignore_above" : 256
            }
          }
        },
        "time-field" : {
          "type" : "date"
        }
      }
    },
    "settings" : {
      "index" : {
        "routing" : {
          "allocation" : {
            "include" : {
              "_tier_preference" : "data_content"
            }
          }
        },
        "number_of_shards" : "5",
        "provided_name" : "<test-{2021-02-10||/d{yyyy-MM-dd|UTC}}>",
        "final_pipeline" : "test-pipeline",
        "creation_date" : "1617704822016",
        "number_of_replicas" : "1",
        "uuid" : "_i_r6vukRjWsUC7vOUBu_w",
        "version" : {
          "created" : "7120099"
        }
      }
    }
  }
}

[elasticmachine]

Pinging @elastic/es-core-features (Team:Core/Features)

[weslambert]

Was this ever merged into a 7.x release?

[martijnvg]

Was this ever merged into a 7.x release?

Looking at #71433, it looks like this bug got fixed in version 7.12.1 and later.