[BUG] Fail to ingest document with nested list into text_embedding processor

[yizheliu-amazon]

What is the bug?

When ingesting document with nested list into text_embedding processor, ingestion would fail with class_cast_exception

How can one reproduce the bug?

1. Create ingest pipeline

PUT /_ingest/pipeline/nlp-ingest-pipeline-v4

{
  "description": "An example neural search pipeline",
  "processors": [
    {
      "text_embedding": {
        "model_id": "H7tN0ZMBSNRyUB1vcEi2",
        "field_map": {
          "category": {
            "name": {
              "en": "category_name_vector"
            }
          }
        }
      }
    }
  ]
}

2. Simulate

POST /_ingest/pipeline/nlp-ingest-pipeline-v4/_simulate

{
    "docs": [
        {
            "_index": "neural-search-index-v2",
            "_id": "1",
            "_source": {
                "category": [
                    {
                        "name": {
                            "en": "this is 1st name"
                        }
                    },
                    {
                        "name": {
                            "en": "this is 2nd name"
                        }
                    }
                ]
            }
        }
    ]
}

3. Result

{
    "docs": [
        {
            "error": {
                "root_cause": [
                    {
                        "type": "class_cast_exception",
                        "reason": "class java.util.LinkedHashMap cannot be cast to class java.util.List (java.util.LinkedHashMap and java.util.List are in module java.base of loader 'bootstrap')"
                    }
                ],
                "type": "class_cast_exception",
                "reason": "class java.util.LinkedHashMap cannot be cast to class java.util.List (java.util.LinkedHashMap and java.util.List are in module java.base of loader 'bootstrap')"
            }
        }
    ]
}

Exception is thrown at this line

It seems to be introduced by PR #913 .

What is the expected behavior?

Expected result should be like below:

{
    "docs": [
        {
            "doc": {
                "_index": "neural-search-index-v2",
                "_id": "1",
                "_source": {
                    "category": [
                        {
                            "name": {
                                "category_name_vector": [
                                    -0.10758455,
                                    0.07971476,
                                    -0.04948872,
                                    ...
                                ],
                                "en": "this is 1st name"
                            }
                            
                        },
                        {
                            "name": {
                                "name": [
                                    -0.034477253,
                                    0.031023245,
                                    0.006734962,
                                    ...
                                ],
                                "en": "this is 2nd name"
                            }
                        }
                    ]
                },
                "_ingest": {
                    "timestamp": "2024-04-10T03:51:53.496385Z"
                }
            }
        }
    ]
}

What is your host/environment?

MacOS 13.7.1 (22H221)
Neural Search version: based on top of this commit of main branch

Do you have any screenshots?

N/A

Do you have any additional context?

N/A

[yizheliu-amazon]

@heemin32 Hi Heemin, can you help assign this to me? Thanks.