Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG] Incorrect validation logic for map type in xxxProcessor #739

Closed
zane-neo opened this issue May 8, 2024 · 3 comments
Closed

[BUG] Incorrect validation logic for map type in xxxProcessor #739

zane-neo opened this issue May 8, 2024 · 3 comments
Assignees
Labels
bug Something isn't working

Comments

@zane-neo
Copy link
Collaborator

zane-neo commented May 8, 2024

What is the bug?
When user use map type configuration in several processors, the validation can fail since validation is been done on extra fields in that map.
How can one reproduce the bug?
Steps to reproduce the behavior:

PUT /_ingest/pipeline/neural-search-pipeline-v2
{
  "description": "An example neural search pipeline",
  "processors": [
    {
      "text_embedding": {
        "model_id": "WeliNowB6EaQJ_XFf05V",
        "field_map": {
          "category": {
            "name": {
              "en": "category_name_vector"
            }
          }
        }
      }
    }
  ]
}

And simulate the ingestion:

POST _ingest/pipeline/neural-search-pipeline-v2/_simulate
{
  "docs": [
    {
      "_index": "neural-search-index-v2",
      "_id": "1",
      "_source": {
        "category": {
          "id": 1,
          "name": {
            "en": "category 1"
          }
        }
      }
    }
  ]
}

Then user can get error like below:

{
  "docs": [
    {
      "error": {
        "root_cause": [
          {
            "type": "illegal_argument_exception",
            "reason": "map type field [category] has non-string type, cannot process it"
          }
        ],
        "type": "illegal_argument_exception",
        "reason": "map type field [category] has non-string type, cannot process it"
      }
    }
  ]
}

What is the expected behavior?
Correct embedding should be generated and inserted to the document.

What is your host/environment?

  • OS: [e.g. iOS]
  • Version [e.g. 22]
  • Plugins

Do you have any screenshots?
If applicable, add screenshots to help explain your problem.

Do you have any additional context?
Root cause is when validating map type data, not only the expected field is validated, but also the unrelated fields are been validated, in above example, "id" is been validated since it's under the category map, and it's value is integer which doesn't supported in text embedding thus the error.

@zane-neo zane-neo added bug Something isn't working untriaged labels May 8, 2024
@zane-neo zane-neo self-assigned this May 8, 2024
@zane-neo zane-neo removed the untriaged label May 8, 2024
@yuye-aws
Copy link
Member

yuye-aws commented May 9, 2024

How about resolve this bug in #687?

@zane-neo
Copy link
Collaborator Author

How about resolve this bug in #687?

Yes, that's the PR to fix this.

@zhichao-aws
Copy link
Member

Fixed in #687

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

3 participants