Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG] ML task index mapping is wrong #2948

Closed
ylwu-amzn opened this issue Sep 13, 2024 · 3 comments · Fixed by #2949
Closed

[BUG] ML task index mapping is wrong #2948

ylwu-amzn opened this issue Sep 13, 2024 · 3 comments · Fixed by #2949
Assignees
Labels
bug Something isn't working untriaged

Comments

@ylwu-amzn
Copy link
Collaborator

ML task index mapping is wrong, remote_job field part will be ignored.
https://github.com/opensearch-project/ml-commons/blob/main/common/src/main/java/org/opensearch/ml/common/CommonValue.java#L353

{
    "_meta": {"schema_version": 3},
    "properties": {
      "model_id": {"type": "keyword"},
      "task_type": {"type": "keyword"},
      "function_name": {"type": "keyword"},
      "state": {"type": "keyword"},
      "input_type": {"type": "keyword"},
      "progress": {"type": "float"},
      "output_index": {"type": "keyword"},
      "worker_node": {"type": "keyword"},
      "create_time": {"type": "date", "format": "strict_date_time||epoch_millis"},
      "last_update_time": {"type": "date", "format": "strict_date_time||epoch_millis"},
      "error": {"type": "text"},
      "is_async" : {"type" : "boolean"}, 
      "user": {
        "type": "nested",
        "properties": {
          "name": {"type":"text", "fields":{"keyword":{"type":"keyword", "ignore_above":256}}},
          "backend_roles": {"type":"text", "fields":{"keyword":{"type":"keyword"}}},
          "roles": {"type":"text", "fields":{"keyword":{"type":"keyword"}}},
          "custom_attribute_names": {"type":"text", "fields":{"keyword":{"type":"keyword"}}}
        }
      }
    }
},"remote_job" : {"type": "flat_object"}
    }
}
@ylwu-amzn ylwu-amzn added bug Something isn't working untriaged labels Sep 13, 2024
@ylwu-amzn
Copy link
Collaborator Author

Use java code to construct index mapping is error-prone. Let's create index mapping json file like AD https://github.com/opensearch-project/anomaly-detection/tree/main/src/main/resources/mappings

@dhrubo-os
Copy link
Collaborator

Use java code to construct index mapping is error-prone. Let's create index mapping json file like AD https://github.com/opensearch-project/anomaly-detection/tree/main/src/main/resources/mappings

+1 on this. Should we create a separate issue on this?

@ylwu-amzn
Copy link
Collaborator Author

#2951

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working untriaged
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants