support question answering model

[rbhavna]

Description

support question answering model. This PR adds question answering model to the list of existing list of models supported by ml-commons. It expects question and context and gives the answer based on the context provides. Below is a sample predict API request to QA model and its expected output

POST /_plugins/_ml/models/m6LBVI4BuIvXgszWP7KN/_predict
{
    "question": "Where do I live",
    "context":  "I am Clara. I live in Texas."
}

// Response
{
    "inference_results": [
        {
            "output": [
                {
                    "result": "Texas"
                }
            ]
        }
    ]
}
            

Issues Resolved

#1873

Check List

• New functionality includes testing.
  • All tests pass
• New functionality has been documented.
  • New functionality has javadoc added
• Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

[dhrubo-os]

Let's add a map, this branch will get longer and longer....

[dhrubo-os]

This model will also eventually use TextEmbeddingModelConfig, which is not ideal. Let's create a separate model config for this type of model.

[rbhavna]

I am trying to see what config fields we will need for QA model and add accordingly

[dhrubo-os]

@HenryL27 could you please review the PR?

[codecov]

Codecov Report

Attention: Patch coverage is 62.45734% with 110 lines in your changes are missing coverage. Please review.

Project coverage is 81.70%. Comparing base (189f2a2) to head (4de3643).
Report is 5 commits behind head on main.

❗ Current head 4de3643 differs from pull request most recent head fb127f2. Consider uploading reports for the commit fb127f2 to get more accurate results

Files	Patch %	Lines
.../ml/common/model/QuestionAnsweringModelConfig.java	60.43%	27 Missing and 9 partials ⚠️
...ain/java/org/opensearch/ml/engine/ModelHelper.java	30.76%	34 Missing and 2 partials ⚠️
...n/java/org/opensearch/ml/common/input/MLInput.java	13.33%	11 Missing and 2 partials ⚠️
.../ml/common/input/nlp/QuestionAnsweringMLInput.java	82.05%	3 Missing and 4 partials ⚠️
...ommon/transport/register/MLRegisterModelInput.java	25.00%	3 Missing and 3 partials ⚠️
...rc/main/java/org/opensearch/ml/common/MLModel.java	0.00%	2 Missing and 2 partials ⚠️
...opensearch/ml/common/output/model/ModelTensor.java	0.00%	4 Missing ⚠️
...ansport/upload_chunk/MLRegisterModelMetaInput.java	33.33%	2 Missing and 2 partials ⚠️

Additional details and impacted files

@@             Coverage Diff              @@
##               main    #2208      +/-   ##
============================================
- Coverage     81.90%   81.70%   -0.21%     
- Complexity     5719     5755      +36     
============================================
  Files           547      552       +5     
  Lines         23075    23325     +250     
  Branches       2378     2409      +31     
============================================
+ Hits          18900    19057     +157     
- Misses         3230     3302      +72     
- Partials        945      966      +21     

Flag	Coverage Δ
ml-commons	81.70% <62.45%> (-0.21%)	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[ylwu-amzn]

I remember you plan to add new input/output type for QA model, will you add it in next commit?

[rbhavna]

Yes pushing the new commit with the following input/out formats
// Input { "question": "What color is Apple", "context": "I like Apples. Because they are red" }

// output { "inference_results": [ { "output": [ { "result": "red" } ] } ] }

[dhrubo-os]

Do we still needs to add FunctionName.QUESTION_ANSWERING here?

[rbhavna]

yeah good catch, we don't need it here anymore. Thanks @dhrubo-os will remove it

[opensearch-trigger-bot]

The backport to 2.x failed:

The process '/usr/bin/git' failed with exit code 1

To backport manually, run these commands in your terminal:

# Fetch latest updates from GitHub
git fetch
# Create a new working tree
git worktree add .worktrees/backport-2.x 2.x
# Navigate to the new working tree
cd .worktrees/backport-2.x
# Create a new branch
git switch --create backport/backport-2208-to-2.x
# Cherry-pick the merged commit of this pull request and resolve the conflicts
git cherry-pick -x --mainline 1 c560fcca6ee2e413e7ebc2503a2e64ad691e6e2b
# Push it to GitHub
git push --set-upstream origin backport/backport-2208-to-2.x
# Go back to the original working tree
cd ../..
# Delete the working tree
git worktree remove .worktrees/backport-2.x

Then, create a pull request where the base branch is 2.x and the compare/head branch is backport/backport-2208-to-2.x.

[opensearch-trigger-bot]

The backport to 2.13 failed:

The process '/usr/bin/git' failed with exit code 1

To backport manually, run these commands in your terminal:

# Fetch latest updates from GitHub
git fetch
# Create a new working tree
git worktree add .worktrees/backport-2.13 2.13
# Navigate to the new working tree
cd .worktrees/backport-2.13
# Create a new branch
git switch --create backport/backport-2208-to-2.13
# Cherry-pick the merged commit of this pull request and resolve the conflicts
git cherry-pick -x --mainline 1 c560fcca6ee2e413e7ebc2503a2e64ad691e6e2b
# Push it to GitHub
git push --set-upstream origin backport/backport-2208-to-2.13
# Go back to the original working tree
cd ../..
# Delete the working tree
git worktree remove .worktrees/backport-2.13

Then, create a pull request where the base branch is 2.13 and the compare/head branch is backport/backport-2208-to-2.13.