-
Notifications
You must be signed in to change notification settings - Fork 138
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add Batch Prediction Mode in the Connector Framework for batch inference #2661
Conversation
common/src/main/java/org/opensearch/ml/common/input/remote/RemoteInferenceMLInput.java
Outdated
Show resolved
Hide resolved
common/src/main/java/org/opensearch/ml/common/input/remote/RemoteInferenceMLInput.java
Outdated
Show resolved
Hide resolved
common/src/main/java/org/opensearch/ml/common/input/MLInput.java
Outdated
Show resolved
Hide resolved
common/src/main/java/org/opensearch/ml/common/connector/ConnectorAction.java
Outdated
Show resolved
Hide resolved
common/src/main/java/org/opensearch/ml/common/dataset/remote/RemoteInferenceInputDataSet.java
Show resolved
Hide resolved
Signed-off-by: Xun Zhang <[email protected]>
Signed-off-by: Xun Zhang <[email protected]>
Signed-off-by: Xun Zhang <[email protected]>
Signed-off-by: Xun Zhang <[email protected]>
Signed-off-by: Xun Zhang <[email protected]>
Signed-off-by: Xun Zhang <[email protected]>
Signed-off-by: Xun Zhang <[email protected]>
@@ -124,11 +127,13 @@ public RestChannelConsumer prepareRequest(RestRequest request, NodeClient client | |||
@VisibleForTesting | |||
MLPredictionTaskRequest getRequest(String modelId, String algorithm, RestRequest request) throws IOException { | |||
ActionType actionType = ActionType.from(getActionTypeFromRestRequest(request)); | |||
System.out.println("actionType is " + actionType); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
remove this? or use log?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Already removed. But don't want to waste time running CI again. Will include the removal in my next PR.
if (FunctionName.REMOTE.name().equals(algorithm) && !mlFeatureEnabledSetting.isRemoteInferenceEnabled()) { | ||
throw new IllegalStateException(REMOTE_INFERENCE_DISABLED_ERR_MSG); | ||
} else if (FunctionName.isDLModel(FunctionName.from(algorithm.toUpperCase())) && !mlFeatureEnabledSetting.isLocalModelEnabled()) { | ||
throw new IllegalStateException(LOCAL_MODEL_DISABLED_ERR_MSG); | ||
} else if (!ActionType.isValidActionInModelPrediction(actionType)) { | ||
System.out.println(actionType.toString()); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
same here?
The backport to
To backport manually, run these commands in your terminal: # Fetch latest updates from GitHub
git fetch
# Create a new working tree
git worktree add .worktrees/backport-2.x 2.x
# Navigate to the new working tree
cd .worktrees/backport-2.x
# Create a new branch
git switch --create backport/backport-2661-to-2.x
# Cherry-pick the merged commit of this pull request and resolve the conflicts
git cherry-pick -x --mainline 1 39efbe7eda14b995f8c14a1ece06c7eabc453cd2
# Push it to GitHub
git push --set-upstream origin backport/backport-2661-to-2.x
# Go back to the original working tree
cd ../..
# Delete the working tree
git worktree remove .worktrees/backport-2.x Then, create a pull request where the |
…nce (opensearch-project#2661) * add batch predict job actiontype in connector Signed-off-by: Xun Zhang <[email protected]> * remove async and streaming mode temporarily Signed-off-by: Xun Zhang <[email protected]> * rename predict mode to action type Signed-off-by: Xun Zhang <[email protected]> * use method name in the url path for action type Signed-off-by: Xun Zhang <[email protected]> * add stats for actionType and more UTs Signed-off-by: Xun Zhang <[email protected]> * add bwx for actiontype Signed-off-by: Xun Zhang <[email protected]> * address more comments Signed-off-by: Xun Zhang <[email protected]> --------- Signed-off-by: Xun Zhang <[email protected]>
…nce (#2661) (#2701) * add batch predict job actiontype in connector * remove async and streaming mode temporarily * rename predict mode to action type * use method name in the url path for action type * add stats for actionType and more UTs * add bwx for actiontype * address more comments --------- Signed-off-by: Xun Zhang <[email protected]>
…nce (#2661) (#2701) * add batch predict job actiontype in connector * remove async and streaming mode temporarily * rename predict mode to action type * use method name in the url path for action type * add stats for actionType and more UTs * add bwx for actiontype * address more comments --------- Signed-off-by: Xun Zhang <[email protected]> (cherry picked from commit 310d023)
…nce (#2661) (#2701) (#2705) * add batch predict job actiontype in connector * remove async and streaming mode temporarily * rename predict mode to action type * use method name in the url path for action type * add stats for actionType and more UTs * add bwx for actiontype * address more comments --------- Signed-off-by: Xun Zhang <[email protected]> (cherry picked from commit 310d023) Co-authored-by: Xun Zhang <[email protected]>
…nce (opensearch-project#2661) * add batch predict job actiontype in connector Signed-off-by: Xun Zhang <[email protected]> * remove async and streaming mode temporarily Signed-off-by: Xun Zhang <[email protected]> * rename predict mode to action type Signed-off-by: Xun Zhang <[email protected]> * use method name in the url path for action type Signed-off-by: Xun Zhang <[email protected]> * add stats for actionType and more UTs Signed-off-by: Xun Zhang <[email protected]> * add bwx for actiontype Signed-off-by: Xun Zhang <[email protected]> * address more comments Signed-off-by: Xun Zhang <[email protected]> --------- Signed-off-by: Xun Zhang <[email protected]>
Description
Add the batch mode in the connector to support offline batch inference.
Invoke the offline mode batch prediction
Invoke the realtime predict
Issues Resolved
#2488
Check List
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.