[FEATURE] Is there a way to change the authentication information in the headers when calling the predict API?

[10000-ki]

Is your feature request related to a problem?

Currently, we are configuring it to include authentication information when creating a custom ML connector.

POST /_plugins/_ml/connectors/_create
{
  "name": "Custom Connector: embeddings",
  "description": "The connector to Custom X",
  "version": 1,
  "protocol": "http",
  "credential": {
      "auth": "${token}"
  },
  "parameters": {
      "endpoint": "${endpoint}"
  },
  "actions": [
        {
          "action_type": "predict",
          "method": "POST",
          "url": "https://${parameters.endpoint}",
          "headers": {
            "Authorization": "Bearer ${credential.auth}",
            "Content-Type": "application/json"
          },
          "request_body": "{ \"text\": \"${parameters.inputText}\" }",
        }
    ]
}

The issue is that this authentication token has an expiration period, so it needs to be renewed periodically.

What solution would you like?

Is there a way to pass the auth token value again at the point of calling the predict API or the custom connector API?

POST /_plugins/_ml/models/${model_id}/_predict
{
    "parameters": {
        "inputText": "text embedding..."
    }
}

What alternatives have you considered?
A clear and concise description of any alternative solutions or features you've considered.

Do you have any additional context?
Add any other context or screenshots about the feature request here.

[brianf-aws]

Hi @10000-ki, thank you for posting this. Have you considered the alternative way to update the connector?

You could automate this procedure if you really wanted to you just need the model id and connector id
0. Find associated model id to the connector id

1. un-deploy the associated model
2. Update the connector's credential
3. Redeploy the model
4. Create request

Im trying to think in the long term how an enhancement like this would unfold for example typically connector credentials are stored in a index (I think!) so maybe passing them via a predict would be a security concern?

[brianf-aws]

Actually this is a better alternative instead of un-deploying you can do it like how it states here https://opensearch.org/docs/latest/ml-commons-plugin/remote-models/connectors/#updating-connector-credentials

[ylwu-amzn]

@b4sjoo , I think we have some limitation for updating connector credential. Can you clarify ?

[10000-ki]

@brianf-aws

Actually this is a better alternative instead of un-deploying you can do it like how it states here https://opensearch.org/docs/latest/ml-commons-plugin/remote-models/connectors/#updating-connector-credentials

oh thx!!! i will try

[b4sjoo]

@10000-ki Yeah so currently our stand-alone connector does not support update on-the-fly, right now the work around is to use the inner connector in above link. We have the plan to support the stand-alone connector in the future, but we haven't come into a detailed design yet.

[dhrubo-os]

@10000-ki Does that unblock your use-case?

[10000-ki]

@dhrubo-os

Does that unblock your use-case?

No, the API is not functioning properly. The credentials are not updated correctly. It seems there might be an issue with the API, i think we need to review the code.

[dhrubo-os]

@b4sjoo could you please check?