Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enhance: support skip_validating_missing_parameters in connector #2812

Conversation

yuye-aws
Copy link
Member

@yuye-aws yuye-aws commented Aug 8, 2024

Description

When using LLM in ml-commons connector, the prompt may contains substring like ${parameters.xxxx} . The PRs adds a parameter named skip_validating_missing_parameters to enable users to configure whether to skip validating missing parameters in payload.

The user can specify this parameter in the following manner:

POST /_plugins/_ml/connectors/_create
{
  "name": "BedRock test connector",
  "description": "The connector to BedRock service for claude model",
  "version": 1,
  "protocol": "aws_sigv4",
  "parameters": {
      "region": "us-east-1",
      "service_name": "bedrock",
      "anthropic_version": "bedrock-2023-05-31",
      "endpoint": "bedrock-runtime.us-east-1.amazonaws.com",
      "auth": "Sig_V4",
      "content_type": "application/json",
      "max_tokens_to_sample": 8000,
      "temperature": 0.00001,
      "skip_validating_missing_parameters": true
  },
  "credential": {
    "access_key": "xxxx",
    "secret_key": "xxxx"
  },
  "actions": [
    {
      "action_type": "predict",
      "method": "POST",
      "url": "https://bedrock-runtime.us-east-1.amazonaws.com/model/anthropic.claude-instant-v1/invoke",
      "headers": { 
        "content-type": "application/json",
        "x-amz-content-sha256": "required"
      },
      "request_body": "{\"prompt\":\"${parameters.prompt}\", \"max_tokens_to_sample\":${parameters.max_tokens_to_sample}, \"temperature\":${parameters.temperature},  \"anthropic_version\":\"${parameters.anthropic_version}\" }"
    }
  ]
}

Related Issues

Resolves #2712

Check List

  • New functionality includes testing.
  • New functionality has been documented.
  • API changes companion pull request created.
  • Commits are signed per the DCO using --signoff.
  • Public documentation issue/PR created.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

@yuye-aws yuye-aws had a problem deploying to ml-commons-cicd-env August 8, 2024 03:07 — with GitHub Actions Failure
@yuye-aws yuye-aws marked this pull request as draft August 8, 2024 03:07
@yuye-aws yuye-aws had a problem deploying to ml-commons-cicd-env August 8, 2024 03:07 — with GitHub Actions Failure
@yuye-aws yuye-aws force-pushed the Enhance/SkipConnecterParameterValidation branch from b2c8be9 to 89fcef5 Compare August 8, 2024 03:26
@yuye-aws yuye-aws temporarily deployed to ml-commons-cicd-env August 8, 2024 03:26 — with GitHub Actions Inactive
@yuye-aws yuye-aws had a problem deploying to ml-commons-cicd-env August 8, 2024 03:26 — with GitHub Actions Failure
@yuye-aws yuye-aws force-pushed the Enhance/SkipConnecterParameterValidation branch from 89fcef5 to f9a158e Compare August 8, 2024 03:27
@yuye-aws yuye-aws temporarily deployed to ml-commons-cicd-env August 8, 2024 03:27 — with GitHub Actions Inactive
@yuye-aws yuye-aws temporarily deployed to ml-commons-cicd-env August 8, 2024 03:28 — with GitHub Actions Inactive
Signed-off-by: yuye-aws <[email protected]>
@yuye-aws yuye-aws had a problem deploying to ml-commons-cicd-env August 8, 2024 03:54 — with GitHub Actions Failure
@yuye-aws yuye-aws had a problem deploying to ml-commons-cicd-env August 8, 2024 03:54 — with GitHub Actions Failure
Signed-off-by: yuye-aws <[email protected]>
@yuye-aws yuye-aws had a problem deploying to ml-commons-cicd-env August 8, 2024 04:08 — with GitHub Actions Failure
@yuye-aws yuye-aws had a problem deploying to ml-commons-cicd-env August 8, 2024 04:08 — with GitHub Actions Failure
@yuye-aws yuye-aws changed the title Enhance: skip connecter parameter validation Enhance: support skip_validating_missing_parameters in connector Aug 8, 2024
@yuye-aws yuye-aws marked this pull request as ready for review August 8, 2024 05:56
Signed-off-by: yuye-aws <[email protected]>
@yuye-aws
Copy link
Member Author

Please help attach backport 2.x label to this PR. Thanks!

@xinyual xinyual merged commit 9663053 into opensearch-project:main Aug 12, 2024
7 checks passed
opensearch-trigger-bot bot pushed a commit that referenced this pull request Aug 12, 2024
* introduce skip parameter validation

Signed-off-by: yuye-aws <[email protected]>

* implement ut

Signed-off-by: yuye-aws <[email protected]>

* implement it

Signed-off-by: yuye-aws <[email protected]>

* spotless apply

Signed-off-by: yuye-aws <[email protected]>

---------

Signed-off-by: yuye-aws <[email protected]>
(cherry picked from commit 9663053)
@dhrubo-os
Copy link
Collaborator

Is this PR for this RFC?

@yuye-aws
Copy link
Member Author

Is this PR for this RFC?

Yes

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[BUG] KnowledgeBaseTool may return parameter placeholder not filled error
4 participants