Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix bedrock embedding generation issue #2495

Merged
merged 11 commits into from
Jun 11, 2024

Conversation

zane-neo
Copy link
Collaborator

@zane-neo zane-neo commented Jun 4, 2024

Description

Fix bedrock embedding generation issue

Issues Resolved

#2490

Check List

  • New functionality includes testing.
    • All tests pass
  • New functionality has been documented.
    • New functionality has javadoc added
  • Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

@zane-neo zane-neo had a problem deploying to ml-commons-cicd-env June 4, 2024 08:03 — with GitHub Actions Error
@zane-neo zane-neo had a problem deploying to ml-commons-cicd-env June 4, 2024 08:03 — with GitHub Actions Error
@zane-neo zane-neo had a problem deploying to ml-commons-cicd-env June 4, 2024 08:03 — with GitHub Actions Failure
@zane-neo zane-neo had a problem deploying to ml-commons-cicd-env June 4, 2024 08:03 — with GitHub Actions Failure
@zane-neo zane-neo had a problem deploying to ml-commons-cicd-env June 4, 2024 08:03 — with GitHub Actions Error
@zane-neo zane-neo had a problem deploying to ml-commons-cicd-env June 4, 2024 08:03 — with GitHub Actions Error
@zane-neo zane-neo had a problem deploying to ml-commons-cicd-env June 4, 2024 08:10 — with GitHub Actions Failure
@zane-neo zane-neo had a problem deploying to ml-commons-cicd-env June 4, 2024 08:10 — with GitHub Actions Error
@zane-neo zane-neo had a problem deploying to ml-commons-cicd-env June 4, 2024 08:10 — with GitHub Actions Error
@zane-neo zane-neo had a problem deploying to ml-commons-cicd-env June 4, 2024 08:10 — with GitHub Actions Failure
@zane-neo zane-neo had a problem deploying to ml-commons-cicd-env June 4, 2024 08:10 — with GitHub Actions Error
@zane-neo zane-neo had a problem deploying to ml-commons-cicd-env June 4, 2024 08:10 — with GitHub Actions Error
@zane-neo zane-neo temporarily deployed to ml-commons-cicd-env June 4, 2024 08:56 — with GitHub Actions Inactive
@zane-neo zane-neo temporarily deployed to ml-commons-cicd-env June 4, 2024 08:56 — with GitHub Actions Inactive
@zane-neo zane-neo temporarily deployed to ml-commons-cicd-env June 4, 2024 08:56 — with GitHub Actions Inactive
@zane-neo zane-neo had a problem deploying to ml-commons-cicd-env June 4, 2024 08:56 — with GitHub Actions Error
@zane-neo zane-neo had a problem deploying to ml-commons-cicd-env June 4, 2024 08:56 — with GitHub Actions Error
@zane-neo zane-neo had a problem deploying to ml-commons-cicd-env June 4, 2024 08:56 — with GitHub Actions Failure
@zane-neo zane-neo temporarily deployed to ml-commons-cicd-env June 4, 2024 12:02 — with GitHub Actions Inactive
@zane-neo zane-neo force-pushed the fix-bedrock-embedding-gen branch from 148c25f to a9d0963 Compare June 11, 2024 01:45
@zane-neo zane-neo had a problem deploying to ml-commons-cicd-env June 11, 2024 01:45 — with GitHub Actions Error
@zane-neo zane-neo had a problem deploying to ml-commons-cicd-env June 11, 2024 01:45 — with GitHub Actions Error
@zane-neo zane-neo had a problem deploying to ml-commons-cicd-env June 11, 2024 01:45 — with GitHub Actions Failure
@zane-neo zane-neo had a problem deploying to ml-commons-cicd-env June 11, 2024 01:45 — with GitHub Actions Error
@zane-neo zane-neo had a problem deploying to ml-commons-cicd-env June 11, 2024 01:45 — with GitHub Actions Error
@zane-neo zane-neo had a problem deploying to ml-commons-cicd-env June 11, 2024 01:45 — with GitHub Actions Failure
@zane-neo zane-neo had a problem deploying to ml-commons-cicd-env June 11, 2024 02:18 — with GitHub Actions Error
@zane-neo zane-neo had a problem deploying to ml-commons-cicd-env June 11, 2024 02:18 — with GitHub Actions Error
@zane-neo zane-neo had a problem deploying to ml-commons-cicd-env June 11, 2024 02:18 — with GitHub Actions Failure
@zane-neo zane-neo had a problem deploying to ml-commons-cicd-env June 11, 2024 02:18 — with GitHub Actions Error
@zane-neo zane-neo had a problem deploying to ml-commons-cicd-env June 11, 2024 02:18 — with GitHub Actions Error
@zane-neo zane-neo had a problem deploying to ml-commons-cicd-env June 11, 2024 02:18 — with GitHub Actions Failure
@zane-neo zane-neo had a problem deploying to ml-commons-cicd-env June 11, 2024 02:52 — with GitHub Actions Error
@zane-neo zane-neo had a problem deploying to ml-commons-cicd-env June 11, 2024 02:52 — with GitHub Actions Error
@zane-neo zane-neo had a problem deploying to ml-commons-cicd-env June 11, 2024 02:52 — with GitHub Actions Failure
@zane-neo zane-neo had a problem deploying to ml-commons-cicd-env June 11, 2024 02:52 — with GitHub Actions Error
@zane-neo zane-neo had a problem deploying to ml-commons-cicd-env June 11, 2024 02:52 — with GitHub Actions Error
@zane-neo zane-neo had a problem deploying to ml-commons-cicd-env June 11, 2024 02:52 — with GitHub Actions Failure
@zane-neo
Copy link
Collaborator Author

zane-neo commented Jun 11, 2024

The failure checks are caused by circuit breaker, seems github resources are not enough to run some ITs, e.g.:

REPRODUCE WITH: gradlew ':opensearch-ml-plugin:integTest' --tests "org.opensearch.ml.rest.RestMLFlowAgentIT.testAgentSearchIndexTool" -Dtests.seed=C9BBBF8374D42354 -Dtests.security.manager=false -Dtests.locale=lv-LV -Dtests.timezone=America/Argentina/Rio_Gallegos -Druntime.java=21

org.opensearch.ml.rest.RestMLFlowAgentIT > testAgentSearchIndexTool FAILED
    org.opensearch.client.ResponseException: method [POST], host [http://127.0.0.1:64840/], URI [/_plugins/_ml/agents/rvA9BZABoBZUGpIM8j1j/_execute], status line [HTTP/1.1 429 Too Many Requests]
    {"error":{"reason":"System Error","details":"Memory Circuit Breaker is open, please check your resources!","type":"CircuitBreakingException"},"status":429}
        at __randomizedtesting.SeedInfo.seed([C9BBBF8374D42354:742BA860D5C91457]:0)
        at app//org.opensearch.client.RestClient.convertResponse(RestClient.java:385)
        at app//org.opensearch.client.RestClient.performRequest(RestClient.java:355)
        at app//org.opensearch.client.RestClient.performRequest(RestClient.java:330)
        at app//org.opensearch.ml.utils.TestHelper.makeRequest(TestHelper.java:179)
        at app//org.opensearch.ml.utils.TestHelper.makeRequest(TestHelper.java:152)
        at app//org.opensearch.ml.rest.RestMLFlowAgentIT.executeAgent(RestMLFlowAgentIT.java:131)
        at app//org.opensearch.ml.rest.RestMLFlowAgentIT.executeAgentSearchIndexTool(RestMLFlowAgentIT.java:115)
        at app//org.opensearch.ml.rest.RestMLFlowAgentIT.testAgentSearchIndexTool(RestMLFlowAgentIT.java:62)

Have created this issue to track this: #2528.

@zane-neo zane-neo had a problem deploying to ml-commons-cicd-env June 11, 2024 20:58 — with GitHub Actions Failure
@zane-neo zane-neo had a problem deploying to ml-commons-cicd-env June 11, 2024 20:58 — with GitHub Actions Failure
@zane-neo zane-neo had a problem deploying to ml-commons-cicd-env June 11, 2024 20:58 — with GitHub Actions Failure
@jngz-es jngz-es merged commit 62e238f into opensearch-project:main Jun 11, 2024
4 of 10 checks passed
@zane-neo zane-neo had a problem deploying to ml-commons-cicd-env June 11, 2024 21:01 — with GitHub Actions Failure
@zane-neo zane-neo had a problem deploying to ml-commons-cicd-env June 11, 2024 21:02 — with GitHub Actions Error
@zane-neo zane-neo had a problem deploying to ml-commons-cicd-env June 11, 2024 21:02 — with GitHub Actions Error
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants