Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add tutorial for conversational search #2075

Merged
merged 7 commits into from
Feb 13, 2024

Conversation

ylwu-amzn
Copy link
Collaborator

@ylwu-amzn ylwu-amzn commented Feb 12, 2024

Description

  1. add tutorial for how to use Cohere Command model with RAG pipeline
  2. add tutorial for how to build RAG with Bedrock Claude model and conversational flow agent

Issues Resolved

[List any issues this PR will resolve]

Check List

  • New functionality includes testing.
    • All tests pass
  • New functionality has been documented.
    • New functionality has javadoc added
  • Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Copy link

codecov bot commented Feb 12, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Comparison is base (a661176) 81.81% compared to head (d1aa73b) 81.79%.
Report is 7 commits behind head on main.

Additional details and impacted files
@@             Coverage Diff              @@
##               main    #2075      +/-   ##
============================================
- Coverage     81.81%   81.79%   -0.03%     
+ Complexity     5617     5614       -3     
============================================
  Files           543      543              
  Lines         22796    22796              
  Branches       2337     2337              
============================================
- Hits          18651    18646       -5     
- Misses         3209     3215       +6     
+ Partials        936      935       -1     
Flag Coverage Δ
ml-commons 81.79% <ø> (-0.03%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@ylwu-amzn ylwu-amzn temporarily deployed to ml-commons-cicd-env February 12, 2024 19:21 — with GitHub Actions Inactive
@ylwu-amzn ylwu-amzn temporarily deployed to ml-commons-cicd-env February 12, 2024 19:21 — with GitHub Actions Inactive
@ylwu-amzn ylwu-amzn temporarily deployed to ml-commons-cicd-env February 12, 2024 19:21 — with GitHub Actions Inactive
Zhangxunmt
Zhangxunmt previously approved these changes Feb 12, 2024
@@ -0,0 +1,395 @@
# Topic

> Agent Framework is an experimental feature released in OpenSearch 2.12 and not recommended for use in a production environment. For updates on the progress of the feature or if you want to leave feedback, see the associated [GitHub issue](https://github.com/opensearch-project/ml-commons/issues/1161).
Copy link
Collaborator Author

@ylwu-amzn ylwu-amzn Feb 12, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OpenSearch 2.12 not released yet. If you want to try this feature now, you can use the latest RC opensearch-project/opensearch-build#4115 (comment)

"plugins.ml_commons.only_run_on_ml_node": false,
"plugins.ml_commons.native_memory_threshold": 100,
"plugins.ml_commons.agent_framework_enabled": true,
"plugins.ml_commons.memory_feature_enabled": true
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

isn't this enabled by default now?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(memory_feature_enabled - didn't mean to highlight all that)

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Checkd code, it's still disabled, @Zhangxunmt can you enable these once all security issues fixed ? We can remove this part later when it's enabled by default.

Signed-off-by: Yaliang Wu <[email protected]>
@ylwu-amzn ylwu-amzn temporarily deployed to ml-commons-cicd-env February 13, 2024 21:30 — with GitHub Actions Inactive
Signed-off-by: Yaliang Wu <[email protected]>
@ylwu-amzn ylwu-amzn temporarily deployed to ml-commons-cicd-env February 13, 2024 21:40 — with GitHub Actions Inactive
@ylwu-amzn ylwu-amzn temporarily deployed to ml-commons-cicd-env February 13, 2024 21:40 — with GitHub Actions Inactive
@ylwu-amzn ylwu-amzn temporarily deployed to ml-commons-cicd-env February 13, 2024 21:40 — with GitHub Actions Inactive
@ylwu-amzn ylwu-amzn temporarily deployed to ml-commons-cicd-env February 13, 2024 21:41 — with GitHub Actions Inactive
@ylwu-amzn ylwu-amzn temporarily deployed to ml-commons-cicd-env February 13, 2024 21:41 — with GitHub Actions Inactive
@ylwu-amzn ylwu-amzn temporarily deployed to ml-commons-cicd-env February 13, 2024 21:41 — with GitHub Actions Inactive
@ylwu-amzn ylwu-amzn temporarily deployed to ml-commons-cicd-env February 13, 2024 21:50 — with GitHub Actions Inactive
@ylwu-amzn ylwu-amzn temporarily deployed to ml-commons-cicd-env February 13, 2024 21:50 — with GitHub Actions Inactive
@ylwu-amzn ylwu-amzn temporarily deployed to ml-commons-cicd-env February 13, 2024 21:50 — with GitHub Actions Inactive
@ylwu-amzn ylwu-amzn temporarily deployed to ml-commons-cicd-env February 13, 2024 21:51 — with GitHub Actions Inactive
@ylwu-amzn ylwu-amzn temporarily deployed to ml-commons-cicd-env February 13, 2024 21:51 — with GitHub Actions Inactive
@ylwu-amzn ylwu-amzn temporarily deployed to ml-commons-cicd-env February 13, 2024 21:51 — with GitHub Actions Inactive
@ylwu-amzn ylwu-amzn temporarily deployed to ml-commons-cicd-env February 13, 2024 22:19 — with GitHub Actions Inactive
@ylwu-amzn ylwu-amzn temporarily deployed to ml-commons-cicd-env February 13, 2024 22:19 — with GitHub Actions Inactive
@ylwu-amzn ylwu-amzn temporarily deployed to ml-commons-cicd-env February 13, 2024 22:19 — with GitHub Actions Inactive
@Zhangxunmt Zhangxunmt merged commit 7ecc695 into opensearch-project:main Feb 13, 2024
13 of 14 checks passed
opensearch-trigger-bot bot pushed a commit that referenced this pull request Feb 13, 2024
* add tutorial for conversational search

Signed-off-by: Yaliang Wu <[email protected]>

* add more doc links

Signed-off-by: Yaliang Wu <[email protected]>

* add advanced topics

Signed-off-by: Yaliang Wu <[email protected]>

* add demo of PPLTool

Signed-off-by: Yaliang Wu <[email protected]>

* Apply suggestions from code review

Co-authored-by: kolchfa-aws <[email protected]>
Signed-off-by: Yaliang Wu <[email protected]>

* remvoe unnecessary setting

Signed-off-by: Yaliang Wu <[email protected]>

* use memory/message to keep consistent

Signed-off-by: Yaliang Wu <[email protected]>

---------

Signed-off-by: Yaliang Wu <[email protected]>
Co-authored-by: kolchfa-aws <[email protected]>
(cherry picked from commit 7ecc695)
opensearch-trigger-bot bot pushed a commit that referenced this pull request Feb 13, 2024
* add tutorial for conversational search

Signed-off-by: Yaliang Wu <[email protected]>

* add more doc links

Signed-off-by: Yaliang Wu <[email protected]>

* add advanced topics

Signed-off-by: Yaliang Wu <[email protected]>

* add demo of PPLTool

Signed-off-by: Yaliang Wu <[email protected]>

* Apply suggestions from code review

Co-authored-by: kolchfa-aws <[email protected]>
Signed-off-by: Yaliang Wu <[email protected]>

* remvoe unnecessary setting

Signed-off-by: Yaliang Wu <[email protected]>

* use memory/message to keep consistent

Signed-off-by: Yaliang Wu <[email protected]>

---------

Signed-off-by: Yaliang Wu <[email protected]>
Co-authored-by: kolchfa-aws <[email protected]>
(cherry picked from commit 7ecc695)
ylwu-amzn added a commit that referenced this pull request Feb 13, 2024
* add tutorial for conversational search

Signed-off-by: Yaliang Wu <[email protected]>

* add more doc links

Signed-off-by: Yaliang Wu <[email protected]>

* add advanced topics

Signed-off-by: Yaliang Wu <[email protected]>

* add demo of PPLTool

Signed-off-by: Yaliang Wu <[email protected]>

* Apply suggestions from code review

Co-authored-by: kolchfa-aws <[email protected]>
Signed-off-by: Yaliang Wu <[email protected]>

* remvoe unnecessary setting

Signed-off-by: Yaliang Wu <[email protected]>

* use memory/message to keep consistent

Signed-off-by: Yaliang Wu <[email protected]>

---------

Signed-off-by: Yaliang Wu <[email protected]>
Co-authored-by: kolchfa-aws <[email protected]>
(cherry picked from commit 7ecc695)

Co-authored-by: Yaliang Wu <[email protected]>
ylwu-amzn added a commit that referenced this pull request Feb 13, 2024
* add tutorial for conversational search

Signed-off-by: Yaliang Wu <[email protected]>

* add more doc links

Signed-off-by: Yaliang Wu <[email protected]>

* add advanced topics

Signed-off-by: Yaliang Wu <[email protected]>

* add demo of PPLTool

Signed-off-by: Yaliang Wu <[email protected]>

* Apply suggestions from code review

Co-authored-by: kolchfa-aws <[email protected]>
Signed-off-by: Yaliang Wu <[email protected]>

* remvoe unnecessary setting

Signed-off-by: Yaliang Wu <[email protected]>

* use memory/message to keep consistent

Signed-off-by: Yaliang Wu <[email protected]>

---------

Signed-off-by: Yaliang Wu <[email protected]>
Co-authored-by: kolchfa-aws <[email protected]>
(cherry picked from commit 7ecc695)

Co-authored-by: Yaliang Wu <[email protected]>
austintlee pushed a commit to austintlee/ml-commons that referenced this pull request Mar 19, 2024
* add tutorial for conversational search

Signed-off-by: Yaliang Wu <[email protected]>

* add more doc links

Signed-off-by: Yaliang Wu <[email protected]>

* add advanced topics

Signed-off-by: Yaliang Wu <[email protected]>

* add demo of PPLTool

Signed-off-by: Yaliang Wu <[email protected]>

* Apply suggestions from code review

Co-authored-by: kolchfa-aws <[email protected]>
Signed-off-by: Yaliang Wu <[email protected]>

* remvoe unnecessary setting

Signed-off-by: Yaliang Wu <[email protected]>

* use memory/message to keep consistent

Signed-off-by: Yaliang Wu <[email protected]>

---------

Signed-off-by: Yaliang Wu <[email protected]>
Co-authored-by: kolchfa-aws <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants