Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Finish onboarding ML inference search request / response processors #182

Merged
merged 4 commits into from
Jun 17, 2024

Conversation

ohltyler
Copy link
Member

Description

This PR finishes up what was started in #180. Specifically, it finishes the logic for dynamically generating the ReactFlow workspace nodes/edges, and the workflow template with a create_search_pipeline workflow step.

  • refactors and adds logic in workflow_to_template_utils to generate the create_search_pipeline step with any configured search request and search response processors. Also updates the create_index step to gracefully handle the cases of no ingest or search pipeline defined, which is allowed in the UI.
  • refactors and adds logic in the base utils to generate the dynamic list of nodes/edges in the search flow

Demo video showing the workspace flow (template configuration and testing different edge cases of search and ingest pipelines was verified manually - can't test as there is no final implementation of the ML processors on the search side, yet):

screen-capture.45.webm

Issues Resolved

Makes progress on #23

Check List

  • Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Copy link
Member

@owaiskazi19 owaiskazi19 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minor refactoring comments. Rest LGTM

Signed-off-by: Tyler Ohlsen <[email protected]>
@ohltyler ohltyler requested a review from owaiskazi19 June 17, 2024 19:26
Copy link
Member

@owaiskazi19 owaiskazi19 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for addressing the comments

@ohltyler ohltyler merged commit d0a9846 into opensearch-project:main Jun 17, 2024
6 checks passed
@ohltyler ohltyler deleted the ml-search-processors-cont branch June 17, 2024 21:24
opensearch-trigger-bot bot pushed a commit that referenced this pull request Jun 17, 2024
…182)

* Support dynamic workflow generation based on req/resp processors

Signed-off-by: Tyler Ohlsen <[email protected]>

* Onboard search processors to be dynamically added to template

Signed-off-by: Tyler Ohlsen <[email protected]>

* one line improvement

Signed-off-by: Tyler Ohlsen <[email protected]>

* Minor refactorings

Signed-off-by: Tyler Ohlsen <[email protected]>

---------

Signed-off-by: Tyler Ohlsen <[email protected]>
(cherry picked from commit d0a9846)
ohltyler added a commit that referenced this pull request Jun 17, 2024
…182) (#183)

Signed-off-by: Tyler Ohlsen <[email protected]>
(cherry picked from commit d0a9846)

Co-authored-by: Tyler Ohlsen <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants