Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve input/output transformation configuration #504

Merged
merged 3 commits into from
Nov 27, 2024

Conversation

ohltyler
Copy link
Member

@ohltyler ohltyler commented Nov 27, 2024

Description

This PR merges the feature/new-transform feature branch to main. The primary changes are:

  • adding transform types when configuring inputs
  • adding transform types when configuring outputs
  • configuring static values & templates in-line for model inputs (no standalone modals needed anymore)
  • adding new dedicated modals for configuring some of the different input types, including JSONPath expressions, templates, and multi-JSONPath-expressions (output transforms only). These borrow & refactor a lot of the logic used in the legacy modals, which have now been removed

This changes how the configs for input/output transforms are persisted. Hence, old workflows created with the UI will fail.

Testing

Tested several use cases and verified the modals work as expected, including the presets, using connector / model configs based off of the tutorial

  • semantic search
  • RAG
  • hybrid search
  • NER
  • language detection

Issues resolved

Closes #491

Check List

  • Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

* Set up the config -> form -> schema -> template transformations for input_map

Signed-off-by: Tyler Ohlsen <[email protected]>

* Handle empty casse

Signed-off-by: Tyler Ohlsen <[email protected]>

* Add new input type selector

Signed-off-by: Tyler Ohlsen <[email protected]>

* Fix bug of model interface not autopopulating in map

Signed-off-by: Tyler Ohlsen <[email protected]>

* Support preset templates

Signed-off-by: Tyler Ohlsen <[email protected]>

* Dynamically render forms; edge case handling of empty docs

Signed-off-by: Tyler Ohlsen <[email protected]>

* Add bucketing logic and add to model config if applicable

Signed-off-by: Tyler Ohlsen <[email protected]>

* Set up new configure_template_modal; add config button when template type selected

Signed-off-by: Tyler Ohlsen <[email protected]>

* Add nested form state in new template modal

Signed-off-by: Tyler Ohlsen <[email protected]>

* Update layout

Signed-off-by: Tyler Ohlsen <[email protected]>

* Update template val form config to support subvars

Signed-off-by: Tyler Ohlsen <[email protected]>

* Add multi input var component & form integration

Signed-off-by: Tyler Ohlsen <[email protected]>

* Add var copying into input var list

Signed-off-by: Tyler Ohlsen <[email protected]>

* Update spacing

Signed-off-by: Tyler Ohlsen <[email protected]>

* Add fetching source data functionality

Signed-off-by: Tyler Ohlsen <[email protected]>

* Fix some spacing; add output jsonfield

Signed-off-by: Tyler Ohlsen <[email protected]>

* committing but broken

Signed-off-by: Tyler Ohlsen <[email protected]>

* Show transformed prompt

Signed-off-by: Tyler Ohlsen <[email protected]>

* add spacer

Signed-off-by: Tyler Ohlsen <[email protected]>

* Add nested template vars to input map

Signed-off-by: Tyler Ohlsen <[email protected]>

* Set up base expression modal

Signed-off-by: Tyler Ohlsen <[email protected]>

* Get expression modal working

Signed-off-by: Tyler Ohlsen <[email protected]>

* Add dynamic rendering if template or expression is populated

Signed-off-by: Tyler Ohlsen <[email protected]>

---------

Signed-off-by: Tyler Ohlsen <[email protected]>
@ohltyler
Copy link
Member Author

I will update the tutorial in a later PR.

@ohltyler
Copy link
Member Author

There will also be minor tuning to the preview display values for the different input/output transforms, and some helper text, helper examples of JSONPath, etc. to be added in later PRs. This is all being finalized from UX.

@saimedhi
Copy link
Collaborator

The feature branch has been tested and is working fine without breaking any existing functionality. I'm ok to merge it into the main branch.

Copy link
Collaborator

@saimedhi saimedhi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@ohltyler ohltyler merged commit a148ea5 into main Nov 27, 2024
17 of 19 checks passed
opensearch-trigger-bot bot pushed a commit that referenced this pull request Nov 27, 2024
Signed-off-by: Tyler Ohlsen <[email protected]>
(cherry picked from commit a148ea5)
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
ohltyler pushed a commit that referenced this pull request Nov 27, 2024
(cherry picked from commit a148ea5)

Signed-off-by: Tyler Ohlsen <[email protected]>
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[META] [FEATURE] Simplify model input configurations
2 participants