Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Integrate MLCommons API via MLClient into Neural Search Plugin #7

Closed
1 task
navneet1v opened this issue Sep 30, 2022 · 0 comments
Closed
1 task

Integrate MLCommons API via MLClient into Neural Search Plugin #7

navneet1v opened this issue Sep 30, 2022 · 0 comments
Assignees
@navneet1v
Labels
Enhancements Increases software capabilities beyond original client specifications

Comments

@navneet1v
Copy link
Collaborator

Description

As part of Neural Search plugin both ingestion and Search will require the inference API to convert the text or query string into embeddings by calling the right model. This issue tracks the development of Embedding module that will provide the abstraction in neural search plugin to call ML APIs via ML Client.

Tasks

  • Create Accessors which uses ML Client to call the ML APIs(inference APIs).
@navneet1v navneet1v changed the title Integrate MLCommons API into Neural Search Plugin Integrate MLCommons API via MLClient into Neural Search Plugin Sep 30, 2022
@navneet1v navneet1v self-assigned this Sep 30, 2022
@navneet1v navneet1v mentioned this issue Oct 10, 2022
Merged
5 tasks
@navneet1v navneet1v added the Enhancements Increases software capabilities beyond original client specifications label Oct 10, 2022
zane-neo added a commit to zane-neo/neural-search that referenced this issue Oct 20, 2022
@zane-neo
# This is a combination of 14 commits.
0dab8b1 
# This is the 1st commit message:

Add text embedding processor to neural search

Signed-off-by: Zan Niu <[email protected]>

# The commit message opensearch-project#2 will be skipped:

# Code format
#
# Signed-off-by: Zan Niu <[email protected]>

# The commit message opensearch-project#3 will be skipped:

# Address review comments
#
# Signed-off-by: Zan Niu <[email protected]>

# The commit message opensearch-project#4 will be skipped:

# Add blocking text embedding method for pipeline processor
#
# Signed-off-by: Zan Niu <[email protected]>

# The commit message opensearch-project#5 will be skipped:

# Add BaseNeuralSearchIT and address other review comments
#
# Signed-off-by: Zan Niu <[email protected]>

# The commit message opensearch-project#6 will be skipped:

# Add BaseNeuralSearchIT and address other review comments
#
# Signed-off-by: Zan Niu <[email protected]>

# The commit message opensearch-project#7 will be skipped:

# Add BaseNeuralSearchIT and address other review comments
#
# Signed-off-by: Zan Niu <[email protected]>

# The commit message opensearch-project#8 will be skipped:

# Fix naming convention and IT function move to base
#
# Signed-off-by: Zan Niu <[email protected]>

# The commit message opensearch-project#9 will be skipped:

# Fix naming convention and IT function move to base
#
# Signed-off-by: Zan Niu <[email protected]>

# The commit message opensearch-project#10 will be skipped:

# Update src/main/java/org/opensearch/neuralsearch/ml/MLCommonsClientAccessor.java
#
# Co-authored-by: Navneet Verma <[email protected]>

# The commit message opensearch-project#11 will be skipped:

# Update src/main/java/org/opensearch/neuralsearch/processor/TextEmbeddingProcessor.java
#
# Co-authored-by: Navneet Verma <[email protected]>

# The commit message opensearch-project#12 will be skipped:

# Fix code review comments
#
# Signed-off-by: Zan Niu <[email protected]>

# The commit message opensearch-project#13 will be skipped:

# Fix text embedding processor NPE
#
# Signed-off-by: Zan Niu <[email protected]>

# The commit message opensearch-project#14 will be skipped:

# Remove jackson dependencies and fix tests with XCoontent
#
# Signed-off-by: Zan Niu <[email protected]>
zane-neo added a commit that referenced this issue Oct 20, 2022
@zane-neo
Add text embedding processor to neural search (#18)
799c402 
* # This is a combination of 14 commits.
# This is the 1st commit message:

Add text embedding processor to neural search

Signed-off-by: Zan Niu <[email protected]>

# The commit message #2 will be skipped:

# Code format
#
# Signed-off-by: Zan Niu <[email protected]>

# The commit message #3 will be skipped:

# Address review comments
#
# Signed-off-by: Zan Niu <[email protected]>

# The commit message #4 will be skipped:

# Add blocking text embedding method for pipeline processor
#
# Signed-off-by: Zan Niu <[email protected]>

# The commit message #5 will be skipped:

# Add BaseNeuralSearchIT and address other review comments
#
# Signed-off-by: Zan Niu <[email protected]>

# The commit message #6 will be skipped:

# Add BaseNeuralSearchIT and address other review comments
#
# Signed-off-by: Zan Niu <[email protected]>

# The commit message #7 will be skipped:

# Add BaseNeuralSearchIT and address other review comments
#
# Signed-off-by: Zan Niu <[email protected]>

# The commit message #8 will be skipped:

# Fix naming convention and IT function move to base
#
# Signed-off-by: Zan Niu <[email protected]>

# The commit message #9 will be skipped:

# Fix naming convention and IT function move to base
#
# Signed-off-by: Zan Niu <[email protected]>

# The commit message #10 will be skipped:

# Update src/main/java/org/opensearch/neuralsearch/ml/MLCommonsClientAccessor.java
#
# Co-authored-by: Navneet Verma <[email protected]>

# The commit message #11 will be skipped:

# Update src/main/java/org/opensearch/neuralsearch/processor/TextEmbeddingProcessor.java
#
# Co-authored-by: Navneet Verma <[email protected]>

# The commit message #12 will be skipped:

# Fix code review comments
#
# Signed-off-by: Zan Niu <[email protected]>

# The commit message #13 will be skipped:

# Fix text embedding processor NPE
#
# Signed-off-by: Zan Niu <[email protected]>

# The commit message #14 will be skipped:

# Remove jackson dependencies and fix tests with XCoontent
#
# Signed-off-by: Zan Niu <[email protected]>

* Add text embedding processor to neural search

Signed-off-by: Zan Niu <[email protected]>

* Remove unnecessary parameters in TextEmbeddingProcessor method

Signed-off-by: Zan Niu <[email protected]>

* Remove unnecessary empty string checks

Signed-off-by: Zan Niu <[email protected]>

* Add field max depth limit to prevent malicious attack

Signed-off-by: Zan Niu <[email protected]>

Signed-off-by: Zan Niu <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@navneet1v navneet1v

Labels
Enhancements Increases software capabilities beyond original client specifications
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@navneet1v