Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add cross-encoder model documentation #6357

Merged
merged 9 commits into from
Feb 16, 2024
Merged

Add cross-encoder model documentation #6357

merged 9 commits into from
Feb 16, 2024

Conversation

kolchfa-aws
Copy link
Collaborator

Closes #6352

Checklist

  • By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license and subject to the Developers Certificate of Origin.
    For more information on following Developer Certificate of Origin and signing off your commits, please check here.

@kolchfa-aws kolchfa-aws self-assigned this Feb 6, 2024
@kolchfa-aws
Model id format
7e9d05e 
Signed-off-by: Fanit Kolchina <[email protected]>
@hdhalter hdhalter added 2 - In progress Issue/PR: The issue or PR is in progress. release-notes PR: Include this PR in the automated release notes v2.12.0 labels Feb 7, 2024
@kolchfa-aws kolchfa-aws mentioned this pull request Feb 13, 2024
Merged
1 task
@hdhalter hdhalter added 3 - Tech review PR: Tech review in progress and removed 2 - In progress Issue/PR: The issue or PR is in progress. labels Feb 13, 2024
HenryL27
HenryL27 approved these changes Feb 15, 2024
View reviewed changes
Copy link
Contributor

@HenryL27 HenryL27 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Couple tweaks, but otherwise lgtm!

_ml-commons-plugin/custom-local-models.md Outdated

Cross-encoder models support query reranking.

To register a cross-encoder model, send a request in the following format. The `model_config` object is optinoal. Cross-encoder models' `function_name` is `TEXT_SIMILARITY`. For example, the following request registers a `ms-marco-TinyBERT-L-2-v2` model:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

optinoal -> optional

_search-plugins/search-relevance/reranking-search-results.md Outdated
@@ -13,7 +13,7 @@ Introduced 2.12
You can rerank search results using a cross-encoder reranker in order to improve search relevance. To implement reranking, you need to configure a [search pipeline]({{site.url}}{{site.baseurl}}/search-plugins/search-pipelines/index/) that runs at search time. The search pipeline intercepts search results and applies the [`rerank` processor]({{site.url}}{{site.baseurl}}/search-plugins/search-pipelines/rerank-processor/) to them. The `rerank` processor evaluates the search results and sorts them based on the new scores provided by the cross-encoder model.

**PREREQUISITE**<br>
Before using hybrid search, you must set up a cross-encoder model. For more information, see [Choosing a model]({{site.url}}{{site.baseurl}}/ml-commons-plugin/integrating-ml-models/#choosing-a-model).
Before using hybrid search, you must set up a cross-encoder model. For more information, see [Cross-encoder models]({{site.url}}{{site.baseurl}}/ml-commons-plugin/custom-local-models/#cross-encoder-models).
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Prerequisite for reranking, not hybrid search.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you! Good catch.

vagimeli
vagimeli approved these changes Feb 16, 2024
View reviewed changes
Copy link
Contributor

@vagimeli vagimeli left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, with minimal edits.

_ml-commons-plugin/custom-local-models.md Outdated

Cross-encoder models support query reranking.

To register a cross-encoder model, send a request in the following format. The `model_config` object is optinoal. Cross-encoder models' `function_name` is `TEXT_SIMILARITY`. For example, the following request registers a `ms-marco-TinyBERT-L-2-v2` model:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
To register a cross-encoder model, send a request in the following format. The `model_config` object is optinoal. Cross-encoder models' `function_name` is `TEXT_SIMILARITY`. For example, the following request registers a `ms-marco-TinyBERT-L-2-v2` model:
To register a cross-encoder model, send a request in the following format. The `model_config` object is optional. The cross-encoder model's `function_name` is `TEXT_SIMILARITY`. For example, the following request registers a `ms-marco-TinyBERT-L-2-v2` model:

dhrubo-os
dhrubo-os reviewed Feb 16, 2024
View reviewed changes
_ml-commons-plugin/custom-local-models.md Outdated
"embedding_dimension": 1,
"framework_type": "huggingface_transformers",
"total_chunks":2,
"all_config": "{\"total_chunks\":2,\"is_hidden\":false}"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

\"is_hidden\":false is unnecessary here.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

otherwise looks good.

@kolchfa-aws
Copy link
Collaborator Author

Thank you for the quick review, @HenryL27, @dhrubo-os, and @vagimeli!

natebower
natebower reviewed Feb 16, 2024
View reviewed changes
Copy link
Collaborator

@natebower natebower left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@kolchfa-aws Just a few changes. Thanks!

_ml-commons-plugin/custom-local-models.md Outdated Show resolved Hide resolved
_ml-commons-plugin/custom-local-models.md Outdated
}
```

Higher document score means higher similarity. In the preceding response, documents are scored as follows against the query text `today is sunny`:
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Either "A higher document score" or "Higher document scores"

_ml-commons-plugin/pretrained-models.md Outdated
| `amazon/neural-sparse/opensearch-neural-sparse-tokenizer-v1` | 1.0.1 | Yes | - [model_url](https://artifacts.opensearch.org/models/ml-models/amazon/neural-sparse/opensearch-neural-sparse-tokenizer-v1/1.0.1/torch_script/neural-sparse_opensearch-neural-sparse-tokenizer-v1-1.0.1-torch_script.zip)<br>- [config_url](https://artifacts.opensearch.org/models/ml-models/amazon/neural-sparse/opensearch-neural-sparse-tokenizer-v1/1.0.1/torch_script/config.json) | A neural sparse tokenizer model. The model tokenizes text into tokens and assigns each token a predefined weight, which is the token's IDF (if the IDF file is not provided, the weight defaults to 1). For more information, see [Preparing a model]({{site.url}}{{site.baseurl}}/ml-commons-plugin/custom-local-models/#preparing-a-model). |
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please define IDF.

@kolchfa-aws @natebower
Apply suggestions from code review
4d94879 
Co-authored-by: Nathan Bower <[email protected]>
Signed-off-by: kolchfa-aws <[email protected]>
@kolchfa-aws kolchfa-aws merged commit e76ec7c into main Feb 16, 2024
4 checks passed
oeyh pushed a commit to oeyh/documentation-website that referenced this pull request Mar 14, 2024
@kolchfa-aws @natebower @oeyh
Add cross-encoder model documentation (opensearch-project#6357)
1b7cd2c 
* Add cross-ranking model documentation

Signed-off-by: Fanit Kolchina <[email protected]>

* Model id format

Signed-off-by: Fanit Kolchina <[email protected]>

* Move to custom models

Signed-off-by: Fanit Kolchina <[email protected]>

* Update _search-plugins/search-relevance/reranking-search-results.md

Signed-off-by: kolchfa-aws <[email protected]>

* Update _ml-commons-plugin/custom-local-models.md

Signed-off-by: kolchfa-aws <[email protected]>

* Tech review and doc review comments

Signed-off-by: Fanit Kolchina <[email protected]>

* Apply suggestions from code review

Co-authored-by: Nathan Bower <[email protected]>
Signed-off-by: kolchfa-aws <[email protected]>

* Update _ml-commons-plugin/pretrained-models.md

Signed-off-by: kolchfa-aws <[email protected]>

---------

Signed-off-by: Fanit Kolchina <[email protected]>
Signed-off-by: kolchfa-aws <[email protected]>
Co-authored-by: Nathan Bower <[email protected]>
@kolchfa-aws kolchfa-aws deleted the new-model branch March 28, 2024 21:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@natebower natebower natebower left review comments

@dhrubo-os dhrubo-os dhrubo-os approved these changes

@vagimeli vagimeli vagimeli approved these changes

@HenryL27 HenryL27 HenryL27 approved these changes

@hdhalter hdhalter Awaiting requested review from hdhalter

@Naarcha-AWS Naarcha-AWS Awaiting requested review from Naarcha-AWS Naarcha-AWS is a code owner

@AMoo-Miki AMoo-Miki Awaiting requested review from AMoo-Miki AMoo-Miki is a code owner

@dlvenable dlvenable Awaiting requested review from dlvenable dlvenable is a code owner

Assignees

@kolchfa-aws kolchfa-aws

Labels
3 - Tech review PR: Tech review in progress release-notes PR: Include this PR in the automated release notes v2.12.0
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

[DOC] Add cross-encoder model
6 participants
@kolchfa-aws @HenryL27 @natebower @vagimeli @dhrubo-os @hdhalter