Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[DOC] Support for Lucene Byte Sized Vector #4344

Closed
1 of 4 tasks
naveentatikonda opened this issue Jun 15, 2023 · 1 comment · Fixed by #4475
Closed
1 of 4 tasks

[DOC] Support for Lucene Byte Sized Vector #4344

naveentatikonda opened this issue Jun 15, 2023 · 1 comment · Fixed by #4475
Assignees
Labels
3 - Done Issue is done/complete v2.9.0
Milestone

Comments

@naveentatikonda
Copy link
Member

What do you want to do?

  • Request a change to existing documentation
  • Add new documentation
  • Report a technical problem with the documentation
  • Other

Tell us about your request. Provide a summary of the request and all versions that are affected.
As of today, k-NN plugin only supports vectors of type float for each dimension which is 4 bytes. This is getting expensive in terms of storage especially for those use cases that requires ingestion on a large scale as we need to construct, load, save and search graphs which gets more and more costlier. There are few use cases where customers prefer to reduce memory footprints by trading off recall with a minimal loss.

Using the Lucene ByteVector feature, we can add support for Byte Sized Vector where each dimension of the vector is a byte integer with range [-128 to 127].

As of now, we are planning to support this feature from 2.9 and future versions of OpenSearch.

What other resources are available? Provide links to related issues, POCs, steps for testing, etc.
opensearch-project/k-NN#812

@naveentatikonda
Copy link
Member Author

Please add v2.9.0 label

@Naarcha-AWS Naarcha-AWS added 1 - Backlog Issue: The issue is unassigned or assigned but not started v2.9.0 and removed untriaged labels Jun 15, 2023
@hdhalter hdhalter added this to the v2.9 milestone Jun 15, 2023
@kolchfa-aws kolchfa-aws self-assigned this Jun 29, 2023
@kolchfa-aws kolchfa-aws added 3 - Tech review PR: Tech review in progress and removed 3 - Tech review PR: Tech review in progress labels Jul 3, 2023
@hdhalter hdhalter added 2 - In progress Issue/PR: The issue or PR is in progress. 3 - Done Issue is done/complete and removed 1 - Backlog Issue: The issue is unassigned or assigned but not started 2 - In progress Issue/PR: The issue or PR is in progress. labels Jul 11, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
3 - Done Issue is done/complete v2.9.0
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants