Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Backport 2.x] Introduce a loading layer in NMSLIB. (#2185) #2220

Merged
merged 3 commits into from
Oct 21, 2024
Merged

[Backport 2.x] Introduce a loading layer in NMSLIB. (#2185) #2220

merged 3 commits into from
Oct 21, 2024

Conversation

0ctopus13prime
Copy link
Contributor

Backport 7cf45c8 from #2185

Introduce a loading layer in NMSLIB. (opensearch-project#2185)
5f39f57 
* Introduce a loading layer in NMSLIB.

Signed-off-by: Dooyong Kim <[email protected]>

* Added NMSLIB istream implementation.

Signed-off-by: Dooyong Kim <[email protected]>

* Fix integer overflow issue when passing read size for loading NMSLIB vector index.

Signed-off-by: Dooyong Kim <[email protected]>

* Added unit test for NMSLIB loading layer.

Signed-off-by: Dooyong Kim <[email protected]>

* Made a patch in NMSLIB to avoid frequently calling JNI for better loading index performance.

Signed-off-by: Dooyong Kim <[email protected]>

* Compliance constexpr function in C++11 having nullstatement.

Signed-off-by: Dooyong Kim <[email protected]>

---------

Signed-off-by: Dooyong Kim <[email protected]>
Co-authored-by: Dooyong Kim <[email protected]>
@jmazanec15
Copy link
Member

@0ctopus13prime can you fix CI/build?

@0ctopus13prime
Copy link
Contributor Author

0ctopus13prime commented Oct 18, 2024 •
edited
Loading

@jmazanec15 can you fix CI/build?

Sure, seems like it's a gradle issue from the error message.
All failed CI are failed to resolve a dependency of Apache hc package.
Could you share how I should deal with this in such case?

  in org.opensearch.knn.TestUtils (TestUtils.java:231)
/Users/runner/work/k-NN/k-NN/src/test/java/org/opensearch/knn/index/KNNCircuitBreakerIT.java:9: error: package org.apache.hc.core5.http.io.entity does not exist

import org.apache.hc.core5.http.io.entity.EntityUtils;

@0ctopus13prime
Copy link
Contributor Author

Found the root cause for failing.
In 2.x, we are using org.apache.http.util.EntityUtils. Will change import statements in the next commit.

-import org.apache.hc.core5.http.io.entity.EntityUtils;
+import org.apache.http.util.EntityUtils;

jmazanec15
jmazanec15 reviewed Oct 21, 2024
View reviewed changes
CHANGELOG.md Outdated
@@ -7,6 +7,7 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
## [Unreleased 3.0](https://github.com/opensearch-project/k-NN/compare/2.x...HEAD)
### Features
### Enhancements
* Introduce a loading layer in native engine [#2185](https://github.com/opensearch-project/k-NN/pull/2185)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should be in 2.x

jmazanec15
jmazanec15 reviewed Oct 21, 2024
View reviewed changes
Copy link
Member

@jmazanec15 jmazanec15 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minor comment on CHANGELOG - other than that looks good

jmazanec15
jmazanec15 approved these changes Oct 21, 2024
View reviewed changes
Copy link
Member

@jmazanec15 jmazanec15 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@shatejas shatejas merged commit 4a224e3 into opensearch-project:2.x Oct 21, 2024
95 checks passed
@0ctopus13prime 0ctopus13prime deleted the backport-2185-to-2.x branch October 21, 2024 22:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@shatejas shatejas shatejas approved these changes

@jmazanec15 jmazanec15 jmazanec15 approved these changes

@heemin32 heemin32 Awaiting requested review from heemin32 heemin32 is a code owner

@navneet1v navneet1v Awaiting requested review from navneet1v navneet1v is a code owner

@VijayanB VijayanB Awaiting requested review from VijayanB VijayanB is a code owner

@vamshin vamshin Awaiting requested review from vamshin vamshin is a code owner

@naveentatikonda naveentatikonda Awaiting requested review from naveentatikonda naveentatikonda is a code owner

@junqiu-lei junqiu-lei Awaiting requested review from junqiu-lei junqiu-lei is a code owner

@martin-gaievski martin-gaievski Awaiting requested review from martin-gaievski martin-gaievski is a code owner

@ryanbogan ryanbogan Awaiting requested review from ryanbogan ryanbogan is a code owner

@luyuncheng luyuncheng Awaiting requested review from luyuncheng luyuncheng is a code owner

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@0ctopus13prime @jmazanec15 @shatejas