Adding support for FAISS JNI #285

luyuncheng · 2020-12-25T07:51:55Z

*As Issue #225 mentioned. It is possible to support faiss as future.

But i do not implement the codec services, because i want to ask for your opinions about this part.

I prefer to use the same interface(saveIndex, init, queryIndex) as same as nmslib's code with the same input parameter, this would i have more compatibility for knn plugin. so i PR this using the same JNI interface.
i think with the same interface we can support nmslib and faiss in knn plugin at the same time.

Description of changes:

This PR is support Faiss index and Query with the same interface in jni code.
Add index.knn.knnEngine to distinguish faiss and nmslib.
When Write an Index in DocValuesConsumer, using index.knn.knnEngine to load the explicit Library.
When Query an Index in KNNWeigh, using index.knn.knnEngine to load the explicit Library.
In KNNJNIFaissTests tests, it can do the same logic as KNNIndex and KNNJNITests
It almost PASS all integTest when using Faiss as Library
We can modified interface code as less as mush.
To build this, we need to build faiss firstly which depends on some external Libs like OpenMP,BLAS etc.

I think this PR, there is a lot of details need to be discussion.
PS. Is there any possible to open a faiss branch?

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

2. Right Now Only Support JNI

modified make without clang

modified OPENMP build in CMakeLists.txt

2. When Write index and query index, we use knnEngine to verify search in faiss or nmslib

Add index.knn.knnEngine in the ES Settings. When Write a Segment OR Query Index. It would use this settings to find the knn lib

2. almost integTest and unitest PASSED

Gradle build jni and jniFaiss

jmazanec15 · 2020-12-29T17:21:53Z

Hi @luyuncheng , thanks for making this PR! This is a very big change, so it will take some time to review. I will start looking at it in January.

jmazanec15

I think we will review the PR first and then check into a develop branch for further testing.

Maintaining the same interface makes sense.

I added a preliminary review. I will continue to review as I think more about it.

A couple comments:

For FAISS submodule, we should align it with a release. It appears to be aligned to an arbitrary commit
In general, I do not think it is a good idea to check a binary in (libfaiss.a). What is the reason for doing this?

I need to think more on best way to support 2 libraries from the user perspective. I will get back on this.

jmazanec15 · 2021-01-07T20:25:34Z

jniFaiss/CMakeLists.txt

@@ -0,0 +1,144 @@
+cmake_minimum_required(VERSION 2.8)


Design question: Should we have separate JNI libraries for FAISS and nmslib, or should they be contained in one?

i am not sure which is better.
At this, i separate libraries just to elaborate this faiss engine can work with knn-plugin.
may be one jni interface can make jni code more simple to maintain

I feel having seperate JNI would be more cleaner and easy to abstract out the underlying business logic to dedicated files. I like the current approach.

I think having 2 separate libraries is okay.

jmazanec15 · 2021-01-07T20:37:01Z

src/main/java/com/amazon/opendistroforelasticsearch/knn/index/KNNIndexCache.java

@@ -151,12 +153,22 @@ private void onRemoval(RemovalNotification<String, KNNIndexCacheEntry> removalNo
     */
    public KNNIndex getIndex(String key, final String indexName) {
        try {
+            //TODO if Type Not consistent


What do you mean by this?

when cached an Index, should we verify which engine for this index, and then load the exactly library?

I see. I think we should add a different file extension for faiss graphs as opposed to nmslib graphs. Maybe .faiss_hnsw.

jmazanec15 · 2021-01-07T20:38:43Z

src/main/java/com/amazon/opendistroforelasticsearch/knn/index/KNNSettings.java

@@ -83,6 +85,7 @@
    /**
     * Default setting values
     */
+    public static final String INDEX_KNN_DEFAULT_ENGINE = "Faiss"; // nmslib, faiss


I think we should keep nmslib as the default engine.

Yes, you are right. this is just for integration test can cover the faiss engine code.

upstream master merge into local master

…us field create.

vamshin

Few questions

Does faiss support all the library functions nmslib supports. We are primarily focusing on
L2, cosine, L1, negative dot product. Accordingly we may need to have relevant validations and throw the exceptions with proper message.
Can we confirm From Elasticsearch point of view following work? Manual test is fine for now and ideally we would prefer to have integration tests.
1. A nmslib index should be possible to reindex into faiss index and vice versa
2. Forcemerge calls work on faiss index (recall should be about same)
3. snapshot/restore work

vamshin · 2021-01-12T02:20:10Z

build.gradle

+task buildJniLib() {
+    dependsOn cmakeJniLib
+    dependsOn buildJniNmsLib
+    dependsOn buildJniFaissLib
+}


You could remove dependsOn cmakeJniLib and just rely on dependsOn buildJniNmsLib dependsOn buildJniFaissLib as they internally depends on cmakeJniNmsLib and cmakeJniFaissLib

Thanks, I fixed it

vamshin · 2021-01-12T02:23:05Z

jniFaiss/include/com_amazon_opendistroforelasticsearch_knn_index_faiss_KNNFIndex.h

+ * Method:    saveIndex
+ * Signature: ([I[[FLjava/lang/String;[Ljava/lang/String;Ljava/lang/String;)V
+ */
+JNIEXPORT void JNICALL Java_com_amazon_opendistroforelasticsearch_knn_index_faiss_KNNFIndex_saveIndex


Is faiss library backward compatible?
We should have faiss library version in the function names to support backward compatibility issues if any arises in the future? You could refer nmslib apis.

Yes, your are right. i will add the version number into it.

vamshin · 2021-01-12T02:28:36Z

jniFaiss/src/com_amazon_opendistroforelasticsearch_knn_index_faiss_KNNFIndex.cpp

+ * When autoclose class do close, then delete the pointer
+ * Method GC pointer
+ */
+JNIEXPORT void JNICALL Java_com_amazon_opendistroforelasticsearch_knn_index_faiss_KNNFIndex_gc


We should run memory leak tests for faiss jni code. @jmazanec15 ran some memory leak tests for nmslib, we could probably ran it on the faiss branch once we create. Lets see if we can test it as part of this PR or atleast we can create an issue to track all TODOs later.

is there any tools to detect memory leak in JNI code?
when i submit the faiss jni code, i have been writed a demo with valgrind to detect memory leak. but i do not know how to detect it from java test code.

Good point. We used valgrind to detect memory leaks. Last time I remember we build C++ class to call the functions directly and then build the artifact for running valgrind. We could probably help here.

vamshin · 2021-01-12T02:37:48Z

src/main/java/com/amazon/opendistroforelasticsearch/knn/index/util/NmsLibVersion.java

@@ -25,9 +28,17 @@
        public String indexLibraryVersion() {
            return "KNNIndexV2_0_6";
        }
+    },
+    VFaiss("Faiss") {


Having Faiss part of NmsLibVersion class would add confusion. We should probably have dedicated enum class for Faiss versions? Intension of this class is to hold different versions of same library and refer them from this class incase we happen to maintain more than one version of the library because of compatability issues.

Nice idea! i'll do it

How about make
nmslib: KNNIndex_NMSLIB_V2_0_6,
faiss: KNNIndex_FAISS_V1_6_4
like this:

public enum NmsLibVersion { VNMSLIB_208("NMSLIB_208"){ @Override public String indexLibraryVersion() { return "KNNIndex_NMSLIB_V2_0_8"; } }, VFAISS_164("FAISS_164") { @Override public String indexLibraryVersion() { return "KNNIndex_FAISS_V1_6_4"; } }; }

or do you have any idea to distinguish the name between nmslib and faiss library.

We could have new enum class for FAISS that just holds FAISS versions. Example

public enum FAISSLibVersion { /** * Latest available faiss version */ V_164("164"){ @Override public String indexLibraryVersion() { return "KNNIndexV2_0_11"; } }; public static final FAISSLibVersion LATEST = V164; public String buildVersion; FAISSLibVersion(String buildVersion) { this.buildVersion = buildVersion; } /** * FAISS library version used by the KNN codec * @return name */ public abstract String indexLibraryVersion(); }

We could have new enum class for FAISS that just holds FAISS versions. Example

yes, it is great. i added FAISSLibVersion in the latest commits

src/test/java/com/amazon/opendistroforelasticsearch/knn/index/KNNJNIFaissTests.java

vamshin · 2021-01-12T02:50:02Z

src/main/java/com/amazon/opendistroforelasticsearch/knn/index/faiss/KNNFIndex.java

@@ -0,0 +1,138 @@
+package com.amazon.opendistroforelasticsearch.knn.index.faiss;


package name should have version information similar to nmslib.

in the latest commits, i add faiss version into package and function names.

vamshin · 2021-01-12T03:05:45Z

...com/amazon/opendistroforelasticsearch/knn/index/codec/KNN80Codec/KNN80DocValuesConsumer.java

+
+            if(knnEngine.contains(NmsLibVersion.VFaiss.getBuildVersion())) {
+                AccessController.doPrivileged(
+                        new PrivilegedAction<Void>() {
+                            public Void run() {
+                                KNNFIndex.saveIndex(pair.docs, pair.vectors, tempIndexPath, algoParams, spaceType);
+                                return null;
+                            }
+                        }
+                );
+            } else {
+                AccessController.doPrivileged(
+                        new PrivilegedAction<Void>() {
+                            public Void run() {
+                                KNNIndex.saveIndex(pair.docs, pair.vectors, tempIndexPath, algoParams, spaceType);
+                                return null;
+                            }
                        }
-                    }
-            );
+                );
+            }


How about we move if else check inside the AccessController.doPrivileged. Something like below example

AccessController.doPrivileged( new PrivilegedAction<Void>() { public Void run() { if(knnindex) KNNIndex.saveIndex(pair.docs, pair.vectors, tempIndexPath, algoParams, spaceType); if(faissindex) KNNFIndex.saveIndex(pair.docs, pair.vectors, tempIndexPath, algoParams, spaceType); return null; } ```

vamshin · 2021-01-12T03:21:30Z

src/main/java/com/amazon/opendistroforelasticsearch/knn/index/KNNWeight.java

-            );
+            final KNNQueryResult[] results;
+
+            if (fieldAttributes.containsValue(NmsLibVersion.V206.getBuildVersion())) {


How about we pull out the methods in KNNFIndex and KNNIndex to interface so that we could refer with one object to keep it more cleaner? Something like

KNNIndex index; if (fieldAttributes.containsValue(NmsLibVersion.V206.getBuildVersion())) { index = knnIndexCache.getIndex(indexPath.toString(), knnQuery.getIndexName()); } else { index = knnIndexCache.getFIndex(indexPath.toString(), knnQuery.getIndexName()); } results = index.queryIndex(knnQuery.getQueryVector(), knnQuery.getK());

this is may make the log of changes in KNNIndex. In KNNIndex class, it would load the specific library, if we want to pull out the methods getIndex, we need an abstraction of KNNIndex and verify the engine we are using.
Let me think about it how to do it.

How about we pull out the methods in KNNFIndex and KNNIndex to interface so that we could refer with one object to keep it more cleaner? Something like

in the latest commits, I used KNNIndex as abstract class. KNNFaissIndex and KNNNmsLibIndex extends the KNNINdex.
KNNIndexCacheEntry only save the KNNIndex.

if you have any other idea, pls let me know

vamshin · 2021-01-12T03:22:46Z

jniFaiss/src/com_amazon_opendistroforelasticsearch_knn_index_faiss_KNNFIndex.cpp

@@ -0,0 +1,272 @@
+#include "com_amazon_opendistroforelasticsearch_knn_index_faiss_KNNFIndex.h"


This class has core logic and would need some insights into way faiss library operates to review the code. I plan to visit this code once I have understanding of faiss interface.

vamshin · 2021-01-12T03:24:20Z

jniFaiss/CMakeLists.txt

@@ -0,0 +1,144 @@
+cmake_minimum_required(VERSION 2.8)


I feel having seperate JNI would be more cleaner and easy to abstract out the underlying business logic to dedicated files. I like the current approach.

update CMakeLists.txt

Fixed depends On in gradle

…x.cpp

…_dev_libversion

Add Faiss and nmslib LibVersion into knn plugin

1. Add NmsLib Version And Faiss Version into KNN Plugin

Update Upstream Master

2. Make Faiss Version into code 3. Update Hnswlib to 2.0.10 # Conflicts: # jni/CMakeLists.txt # jni/include/com_amazon_opendistroforelasticsearch_knn_index_nmslib_v208_KNNIndex.h # jni/include/com_amazon_opendistroforelasticsearch_knn_index_v2011_KNNIndex.h # jni/include/com_amazon_opendistroforelasticsearch_knn_index_v208_KNNIndex.h # jni/src/com_amazon_opendistroforelasticsearch_knn_index_nmslib_v208_KNNIndex.cpp # jni/src/com_amazon_opendistroforelasticsearch_knn_index_v2011_KNNIndex.cpp # jni/src/com_amazon_opendistroforelasticsearch_knn_index_v208_KNNIndex.cpp # src/main/java/com/amazon/opendistroforelasticsearch/knn/index/KNNIndexCache.java # src/main/java/com/amazon/opendistroforelasticsearch/knn/index/KNNIndexShard.java # src/main/java/com/amazon/opendistroforelasticsearch/knn/index/KNNWeight.java # src/main/java/com/amazon/opendistroforelasticsearch/knn/index/codec/KNN80Codec/KNN80DocValuesConsumer.java # src/main/java/com/amazon/opendistroforelasticsearch/knn/index/nmslib/v208/KNNIndex.java # src/main/java/com/amazon/opendistroforelasticsearch/knn/index/util/NmsLibVersion.java # src/main/java/com/amazon/opendistroforelasticsearch/knn/index/v2011/KNNIndex.java # src/main/java/com/amazon/opendistroforelasticsearch/knn/index/v208/KNNIndex.java # src/main/plugin-metadata/plugin-security.policy # src/test/java/com/amazon/opendistroforelasticsearch/knn/index/KNNIndexCacheTests.java # src/test/java/com/amazon/opendistroforelasticsearch/knn/index/KNNJNITests.java

jmazanec15

Hi @luyuncheng, could you remove the jniFaiss/external/libfaiss.a? We don't need to ship binaries.

Also, can you update FAISS library commit to their latest release? v1.6.5. Right now, it points to a more recent commit.

jmazanec15 · 2021-01-21T23:14:03Z

.github/workflows/CI.yml

@@ -4,6 +4,7 @@ on:
    branches:
      - master
      - opendistro-*
+      - faiss*


I dont think we need to add this branch to CI action, given that it is a development branch. Please remove. Once review finishes, we will check into faiss branch and develop/test from there. Then, once we are ready, we can create another PR to master.

ok，i'll fix it

.github/workflows/test-workflow.yml

jmazanec15 · 2021-01-22T18:12:47Z

jniFaiss/CMakeLists.txt

@@ -0,0 +1,144 @@
+cmake_minimum_required(VERSION 2.8)


I think having 2 separate libraries is okay.

jniFaiss/CMakeLists.txt

jmazanec15 · 2021-01-22T22:11:36Z

jniFaiss/src/com_amazon_opendistroforelasticsearch_knn_index_faiss_v164_KNNFIndex.cpp

+
+std::unordered_map<string, faiss::MetricType> mapMetric = {
+        {"l2", faiss::METRIC_L2},
+        {"innerproduct", faiss::METRIC_INNER_PRODUCT}


Doesn't FAISS also have cosine space?

In Doc Faiss-indexes , i added inner product, which means If we have normalized vectors, the inner product becomes cosine similarity.
And there is some issues talk about this: facebookresearch/faiss#593

Oh interesting. That makes sense.

jmazanec15 · 2021-01-22T22:18:10Z

jniFaiss/src/com_amazon_opendistroforelasticsearch_knn_index_faiss_v164_KNNFIndex.cpp

+            paramsList.push_back(rawString);
+
+            int M = 32;
+            if (sscanf(rawString, "M=%d", &M) == 1) {


Why do we need to sscanf here?

Why do we need to sscanf here?

In the KNN plugin we would make algoParam like: M=32, efSearch=20, As faiss index factory docs shows, if M=32 we need use string: HNSW32 for index Description. so i use sscanf to get M.

i think we need an adapter to match different knnEngine algorithms params. if you have good idea, pls let me know.

Ah got it. I think makes sense.

jmazanec15 · 2021-01-22T22:19:18Z

jniFaiss/src/com_amazon_opendistroforelasticsearch_knn_index_faiss_v164_KNNFIndex.cpp

+		//---- Do Index
+		//----- 1. Train
+        if(!indexWriter->is_trained) {
+			//TODO if we use like PQ, we have to train dataset


Yeah I was thinking about this a little bit. It seems like it might make sense to fall back to a flat index if we do not have enough points to train. But we can think about this later.

jmazanec15 · 2021-01-22T22:19:46Z

jniFaiss/src/com_amazon_opendistroforelasticsearch_knn_index_faiss_v164_KNNFIndex.cpp

+{
+	//set thread 1 cause ES has Search thread
+	//TODO make it different at search and write
+//	omp_set_num_threads(1);


Why is this commented out?

i am no sure is it appropriate to set threads as 1 or reuse the settings:
public static final String KNN_ALGO_PARAM_INDEX_THREAD_QTY = "knn.algo_param.index_thread_qty";

I think it probably makes sense to reuse KNN_ALGO_PARAM_INDEX_THREAD_QTY.

jmazanec15 · 2021-01-22T22:23:36Z

src/main/java/com/amazon/opendistroforelasticsearch/knn/index/KNNIndexCache.java

@@ -151,12 +153,22 @@ private void onRemoval(RemovalNotification<String, KNNIndexCacheEntry> removalNo
     */
    public KNNIndex getIndex(String key, final String indexName) {
        try {
+            //TODO if Type Not consistent


I see. I think we should add a different file extension for faiss graphs as opposed to nmslib graphs. Maybe .faiss_hnsw.

jmazanec15 · 2021-01-22T22:25:22Z

src/main/java/com/amazon/opendistroforelasticsearch/knn/index/KNNIndexCache.java

        private final String indexPathUrl;
        private final String esIndexName;
        private final WatcherHandle<FileWatcher> fileWatcherHandle;

        private KNNIndexCacheEntry(final KNNIndex knnIndex, final String indexPathUrl, final String esIndexName,
                                   final WatcherHandle<FileWatcher> fileWatcherHandle) {
            this.knnIndex = knnIndex;
+            this.knnFindex = null;


Seems to me that knnIndex and knnFindex should inherit from some kind of abstract class.

I see. I think we should add a different file extension for faiss graphs as opposed to nmslib graphs. Maybe .faiss_hnsw.

in the latest commit, i use knnEngine name in the index file names like:

String hnswFileName = String.format("%s_%s_%s%s", state.segmentInfo.name, knnEngine.getLatestBuildVersion(), field.name, KNNCodecUtil.HNSW_EXTENSION);

Seems to me that knnIndex and knnFindex should inherit from some kind of abstract class.

yes, you are right. let me think about how to abstract this.

Seems to me that knnIndex and knnFindex should inherit from some kind of abstract class.

I used KNNIndex as abstract class. KNNFaissIndex and KNNNmsLibIndex extends the KNNINdex.
KNNIndexCacheEntry only save the KNNIndex.

if you have any other idea, pls let me know

I think that makes sense. Thanks for making the change!

2. Remove no useful libfaiss.a 3. Update JniFaiss CmakeLists.txt

2. Remove no use comments

Add NmsLib Version And Faiss Version into KNN Plugin

2. Add KNNNmsLibIndex and KNNFaissIndex inherit the KNNIndex

2. Add KNNNmsLibIndex and KNNFaissIndex inherit the KNNIndex 3. Move JNITests into JNINmsLibTests 4. Add New JNITests which would mixed knnEngine, and test with exception

Make KNNIndex as abstract class

jmazanec15

Thanks for making this change @luyuncheng!

I think at this point we can check it into a development branch "faiss-support" and I will start working on it/testing it as well.

I created "faiss-support". Could you merge in all the changes to that branch so that we can merge this change in?

luyuncheng · 2021-01-26T05:56:10Z

Thanks for making this change @luyuncheng!

I think at this point we can check it into a development branch "faiss-support" and I will start working on it/testing it as well.

I created "faiss-support". Could you merge in all the changes to that branch so that we can merge this change in?

i have changed the base branch from master to faiss-support.
Thanks for reviewing this PR ! @jmazanec15 @vamshin
Let me know if you need anything else

vamshin

LGTM! to push to dedicate branch. Once again thanks for the great work @luyuncheng.

We can continue development/testing on top of this code in dedicated branch.

luyuncheng added 3 commits December 25, 2020 15:03

1. New Branch Support Faiss

1c3d373

2. Right Now Only Support JNI

1. add submmodule faiss

6066a65

1. External Submodule

5165e70

luyuncheng mentioned this pull request Dec 25, 2020

Feature: Support for FAISS JNI #286

Closed

luyuncheng and others added 7 commits December 25, 2020 16:04

Update CMakeLists.txt

fb10190

modified make without clang

Update CMakeLists.txt

54cf6a5

modified OPENMP build in CMakeLists.txt

1. add knnEngine as a Settings

0f4bdf0

2. When Write index and query index, we use knnEngine to verify search in faiss or nmslib

1. Modified some FIXME

ab8e950

Merge pull request #1 from luyuncheng/faiss_withIT

950e460

Add index.knn.knnEngine in the ES Settings. When Write a Segment OR Query Index. It would use this settings to find the knn lib

1. gradle build jni and jniFaiss

22ce605

2. almost integTest and unitest PASSED

Merge pull request #2 from luyuncheng/faiss_withIT

1a44194

Gradle build jni and jniFaiss

jmazanec15 reviewed Jan 7, 2021

View reviewed changes

luyuncheng and others added 8 commits January 11, 2021 18:29

Merge pull request #6 from opendistro-for-elasticsearch/master

3d13a7b

upstream master merge into local master

1. FIXED SpaceType Get From IndexSettings may be EMPTY when a anonymo…

6f7094b

…us field create.

Update CI.yml

d94881c

1. Add CI branches for faiss

3dfb714

1. Update Cmake Files

3834ea0

OFF GPU

a775a3d

OFF PYTHON

051a54b

delete COMPILER

8cd57a7

vamshin reviewed Jan 12, 2021

View reviewed changes

luyuncheng added 8 commits January 12, 2021 17:21

Update CMakeLists.txt

ff99c4c

update CMakeLists.txt

Update CMakeLists.txt

d3d9baa

Update build.gradle

afa1b5c

Fixed depends On in gradle

Update com_amazon_opendistroforelasticsearch_knn_index_faiss_KNNFInde…

ce389f4

…x.cpp

Update CMakeLists.txt

7165574

Merge branch 'faiss' into faiss_dev

3d1670d

Update KNNCircuitBreakerIT.java

3d0a2de

Update KNNCircuitBreakerIT.java

698fbea

luyuncheng and others added 12 commits January 20, 2021 20:48

Update CMakeLists.txt

ce141a3

Update CMakeLists.txt

652c329

Update KNNWeight.java

6672814

Update CMakeLists.txt

b08d2d9

1. Add NmsLib Version And Faiss Version into KNN Plugin

f4ebd79

Merge remote-tracking branch 'origin/faiss_dev_libversion' into faiss…

df4c218

…_dev_libversion

1. Add NmsLib Version And Faiss Version into KNN Plugin

1777bd1

Merge pull request #11 from luyuncheng/faiss_dev_libversion

576a8eb

Add Faiss and nmslib LibVersion into knn plugin

Add NmsLib Version And Faiss Version into KNN Plugin

d37a523

1. Add NmsLib Version And Faiss Version into KNN Plugin

Merge pull request #13 from opendistro-for-elasticsearch/master

7c83e70

Update Upstream Master

1. Add NmsLib Version And Faiss Version into KNN Plugin

dd179b5

jmazanec15 reviewed Jan 22, 2021

View reviewed changes

luyuncheng and others added 9 commits January 24, 2021 21:04

1. Update Submodule FAISS to new version

9056b85

2. Remove no useful libfaiss.a 3. Update JniFaiss CmakeLists.txt

1. Update Submodule FAISS to new version V1.6.5(HASH:88eabe9)

a33625c

1. Update Submodule FAISS to new version V1.6.5(HASH:88eabe9)

ee0a3fe

2. Remove no use comments

1. Remove nouse CI branches

d613095

Merge pull request #14 from luyuncheng/faiss_dev

7715563

Add NmsLib Version And Faiss Version into KNN Plugin

1. Fixed Indentation

82671ad

1. Make KNNIndex as abstract class

bea16d6

2. Add KNNNmsLibIndex and KNNFaissIndex inherit the KNNIndex

1. Make KNNIndex as abstract class

3dc4f1f

2. Add KNNNmsLibIndex and KNNFaissIndex inherit the KNNIndex 3. Move JNITests into JNINmsLibTests 4. Add New JNITests which would mixed knnEngine, and test with exception

Merge pull request #15 from luyuncheng/opendistro-faiss_dev

1207bac

Make KNNIndex as abstract class

jmazanec15 reviewed Jan 25, 2021

View reviewed changes

luyuncheng changed the base branch from master to faiss-support January 26, 2021 05:42

jmazanec15 approved these changes Jan 26, 2021

View reviewed changes

vamshin approved these changes Jan 26, 2021

View reviewed changes

jmazanec15 merged commit 2fa551c into opendistro-for-elasticsearch:faiss-support Jan 26, 2021

jmazanec15 mentioned this pull request Feb 10, 2021

Refactoring build structure for faiss-support #321

Closed

jmazanec15 mentioned this pull request Jul 22, 2021

[RFC] Support Facebook's faiss library as another Approximate k-NN engine opensearch-project/k-NN#70

Closed

		@@ -0,0 +1,138 @@
		package com.amazon.opendistroforelasticsearch.knn.index.faiss;

		@@ -0,0 +1,272 @@
		#include "com_amazon_opendistroforelasticsearch_knn_index_faiss_KNNFIndex.h"

Adding support for FAISS JNI #285

Adding support for FAISS JNI #285

Conversation

luyuncheng commented Dec 25, 2020 • edited Loading

jmazanec15 commented Dec 29, 2020

jmazanec15 left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

luyuncheng Jan 10, 2021 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

vamshin left a comment • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

vamshin Jan 12, 2021 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

vamshin Jan 12, 2021 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

vamshin Jan 21, 2021 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jmazanec15 left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jmazanec15 left a comment

Choose a reason for hiding this comment

luyuncheng commented Jan 26, 2021

vamshin left a comment

Choose a reason for hiding this comment

luyuncheng commented Dec 25, 2020 •

edited

Loading

luyuncheng Jan 10, 2021 •

edited

Loading

vamshin left a comment •

edited

Loading

vamshin Jan 12, 2021 •

edited

Loading

vamshin Jan 12, 2021 •

edited

Loading

vamshin Jan 21, 2021 •

edited

Loading