You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
The current indexer uses the full set of vectors to generate the trees. The advantage is that the split planes are very representative of the actual vectors we have in the database. The const is that it takes a lot of memory, even if it is memory-mapped.
So, to reduce the memory used to index the vectors, we should only use a subset of the vectors to generate the tree and then add the other vectors, the ones not used to generate the planes, back into the tree. With or without refining the planes, I don't know, but probably without. Those vectors can probably be added by using incremental insertion.
The text was updated successfully, but these errors were encountered:
The current indexer uses the full set of vectors to generate the trees. The advantage is that the split planes are very representative of the actual vectors we have in the database. The const is that it takes a lot of memory, even if it is memory-mapped.
So, to reduce the memory used to index the vectors, we should only use a subset of the vectors to generate the tree and then add the other vectors, the ones not used to generate the planes, back into the tree. With or without refining the planes, I don't know, but probably without. Those vectors can probably be added by using incremental insertion.
The text was updated successfully, but these errors were encountered: