fix: Add schema name if dropping index in pgvector store #1277
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
|
anakin87
reviewed
Jan 7, 2025
Comment on lines
+397
to
+407
| index_exists_after_deletion = bool( | ||
| self._execute_sql( | ||
| "SELECT 1 FROM pg_indexes WHERE schemaname = %s AND tablename = %s AND indexname = %s", | ||
| (self.schema_name, self.table_name, self.hnsw_index_name), | ||
| "Could not check if HNSW index exists", | ||
| ).fetchone() | ||
| ) | ||
|
|
||
| if index_exists and index_exists_after_deletion: | ||
| error_message = "The HNSW was not dropped" | ||
| raise DocumentStoreError(error_message) |
There was a problem hiding this comment.
I would remove this check.
If the HNSW index is not dropped, the following invocation self._create_hnsw_index() will fail. Right?
|
|
||
|
|
||
| @pytest.mark.integration | ||
| def test_hnsw_index_recreation_in_new_schema(): |
There was a problem hiding this comment.
I think the test can be improved/changed.
I would probably first create a Document Store instance,
then create another one with hnsw_recreate_index_if_exists=True and verify in some way that the HNSW index is first dropped and then recreated.
I can help with the implementation, if needed.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Related Issues
Proposed Changes:
In pgvector if HNSW index should drop, only index_name was used, but not schema_name. So if there were multiple schemas, the index was not dropped and could not be recreated.
How did you test it?
The check was added directly to the code, so if the index existed and should be dropped, the Error is raised, because I consider it as critical Error.
Notes for the reviewer
I tested this on multiple machines with similar setups, and the behavior was not consistent. On one PC this fix was needed, but on the other PC this fix was not needed, and it was working even without the fix, but I was not able to discover the difference...
But I believe that the fix is correct, and should be applied.
Checklist
fix:,feat:,build:,chore:,ci:,docs:,style:,refactor:,perf:,test:.