Python: Update Weaviate Memory Connector to new Text Search Design #6839
Assignees
Labels
Ignite
python
Pull requests for the Python Semantic Kernel
sk team issue
A tag to denote issues that where created by the Semantic Kernel team (i.e., not the community)
Comments
markwallace-microsoft
added
python
|
This issue is stale because it has been open for 90 days with no activity. |
github-actions
bot
added
the
stale
github-actions
bot
removed
the
stale
evchaki
added
the
sk team issue
Merged
4 tasks
github-merge-queue bot
pushed a commit
that referenced
this issue
Nov 15, 2024
…to have unnamed vectors (#9684) ### Motivation and Context <!-- Thank you for your contribution to the semantic-kernel repo! Please help reviewers and future users, providing the following information: 1. Why is this change required? 2. What problem does it solve? 3. What scenario does it contribute to? 4. If it fixes an open issue, please link to the issue here. --> This PR adds vector search to Weaviate. All three types of search are supported, however vectorizable_text_search depends on a setup outside SK. Also adds a parameter to the Weaviate Collection called 'named_vectors', default is True. When set to False it uses unnamed vectors instead of named. Because of this there is a slight difference in how vectors are represented to Weaviate which might be breaking. ### Description <!-- Describe your changes, the overall approach, the underlying design. These notes will help understanding how your code works. Thanks! --> The breaking change is that, vector were set to have the name of the content that it vectorized, for instance: With a datamodel like this (shortened some of the class names): ```python class DataModel: content: Annotated[str, Data Field(embedding_property_name="vector")] vector: Annotated[list[float], VectorField()] ``` This would be set as `DataObject(vector: {"content": [vector content]})`, while it will now set this to `DataObject(vector: {"vector": [vector content]})` Where DataObject is the Weaviate object used to insert items. Closes #6839 ### Contribution Checklist <!-- Before submitting this PR, please make sure: --> - [x] The code builds clean without any errors or warnings - [x] The PR follows the [SK Contribution Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md) and the [pre-submission formatting script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts) raises no violations - [x] All unit tests pass, and I have added new tests where possible - [ ] I didn't break anyone 😄
github-project-automation
bot
moved this from Sprint: In Review
to Sprint: Done
in Semantic Kernel
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Text Search ADR is here: #5799
Implement new abstractions
Unit tests
Integration tests
Document any limitations in the implementation
Implement a basic RAG samples
Work with Sofia on a new Blog post
Obsolete the old implementation
Create task to graduate the Text Search Service
The text was updated successfully, but these errors were encountered: