Python SK: Simplify memory_store_base #684
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
awharrison-28
added
enhancement
python
awharrison-28
requested review from
dluc,
mkarle,
tawalke,
alexchaomander and
lemillermicrosoft
This was referenced Apr 26, 2023
lemillermicrosoft
previously approved these changes
Apr 27, 2023
Co-authored-by: Lee Miller <[email protected]>
shawncal
reviewed
Apr 27, 2023
alexchaomander
previously approved these changes
Apr 27, 2023
github-actions
bot
added
the
kernel
awharrison-28
requested review from
shawncal,
lemillermicrosoft and
alexchaomander
alexchaomander
previously approved these changes
Apr 28, 2023
lemillermicrosoft
added
the
PR: feedback to address
|
Looks like there are some lint failures to address. |
github-actions
bot
removed
the
kernel
lemillermicrosoft
approved these changes
Apr 28, 2023
dluc
pushed a commit
that referenced
this pull request
Apr 29, 2023
### Motivation and Context This PR significantly reduces the complexity of implementing a new memory store. Previously, memory stores needed to implement EmbeddingIndexBase, DataStoreBase, and MemoryStoreBase. Now, memory stores only need to implement MemoryStoreBase. This PR also brings MemoryStoreBase to parity with the .NET IMemoryStore interface. ### Description - Removed `data_entry.py`, `data_store_base.py`, `volatile_data_store.py`, and `embedding_index_base.py` - Added `with_embedding` parameter to all get/search methods. Passing the embedding between the application and storage is not optional and defaults to False - `MemoryStoreBase` (Python SK) == `IMemoryStore` (.NET SK) - Updated `volatile_memory_store` to implement `MemoryStoreBase` - Removed `ABC` from `MemoryStoreBase` - Added documentation to memory files Closes #670
daxpryce
reviewed
May 2, 2023
| @@ -12,6 +14,7 @@ class MemoryQueryResult: | |||
| description: Optional[str] | |||
| text: Optional[str] | |||
| relevance: float | |||
| embedding: Optional[ndarray] | |||
There was a problem hiding this comment.
numpy.typing has an NDArray typehint that lets you typehint the dtype too: https://numpy.org/doc/stable/reference/typing.html
it's relatively recent so you may need to verify it works with your numpy version ranges in your dependencies
dehoward
pushed a commit
to lemillermicrosoft/semantic-kernel
that referenced
this pull request
Jun 1, 2023
### Motivation and Context This PR significantly reduces the complexity of implementing a new memory store. Previously, memory stores needed to implement EmbeddingIndexBase, DataStoreBase, and MemoryStoreBase. Now, memory stores only need to implement MemoryStoreBase. This PR also brings MemoryStoreBase to parity with the .NET IMemoryStore interface. ### Description - Removed `data_entry.py`, `data_store_base.py`, `volatile_data_store.py`, and `embedding_index_base.py` - Added `with_embedding` parameter to all get/search methods. Passing the embedding between the application and storage is not optional and defaults to False - `MemoryStoreBase` (Python SK) == `IMemoryStore` (.NET SK) - Updated `volatile_memory_store` to implement `MemoryStoreBase` - Removed `ABC` from `MemoryStoreBase` - Added documentation to memory files Closes microsoft#670
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Motivation and Context
This PR significantly reduces the complexity of implementing a new memory store. Previously, memory stores needed to implement EmbeddingIndexBase, DataStoreBase, and MemoryStoreBase. Now, memory stores only need to implement MemoryStoreBase. This PR also brings MemoryStoreBase to parity with the .NET IMemoryStore interface.
Description
data_entry.py,data_store_base.py,volatile_data_store.py, andembedding_index_base.pywith_embeddingparameter to all get/search methods. Passing the embedding between the application and storage is not optional and defaults to FalseMemoryStoreBase(Python SK) ==IMemoryStore(.NET SK)volatile_memory_storeto implementMemoryStoreBaseABCfromMemoryStoreBaseCloses #670
Contribution Checklist
dotnet format