-
Notifications
You must be signed in to change notification settings - Fork 3.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Python SK: Simplify memory_store_base #684
Merged
lemillermicrosoft
merged 16 commits into
microsoft:main
from
awharrison-28:python/simplifiy_memory_base
Apr 28, 2023
Merged
Changes from all commits
Commits
Show all changes
16 commits
Select commit
Hold shift + click to select a range
8633711
end-to-end memory scenario runs with the current changes
awharrison-28 79cf518
added comments for volatile memory store
awharrison-28 fe4bb33
documented other memory files
awharrison-28 04ee52a
precommit checks
awharrison-28 963b445
remove missed, usused embedding_index_base.py
awharrison-28 f9cfc35
revert test change to memory e2e test
awharrison-28 ea97771
Merge branch 'microsoft:main' into python/simplifiy_memory_base
awharrison-28 ad48a33
Merge branch 'main' into python/simplifiy_memory_base
lemillermicrosoft 0c77e64
Update python/semantic_kernel/memory/memory_store_base.py
awharrison-28 bc27692
removed cancel token TODOs
awharrison-28 f36bbc3
Merge branch 'main' into python/simplifiy_memory_base
awharrison-28 b8d2204
Merge branch 'main' into python/simplifiy_memory_base
lemillermicrosoft a4cedcb
Merge branch 'main' into python/simplifiy_memory_base
awharrison-28 9fc4393
formatting
awharrison-28 e77eeb9
Merge branch 'main' into python/simplifiy_memory_base
awharrison-28 f0ccad5
Merge branch 'main' into python/simplifiy_memory_base
awharrison-28 File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
20 changes: 0 additions & 20 deletions
20
python/semantic_kernel/connectors/ai/embeddings/embedding_index_base.py
This file was deleted.
Oops, something went wrong.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,12 +1,79 @@ | ||
# Copyright (c) Microsoft. All rights reserved. | ||
|
||
from abc import ABC | ||
from abc import abstractmethod | ||
from typing import List, Tuple | ||
|
||
from semantic_kernel.connectors.ai.embeddings.embedding_index_base import ( | ||
EmbeddingIndexBase, | ||
) | ||
from semantic_kernel.memory.storage.data_store_base import DataStoreBase | ||
from numpy import ndarray | ||
|
||
from semantic_kernel.memory.memory_record import MemoryRecord | ||
|
||
class MemoryStoreBase(DataStoreBase, EmbeddingIndexBase, ABC): | ||
pass | ||
|
||
class MemoryStoreBase: | ||
@abstractmethod | ||
async def create_collection_async(self, collection_name: SystemError) -> None: | ||
pass | ||
|
||
@abstractmethod | ||
async def get_collections_async( | ||
self, | ||
) -> List[str]: | ||
pass | ||
|
||
@abstractmethod | ||
async def delete_collection_async(self, collection_name: str) -> None: | ||
pass | ||
|
||
@abstractmethod | ||
async def does_collection_exist_async(self, collection_name: str) -> bool: | ||
pass | ||
|
||
@abstractmethod | ||
async def upsert_async(self, collection_name: str, record: MemoryRecord) -> str: | ||
pass | ||
|
||
@abstractmethod | ||
async def upsert_batch_async( | ||
self, collection_name: str, records: List[MemoryRecord] | ||
) -> List[str]: | ||
pass | ||
|
||
@abstractmethod | ||
async def get_async( | ||
self, collection_name: str, key: str, with_embedding: bool | ||
) -> MemoryRecord: | ||
pass | ||
|
||
@abstractmethod | ||
async def get_batch_async( | ||
self, collection_name: str, keys: List[str], with_embeddings: bool | ||
) -> List[MemoryRecord]: | ||
pass | ||
|
||
@abstractmethod | ||
async def remove_async(self, collection_name: str, key: str) -> None: | ||
pass | ||
|
||
@abstractmethod | ||
async def remove_batch_async(self, collection_name: str, keys: List[str]) -> None: | ||
pass | ||
|
||
@abstractmethod | ||
async def get_nearest_matches_async( | ||
self, | ||
collection_name: str, | ||
embedding: ndarray, | ||
limit: int, | ||
min_relevance_score: float, | ||
with_embeddings: bool, | ||
) -> List[Tuple[MemoryRecord, float]]: | ||
pass | ||
|
||
@abstractmethod | ||
async def get_nearest_match_async( | ||
self, | ||
collection_name: str, | ||
embedding: ndarray, | ||
min_relevance_score: float, | ||
with_embedding: bool, | ||
) -> Tuple[MemoryRecord, float]: | ||
pass |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
numpy.typing has an NDArray typehint that lets you typehint the dtype too: https://numpy.org/doc/stable/reference/typing.html
it's relatively recent so you may need to verify it works with your numpy version ranges in your dependencies