Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

FalkorDB graph database integration #1880

Open
wants to merge 11 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 6 additions & 5 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -160,7 +160,7 @@ history = m.history(memory_id=<memory_id_1>)

### Graph Memory
To initialize Graph Memory you'll need to set up your configuration with graph store providers.
Currently, we support Neo4j as a graph store provider. You can setup [Neo4j](https://neo4j.com/) locally or use the hosted [Neo4j AuraDB](https://neo4j.com/product/auradb/).
Currently, we support FalkorDB and Neo4j as a graph store providers. You can set up [FalkorDB](https://www.falkordb.com/) or [Neo4j](https://neo4j.com/) locally or use the hosted [FalkorDB Cloud](https://app.falkordb.cloud/) or [Neo4j AuraDB](https://neo4j.com/product/auradb/).
Moreover, you also need to set the version to `v1.1` (*prior versions are not supported*).
Here's how you can do it:

Expand All @@ -169,11 +169,12 @@ from mem0 import Memory

config = {
"graph_store": {
"provider": "neo4j",
"provider": "falkordb",
"config": {
"url": "neo4j+s://xxx",
"username": "neo4j",
"password": "xxx"
"host": "---"
"username": "---",
"password": "---",
"port": "---"
}
},
"version": "v1.1"
Expand Down
40 changes: 40 additions & 0 deletions cookbooks/mem0_graph_memory.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
# This example shows how to use graph config to use falkordb graph databese
import os
from mem0 import Memory
from dotenv import load_dotenv

# Loading OpenAI API Key
load_dotenv()
OPENAI_API_KEY = os.environ.get('OPENAI_API_KEY')
USER_ID = "test"

# Creating the config dict from the environment variables
config = {
"llm": { # This is the language model configuration, use your carditionals
"provider": "openai",
"config": {
"model": "gpt-4o-mini",
"temperature": 0
}
},
"graph_store": { # See https://app.falkordb.cloud/ for the carditionals
"provider": "falkordb",
"config": {
"host": os.environ['HOST'],
"username": os.environ['USERNAME'],
"password": os.environ['PASSWORD'],
"port": os.environ['PORT']
}
},
"version": "v1.1"
}

# Create the memory class using from config
memory = Memory.from_config(config_dict=config)

# Use the Mem0 to add and search memories
memory.add("I like painting", user_id=USER_ID)
memory.add("I hate playing badminton", user_id=USER_ID)
print(memory.get_all(user_id=USER_ID))
memory.add("My friend name is john and john has a dog named tommy", user_id=USER_ID)
print(memory.search("What I like to do", user_id=USER_ID))
10 changes: 6 additions & 4 deletions docs/open-source/graph_memory/features.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -19,11 +19,13 @@ from mem0 import Memory

config = {
"graph_store": {
"provider": "neo4j",
"provider": "falkordb",
"config": {
"url": "neo4j+s://xxx",
"username": "neo4j",
"password": "xxx"
"Database": "falkordb",
"host": "---"
"username": "---",
"password": "---",
"port": "---"
},
"custom_prompt": "Please only extract entities containing sports related relationships and nothing else.",
},
Expand Down
25 changes: 14 additions & 11 deletions docs/open-source/graph_memory/overview.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -36,12 +36,11 @@ allowfullscreen
## Initialize Graph Memory

To initialize Graph Memory you'll need to set up your configuration with graph store providers.
Currently, we support Neo4j as a graph store provider. You can setup [Neo4j](https://neo4j.com/) locally or use the hosted [Neo4j AuraDB](https://neo4j.com/product/auradb/).
Moreover, you also need to set the version to `v1.1` (*prior versions are not supported*).
Currently, we support FalkorDB and Neo4j as a graph store providers. You can set up [FalkorDB](https://www.falkordb.com/) or [Neo4j](https://neo4j.com/) locally or use the hosted [FalkorDB Cloud](https://app.falkordb.cloud/) or [Neo4j AuraDB](https://neo4j.com/product/auradb/).
Moreover, you also need to set the version to `v1.1` (*prior versions are not supported*).

<Note>If you are using Neo4j locally, then you need to install [APOC plugins](https://neo4j.com/labs/apoc/4.1/installation/).</Note>


User can also customize the LLM for Graph Memory from the [Supported LLM list](https://docs.mem0.ai/components/llms/overview) with three levels of configuration:

1. **Main Configuration**: If `llm` is set in the main config, it will be used for all graph operations.
Expand All @@ -57,11 +56,13 @@ from mem0 import Memory

config = {
"graph_store": {
"provider": "neo4j",
"provider": "falkordb",
"config": {
"url": "neo4j+s://xxx",
"username": "neo4j",
"password": "xxx"
"Database": "falkordb",
"host": "---"
"username": "---",
"password": "---",
"port": "---"
}
},
"version": "v1.1"
Expand All @@ -83,11 +84,13 @@ config = {
}
},
"graph_store": {
"provider": "neo4j",
"provider": "falkordb",
"config": {
"url": "neo4j+s://xxx",
"username": "neo4j",
"password": "xxx"
"Database": "falkordb",
"host": "---"
"username": "---",
"password": "---",
"port": "---"
},
"llm" : {
"provider": "openai",
Expand Down
10 changes: 6 additions & 4 deletions docs/open-source/quickstart.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -63,11 +63,13 @@ from mem0 import Memory

config = {
"graph_store": {
"provider": "neo4j",
"provider": "falkordb",
"config": {
"url": "neo4j+s://---",
"username": "neo4j",
"password": "---"
"Database": "falkordb",
"host": "---"
"username": "---",
"password": "---",
"port": "---"
}
},
"version": "v1.1"
Expand Down
32 changes: 29 additions & 3 deletions mem0/graphs/configs.py
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
from typing import Optional
from typing import Optional, Union

from pydantic import BaseModel, Field, field_validator, model_validator

Expand All @@ -20,11 +20,31 @@ def check_host_port_or_path(cls, values):
if not url or not username or not password:
raise ValueError("Please provide 'url', 'username' and 'password'.")
return values

class FalkorDBConfig(BaseModel):
host: Optional[str] = Field(None, description="Host address for the graph database")
username: Optional[str] = Field(None, description="Username for the graph database")
password: Optional[str] = Field(None, description="Password for the graph database")
port: Optional[int] = Field(None, description="Port for the graph database")
# Default database name is mandatory in langchain
database: str = "_default_"

@model_validator(mode="before")
def check_host_port_or_path(cls, values):
host, port = (
values.get("host"),
values.get("port"),
)
if not host or not port:
raise ValueError(
"Please provide 'host' and 'port'."
)
return values


class GraphStoreConfig(BaseModel):
provider: str = Field(description="Provider of the data store (e.g., 'neo4j')", default="neo4j")
config: Neo4jConfig = Field(description="Configuration for the specific data store", default=None)
provider: str = Field(description="Provider of the data store (e.g., 'falkordb', 'neo4j')", default="falkordb")
config: Union[FalkorDBConfig, Neo4jConfig] = Field(description="Configuration for the specific data store", default=None)
llm: Optional[LlmConfig] = Field(description="LLM configuration for querying the graph store", default=None)
custom_prompt: Optional[str] = Field(
description="Custom prompt to fetch entities from the given text", default=None
Expand All @@ -35,5 +55,11 @@ def validate_config(cls, v, values):
provider = values.data.get("provider")
if provider == "neo4j":
return Neo4jConfig(**v.model_dump())
elif provider == "falkordb":
config = v.model_dump()
# In case the user try to use diffrent database name
config["database"] = "_default_"

return FalkorDBConfig(**config)
else:
raise ValueError(f"Unsupported graph store provider: {provider}")
46 changes: 46 additions & 0 deletions mem0/graphs/utils.py
Original file line number Diff line number Diff line change
Expand Up @@ -53,6 +53,52 @@

Adhere strictly to these guidelines to ensure high-quality knowledge graph extraction."""

FALKORDB_QUERY = """
MATCH (n)
WHERE n.embedding IS NOT NULL AND n.user_id = $user_id
WITH n,
reduce(dot = 0.0, i IN range(0, size(n.embedding)-1) | dot + n.embedding[i] * $n_embedding[i]) /
(sqrt(reduce(l2 = 0.0, i IN range(0, size(n.embedding)-1) | l2 + n.embedding[i] * n.embedding[i])) *
sqrt(reduce(l2 = 0.0, i IN range(0, size($n_embedding)-1) | l2 + $n_embedding[i] * $n_embedding[i]))) AS similarity
WHERE similarity >= $threshold
MATCH (n)-[r]->(m)
RETURN n.name AS source, Id(n) AS source_id, type(r) AS relation, Id(r) AS relation_id, m.name AS destination, Id(m) AS destination_id, similarity
UNION
MATCH (n)
WHERE n.embedding IS NOT NULL AND n.user_id = $user_id
WITH n,
reduce(dot = 0.0, i IN range(0, size(n.embedding)-1) | dot + n.embedding[i] * $n_embedding[i]) /
(sqrt(reduce(l2 = 0.0, i IN range(0, size(n.embedding)-1) | l2 + n.embedding[i] * n.embedding[i])) *
sqrt(reduce(l2 = 0.0, i IN range(0, size($n_embedding)-1) | l2 + $n_embedding[i] * $n_embedding[i]))) AS similarity
WHERE similarity >= $threshold
MATCH (m)-[r]->(n)
RETURN m.name AS source, Id(m) AS source_id, type(r) AS relation, Id(r) AS relation_id, n.name AS destination, Id(n) AS destination_id, similarity
ORDER BY similarity DESC
"""


NEO4J_QUERY = """
MATCH (n)
WHERE n.embedding IS NOT NULL AND n.user_id = $user_id
WITH n,
round(reduce(dot = 0.0, i IN range(0, size(n.embedding)-1) | dot + n.embedding[i] * $n_embedding[i]) /
(sqrt(reduce(l2 = 0.0, i IN range(0, size(n.embedding)-1) | l2 + n.embedding[i] * n.embedding[i])) *
sqrt(reduce(l2 = 0.0, i IN range(0, size($n_embedding)-1) | l2 + $n_embedding[i] * $n_embedding[i]))), 4) AS similarity
WHERE similarity >= $threshold
MATCH (n)-[r]->(m)
RETURN n.name AS source, elementId(n) AS source_id, type(r) AS relation, elementId(r) AS relation_id, m.name AS destination, elementId(m) AS destination_id, similarity
UNION
MATCH (n)
WHERE n.embedding IS NOT NULL AND n.user_id = $user_id
WITH n,
round(reduce(dot = 0.0, i IN range(0, size(n.embedding)-1) | dot + n.embedding[i] * $n_embedding[i]) /
(sqrt(reduce(l2 = 0.0, i IN range(0, size(n.embedding)-1) | l2 + n.embedding[i] * n.embedding[i])) *
sqrt(reduce(l2 = 0.0, i IN range(0, size($n_embedding)-1) | l2 + $n_embedding[i] * $n_embedding[i]))), 4) AS similarity
WHERE similarity >= $threshold
MATCH (m)-[r]->(n)
RETURN m.name AS source, elementId(m) AS source_id, type(r) AS relation, elementId(r) AS relation_id, n.name AS destination, elementId(n) AS destination_id, similarity
ORDER BY similarity DESC
"""

def get_update_memory_prompt(existing_memories, memory, template):
return template.format(existing_memories=existing_memories, memory=memory)
Expand Down
Loading
Loading