Create docs pages

.readthedocs.yaml

@@ -0,0 +1,19 @@
+# Read the Docs configuration file for MkDocs projects
+# See https://docs.readthedocs.io/en/stable/config-file/v2.html for details
+
+# Required
+version: 2
+
+# Set the version of Python and other tools you might need
+build:
+  os: ubuntu-22.04
+  tools:
+    python: "3.12"
+
+mkdocs:
+  configuration: mkdocs.yml
+
+# Optionally declare the Python requirements required to build your docs
+python:
+  install:
+  - requirements: docs/requirements.txt

README.md

@@ -300,9 +300,9 @@ While using MemGPT via the CLI (not Discord!) you can run various commands:
 /retry
   pops the last answer and tries to get another one
 /rethink <text>
-  will replace the inner dialog of the last assistant message with the <text> to help shaping the conversation 
+  will replace the inner dialog of the last assistant message with the <text> to help shaping the conversation
 /rewrite
-  will replace the last assistant answer with the given text to correct or force the answer 
+  will replace the last assistant answer with the given text to correct or force the answer
 /heartbeat
   send a heartbeat system message to the agent
 /memorywarning

docs/README.md

@@ -0,0 +1,15 @@
+# Building the docs 
+
+Run the following from the MemGPT directory. 
+
+1. Install requirements: 
+```
+pip install -r docs/requirements.txt
+```
+
+2. Serve docs: 
+```
+mkdocs serve
+```
+
+

docs/adding_wrappers.md

@@ -0,0 +1,92 @@
+## Adding support for new LLMs + improving performance
+
+!!! warning "MemGPT + local LLM failure cases"
+
+    When using open LLMs with MemGPT, **the main failure case will be your LLM outputting a string that cannot be understood by MemGPT**. MemGPT uses function calling to manage memory (eg `edit_core_memory(...)` and interact with the user (`send_message(...)`), so your LLM needs generate outputs that can be parsed into MemGPT function calls.
+
+### What is a "wrapper"?
+
+To support function calling with open LLMs for MemGPT, we utilize "wrapper" code that:
+
+1. turns `system` (the MemGPT instructions), `messages` (the MemGPT conversation window), and `functions` (the MemGPT function set) parameters from ChatCompletion into a single unified prompt string for your LLM
+2. turns the output string generated by your LLM back into a MemGPT function call
+
+Different LLMs are trained using different prompt formats (eg `#USER:` vs `<im_start>user` vs ...), and LLMs that are trained on function calling are often trained using different function call formats, so if you're getting poor performance, try experimenting with different prompt formats! We recommend starting with the prompt format (and function calling format) recommended in the HuggingFace model card, and experimenting from there.
+
+We currently only support a few prompt formats in this repo ([located here](https://github.com/cpacker/MemGPT/tree/main/memgpt/local_llm/llm_chat_completion_wrappers))! If you write a new parser, please open a PR and we'll merge it in.
+
+### Adding a new wrapper (change the prompt format + function parser)
+
+To make a new wrapper (for example, because you want to try a different prompt format), you just need to subclass `LLMChatCompletionWrapper`. Your new wrapper class needs to implement two functions:
+
+- One to go from ChatCompletion messages/functions schema to a prompt string
+- And one to go from raw LLM outputs to a ChatCompletion response
+
+```python
+class LLMChatCompletionWrapper(ABC):
+
+    @abstractmethod
+    def chat_completion_to_prompt(self, messages, functions):
+        """Go from ChatCompletion to a single prompt string"""
+        pass
+
+    @abstractmethod
+    def output_to_chat_completion_response(self, raw_llm_output):
+        """Turn the LLM output string into a ChatCompletion response"""
+        pass
+```
+
+You can follow our example wrappers ([located here](https://github.com/cpacker/MemGPT/tree/main/memgpt/local_llm/llm_chat_completion_wrappers)).
+
+
+### Example with [Airoboros](https://huggingface.co/jondurbin/airoboros-l2-70b-2.1) (llama2 finetune)
+
+To help you get started, we've implemented an example wrapper class for a popular llama2 model **finetuned on function calling** (Airoboros). We want MemGPT to run well on open models as much as you do, so we'll be actively updating this page with more examples. Additionally, we welcome contributions from the community! If you find an open LLM that works well with MemGPT, please open a PR with a model wrapper and we'll merge it ASAP.
+
+```python
+class Airoboros21Wrapper(LLMChatCompletionWrapper):
+    """Wrapper for Airoboros 70b v2.1: https://huggingface.co/jondurbin/airoboros-l2-70b-2.1"""
+
+    def chat_completion_to_prompt(self, messages, functions):
+        """
+        Examples for how airoboros expects its prompt inputs: https://huggingface.co/jondurbin/airoboros-l2-70b-2.1#prompt-format
+        Examples for how airoboros expects to see function schemas: https://huggingface.co/jondurbin/airoboros-l2-70b-2.1#agentfunction-calling
+        """
+
+    def output_to_chat_completion_response(self, raw_llm_output):
+        """Turn raw LLM output into a ChatCompletion style response with:
+        "message" = {
+            "role": "assistant",
+            "content": ...,
+            "function_call": {
+                "name": ...
+                "arguments": {
+                    "arg1": val1,
+                    ...
+                }
+            }
+        }
+        """
+```
+
+See full file [here](llm_chat_completion_wrappers/airoboros.py).
+
+---
+
+## Wrapper FAQ
+
+### Status of ChatCompletion w/ function calling and open LLMs
+
+MemGPT uses function calling to do memory management. With [OpenAI's ChatCompletion API](https://platform.openai.com/docs/api-reference/chat/), you can pass in a function schema in the `functions` keyword arg, and the API response will include a `function_call` field that includes the function name and the function arguments (generated JSON). How this works under the hood is your `functions` keyword is combined with the `messages` and `system` to form one big string input to the transformer, and the output of the transformer is parsed to extract the JSON function call.
+
+In the future, more open LLMs and LLM servers (that can host OpenAI-compatable ChatCompletion endpoints) may start including parsing code to do this automatically as standard practice. However, in the meantime, when you see a model that says it supports “function calling”, like Airoboros, it doesn't mean that you can just load Airoboros into a ChatCompletion-compatable endpoint like WebUI, and then use the same OpenAI API call and it'll just work.
+
+1. When a model page says it supports function calling, they probably mean that the model was finetuned on some function call data (not that you can just use ChatCompletion with functions out-of-the-box). Remember, LLMs are just string-in-string-out, so there are many ways to format the function call data. E.g. Airoboros formats the function schema in YAML style (see https://huggingface.co/jondurbin/airoboros-l2-70b-3.1.2#agentfunction-calling) and the output is in JSON style. To get this to work behind a ChatCompletion API, you still have to do the parsing from `functions` keyword arg (containing the schema) to the model's expected schema style in the prompt (YAML for Airoboros), and you have to run some code to extract the function call (JSON for Airoboros) and package it cleanly as a `function_call` field in the response.
+
+2. Partly because of how complex it is to support function calling, most (all?) of the community projects that do OpenAI ChatCompletion endpoints for arbitrary open LLMs do not support function calling, because if they did, they would need to write model-specific parsing code for each one.
+
+### What is this all this extra code for?
+
+Because of the poor state of function calling support in existing ChatCompletion API serving code, we instead provide a light wrapper on top of ChatCompletion that adds parsers to handle function calling support. These parsers need to be specific to the model you're using (or at least specific to the way it was trained on function calling). We hope that our example code will help the community add additional compatability of MemGPT with more function-calling LLMs - we will also add more model support as we test more models and find those that work well enough to run MemGPT's function set.
+
+To run the example of MemGPT with Airoboros, you'll need to host the model behind some LLM web server (for example [webui](https://github.com/oobabooga/text-generation-webui#starting-the-web-ui)). Then, all you need to do is point MemGPT to this API endpoint by setting the environment variables `OPENAI_API_BASE` and `BACKEND_TYPE`. Now, instead of calling ChatCompletion on OpenAI's API, MemGPT will use it's own ChatCompletion wrapper that parses the system, messages, and function arguments into a format that Airoboros has been finetuned on, and once Airoboros generates a string output, MemGPT will parse the response to extract a potential function call (knowing what we know about Airoboros expected function call output).

docs/cli_faq.md

@@ -0,0 +1,35 @@
+# Frequently asked questions
+
+## MemGPT CLI
+
+### How can I use MemGPT to chat with my docs?
+
+Check out our [chat with your docs example](../example_data) to get started.
+
+### How do I save a chat and continue it later?
+
+When you want to end a chat, run `/exit`, and MemGPT will save your current chat with your agent (make a note of the agent name, e.g. `agent_N`). Later, when you want to start a chat with that same agent, you can run `memgpt run --agent <NAME>`.
+
+## OpenAI-related
+
+### How do I get an OpenAI key?
+
+To get an OpenAI key, visit [https://platform.openai.com/](https://platform.openai.com/), and make an account.
+
+Then go to [https://platform.openai.com/account/api-keys](https://platform.openai.com/account/api-keys) to create an API key. API keys start with `sk-...`.
+
+### How can I get gpt-4 access?
+
+[https://help.openai.com/en/articles/7102672-how-can-i-access-gpt-4](https://help.openai.com/en/articles/7102672-how-can-i-access-gpt-4)
+
+### I already pay for ChatGPT, is this the same as GPT API?
+
+No, ChatGPT Plus is a separate product from the OpenAI API. Paying for ChatGPT Plus does not get you access to the OpenAI API, vice versa.
+
+### I don't want to use OpenAI, can I still use MemGPT?
+
+Yes, you can run MemGPT with your own LLMs. See our section on local LLMs for information on how to set them up with MemGPT.
+
+## Local LLMs
+
+### TODO

docs/config.md

@@ -0,0 +1,32 @@
+### Configuring the agent
+You can set agent defaults by running `memgpt configure`.
+
+The `memgpt run` command supports the following optional flags (if set, will override config defaults):
+
+* `--agent`: (str) Name of agent to create or to resume chatting with.
+* `--human`: (str) Name of the human to run the agent with.
+* `--persona`: (str) Name of agent persona to use.
+* `--model`: (str) LLM model to run [gpt-4, gpt-3.5].
+* `--preset`: (str) MemGPT preset to run agent with.
+* `--first`: (str) Allow user to sent the first message.
+* `--debug`: (bool) Show debug logs (default=False)
+* `--no-verify`: (bool) Bypass message verification (default=False)
+* `--yes`/`-y`: (bool) Skip confirmation prompt and use defaults (default=False)
+
+
+### Adding Custom Personas/Humans
+You can add new human or persona definitions either by providing a file (using the `-f` flag) or text (using the `--text` flag).
+```sh
+# add a human
+memgpt add human [--name <NAME>] [-f <FILENAME>] [--text <TEXT>]
+
+# add a persona
+memgpt add persona [--name <NAME>] [-f <FILENAME>] [--text <TEXT>]
+```
+
+You can view available persona and human files with the following command:
+```sh
+memgpt list [humans/personas]
+```
+
+### Custom Presets

docs/contributing.md

@@ -0,0 +1,23 @@
+# Contributing
+
+## Installing from source
+First, install Poetry using [the official instructions here](https://python-poetry.org/docs/#installing-with-the-official-installer).
+
+Then, you can install MemGPT from source with:
+```sh
+git clone [email protected]:cpacker/MemGPT.git
+poetry shell
+poetry install
+```
+We recommend installing pre-commit to ensure proper formatting during development:
+```sh
+pip install pre-commit
+pre-commit install
+pre-commit run --all-files
+```
+
+### Formatting
+We welcome pull requests! Please run the formatter before submitting a pull request:
+```sh
+poetry run black . -l 140
+```

docs/data_sources.md

@@ -0,0 +1,86 @@
+## Loading External Data
+MemGPT supports pre-loading data into archival memory. In order to made data accessible to your agent, you must load data in with `memgpt load`, then attach the data source to your agent. You can configure where archival memory is stored by configuring the [storage backend](storage.md).
+
+### Viewing available data sources
+You can view available data sources with: 
+```
+memgpt list sources
+```
+```
++----------------+----------+----------+
+|      Name      | Location | Agents   |
++----------------+----------+----------+
+| short-stories  |  local   |  agent_1 |
+|      arxiv     |  local   |          |
+|  memgpt-docs   |  local   |  agent_1 |
++----------------+----------+----------+
+```
+The `Agents` column indicates which agents have access to the data, while `Location` indicates what storage backend the data has been loaded into. 
+
+### Attaching data to agents
+Attaching a data source to your agent loads the data into your agent's archival memory to access. You can attach data to your agent in two ways:
+
+*[Option 1]* From the CLI, run: 
+```
+memgpt attach --agent <AGENT-NAME> --data-source <DATA-SOURCE-NAME>
+```
+
+*[Option 2]*  While chatting with the agent, enter the `/attach` command and select the data source
+```
+> Enter your message: /attach
+? Select data source (Use arrow keys)
+ » short-stories
+   arxiv
+   memgpt-docs
+```
+
+
+!!! note "Hint"
+    To encourage your agent to reference its archival memory, we recommend adding phrases like "_search your archival memory..._" for the best results.
+
+
+
+### Loading a file or directory
+You can load a file, list of files, or directry into MemGPT with the following command: 
+```sh
+memgpt load directory --name <NAME> \
+    [--input-dir <DIRECTORY>] [--input-files <FILE1> <FILE2>...] [--recursive]
+```
+
+
+### Loading a database dump 
+You can load database into MemGPT, either from a database dump or a database connection, with the following command: 
+```sh
+memgpt load database --name <NAME>  \
+    --query <QUERY> \ # Query to run on database to get data
+    --dump-path <PATH> \ # Path to dump file
+    --scheme <SCHEME> \ # Database scheme
+    --host <HOST> \ # Database host
+    --port <PORT> \ # Database port
+    --user <USER> \ # Database user
+    --password <PASSWORD> \ # Database password
+    --dbname <DB_NAME> # Database name
+```
+
+### Loading a vector database 
+If you already have a vector database containing passages and embeddings, you can load them into MemGPT by specifying the table name, database URI, and the columns containing the passage text and embeddings.  
+```sh
+memgpt load vector-database --name <NAME> \
+    --uri <URI> \ # Database URI
+    --table_name <TABLE-NAME> \ # Name of table containing data 
+    --text_column <TEXT-COL> \ # Name of column containing text
+    --embedding_column <EMBEDDING-COL> # Name of column containing embedding
+```
+Since embeddings are already provided, MemGPT will not re-compute the embeddings. 
+
+### Loading a LlamaIndex dump 
+If you have a Llama Index `VectorIndex` which was saved to disk, you can load it into MemGPT by specifying the directory the index was saved to: 
+```sh
+memgpt load index --name <NAME> --dir <INDEX-DIR>
+```
+Since Llama Index will have already computing embeddings, MemGPT will not re-compute embeddings. 
+
+
+### Loading other types of data
+We highly encourage contributions for new data sources, which can be added as a new [CLI data load command](https://github.com/cpacker/MemGPT/blob/main/memgpt/cli/cli_load.py). We recommend checking for [Llama Index connectors](https://gpt-index.readthedocs.io/en/v0.6.3/how_to/data_connectors.html) that may support ingesting the data you're interested in loading. 
+

docs/discord_bot.md

@@ -0,0 +1,19 @@
+## Chatting with the MemGPT Discord Bot
+
+The fastest way to experience MemGPT is to chat with the MemGPT Discord Bot.
+
+Join <a href="https://discord.gg/9GEQrxmVyE">Discord</a></strong> and message the MemGPT bot (in the `#memgpt` channel). Then run the following commands (messaged to "MemGPT Bot"):
+* `/profile` (to create your profile)
+* `/key` (to enter your OpenAI key)
+* `/create` (to create a MemGPT chatbot)
+
+Make sure your privacy settings on this server are open so that MemGPT Bot can DM you: \
+MemGPT → Privacy Settings → Direct Messages set to ON
+<div align="center">
+ <img src="https://memgpt.ai/assets/img/discord/dm_settings.png" alt="set DMs settings on MemGPT server to be open in MemGPT so that MemGPT Bot can message you" width="400">
+</div>
+
+You can see the full list of available commands when you enter `/` into the message box.
+<div align="center">
+ <img src="https://memgpt.ai/assets/img/discord/slash_commands.png" alt="MemGPT Bot slash commands" width="400">
+</div>

docs/endpoints.md

@@ -0,0 +1,34 @@
+### LLM Backends
+
+You can use MemGPT with various LLM backends, including the OpenAI API, Azure OpenAI, and various local (or self-hosted) LLM backends.
+
+#### OpenAI
+To use MemGPT with an OpenAI API key, simply set the `OPENAI_API_KEY` variable:
+```sh
+export OPENAI_API_KEY=YOUR_API_KEY # on Linux/Mac
+set OPENAI_API_KEY=YOUR_API_KEY # on Windows
+$Env:OPENAI_API_KEY = "YOUR_API_KEY" # on Windows (PowerShell)
+```
+
+#### Azure
+To use MemGPT with Azure, expore the following variables and then re-run `memgpt configure`:
+```sh
+# see https://github.com/openai/openai-python#microsoft-azure-endpoints
+export AZURE_OPENAI_KEY = ...
+export AZURE_OPENAI_ENDPOINT = ...
+export AZURE_OPENAI_VERSION = ...
+
+# set the below if you are using deployment ids
+export AZURE_OPENAI_DEPLOYMENT = ...
+export AZURE_OPENAI_EMBEDDINGS_DEPLOYMENT = ...
+```
+
+Replace `export` with `set` or `$Env:` if you are on Windows (see the OpenAI example).
+
+Note: **your Azure endpoint must support functions** or you will get an error. See [this GitHub issue](https://github.com/cpacker/MemGPT/issues/91) for more information.
+
+#### Custom endpoints
+To use custom OpenAI endpoints, run `export OPENAI_API_BASE=<MY_CUSTOM_URL>` and then re-run `memgpt configure` to set the custom endpoint as the default endpoint.
+
+#### Local LLMs
+Setting up MemGPT to run with local LLMs requires a bit more setup, follow [the instructions here](../local_llm).