Verba

The Golden RAGtriever

Welcome to Verba: The Golden RAGtriever, an open-source application designed to offer an end-to-end, streamlined, and user-friendly interface for Retrieval-Augmented Generation (RAG) out of the box. In just a few easy steps, explore your datasets and extract insights with ease, either locally with Ollama and Huggingface or through LLM providers such as Anthrophic, Cohere, and OpenAI.

pip install goldenverba

• Verba
  • 🎯 What Is Verba?
  • ✨ Features
• ✨ Getting Started with Verba
• 🔑 API Keys
  • Weaviate
  • Ollama
  • Unstructured
  • AssemblyAI
  • OpenAI
  • HuggingFace
  • Groq
• Quickstart: Deploy with pip
• Quickstart: Build from Source
• Quickstart: Deploy with Docker
• 💾 Verba Walkthrough
• 💖 Open Source Contribution
• 🚩 Known Issues
• ❔FAQ

What Is Verba?

Verba is a fully-customizable personal assistant utilizing Retrieval Augmented Generation (RAG) for querying and interacting with your data, either locally or deployed via cloud. Resolve questions around your documents, cross-reference multiple data points or gain insights from existing knowledge bases. Verba combines state-of-the-art RAG techniques with Weaviate's context-aware database. Choose between different RAG frameworks, data types, chunking & retrieving techniques, and LLM providers based on your individual use-case.

Open Source Spirit

Weaviate is proud to offer this open-source project for the community. While we strive to address issues as fast as we can, please understand that it may not be maintained with the same rigor as production software. We welcome and encourage community contributions to help keep it running smoothly. Your support in fixing open issues quickly is greatly appreciated.

Watch our newest Verba video here:

Feature Lists

🤖 Model Support	Implemented	Description
Ollama (e.g. Llama3)	✅	Local Embedding and Generation Models powered by Ollama
HuggingFace (e.g. MiniLMEmbedder)	✅	Local Embedding Models powered by HuggingFace
Cohere (e.g. Command R+)	✅	Embedding and Generation Models by Cohere
Anthrophic (e.g. Claude Sonnet)	✅	Embedding and Generation Models by Anthrophic
OpenAI (e.g. GPT4)	✅	Embedding and Generation Models by OpenAI
Groq (e.g. Llama3)	✅	Generation Models by Groq (LPU inference)

🤖 Embedding Support	Implemented	Description
Weaviate	✅	Embedding Models powered by Weaviate
Ollama	✅	Local Embedding Models powered by Ollama
SentenceTransformers	✅	Embedding Models powered by HuggingFace
Cohere	✅	Embedding Models by Cohere
VoyageAI	✅	Embedding Models by VoyageAI
OpenAI	✅	Embedding Models by OpenAI

📁 Data Support	Implemented	Description
UnstructuredIO	✅	Import Data through Unstructured
Firecrawl	✅	Scrape and Crawl URL through Firecrawl
PDF Ingestion	✅	Import PDF into Verba
GitHub & GitLab	✅	Import Files from Github and GitLab
CSV/XLSX Ingestion	✅	Import Table Data into Verba
.DOCX	✅	Import .docx files
Multi-Modal (using AssemblyAI)	✅	Import and Transcribe Audio through AssemblyAI

✨ RAG Features	Implemented	Description
Hybrid Search	✅	Semantic Search combined with Keyword Search
Autocomplete Suggestion	✅	Verba suggests autocompletion
Filtering	✅	Apply Filters (e.g. documents, document types etc.) before performing RAG
Customizable Metadata	✅	Free control over Metadata
Async Ingestion	✅	Ingest data asynchronously to speed up the process
Advanced Querying	planned ⏱️	Task Delegation Based on LLM Evaluation
Reranking	planned ⏱️	Rerank results based on context for improved results
RAG Evaluation	planned ⏱️	Interface for Evaluating RAG pipelines

🗡️ Chunking Techniques	Implemented	Description
Token	✅	Chunk by Token powered by spaCy
Sentence	✅	Chunk by Sentence powered by spaCy
Semantic	✅	Chunk and group by semantic sentence similarity
Recursive	✅	Recursively chunk data based on rules
HTML	✅	Chunk HTML files
Markdown	✅	Chunk Markdown files
Code	✅	Chunk Code files
JSON	✅	Chunk JSON files

🆒 Cool Bonus	Implemented	Description
Docker Support	✅	Verba is deployable via Docker
Customizable Frontend	✅	Verba's frontend is fully-customizable via the frontend
Vector Viewer	✅	Visualize your data in 3D

🤝 RAG Libraries	Implemented	Description
LangChain	✅	Implement LangChain RAG pipelines
Haystack	planned ⏱️	Implement Haystack RAG pipelines
LlamaIndex	planned ⏱️	Implement LlamaIndex RAG pipelines

Something is missing? Feel free to create a new issue or discussion with your idea!

---

Getting Started with Verba

You have three deployment options for Verba:

• Install via pip

pip install goldenverba

• Build from Source

git clone https://github.com/weaviate/Verba

pip install -e .

• Use Docker for Deployment

Prerequisites: If you're not using Docker, ensure that you have Python >=3.10.0 installed on your system.

git clone https://github.com/weaviate/Verba

docker compose --env-file <your-env-file> up -d --build

If you're unfamiliar with Python and Virtual Environments, please read the python tutorial guidelines.

API Keys

You can set all API keys in the Verba frontend, but to make your life easier, we can also prepare a .env file in which Verba will automatically look for the keys. Create a .env in the same directory you want to start Verba in. You can find an .env.example file in the goldenverba directory.

Make sure to only set environment variables you intend to use, environment variables with missing or incorrect values may lead to errors.

Below is a comprehensive list of the API keys and variables you may require:

Environment Variable	Value	Description
WEAVIATE_URL_VERBA	URL to your hosted Weaviate Cluster	Connect to your WCS Cluster
WEAVIATE_API_KEY_VERBA	API Credentials to your hosted Weaviate Cluster	Connect to your WCS Cluster
ANTHROPIC_API_KEY	Your Anthropic API Key	Get Access to Anthropic Models
OPENAI_API_KEY	Your OpenAI Key	Get Access to OpenAI Models
OPENAI_BASE_URL	URL to OpenAI instance	Models
COHERE_API_KEY	Your API Key	Get Access to Cohere Models
GROQ_API_KEY	Your Groq API Key	Get Access to Groq Models
OLLAMA_URL	URL to your Ollama instance (e.g. http://localhost:11434 )	Get Access to Ollama Models
UNSTRUCTURED_API_KEY	Your API Key	Get Access to Unstructured Data Ingestion
UNSTRUCTURED_API_URL	URL to Unstructured Instance	Get Access to Unstructured Data Ingestion
ASSEMBLYAI_API_KEY	Your API Key	Get Access to AssemblyAI Data Ingestion
GITHUB_TOKEN	Your GitHub Token	Get Access to Data Ingestion via GitHub
GITLAB_TOKEN	Your GitLab Token	Get Access to Data Ingestion via GitLab
FIRECRAWL_API_KEY	Your Firecrawl API Key	Get Access to Data Ingestion via Firecrawl
VOYAGE_API_KEY	Your VoyageAI API Key	Get Access to Embedding Models via VoyageAI
EMBEDDING_SERVICE_URL	URL to your Embedding Service Instance	Get Access to Embedding Models via Weaviate Embedding Service
EMBEDDING_SERVICE_KEY	Your Embedding Service Key	Get Access to Embedding Models via Weaviate Embedding Service

Weaviate

Verba provides flexibility in connecting to Weaviate instances based on your needs. You have three options:

1. Local Deployment: Use Weaviate Embedded which runs locally on your device (except Windows, choose the Docker/Cloud Deployment)
2. Docker Deployment: Choose this option when you're running Verba's Dockerfile.
3. Cloud Deployment: Use an existing Weaviate instance hosted on WCD to run Verba

🌩️ Weaviate Cloud Deployment (WCD)

If you prefer a cloud-based solution, Weaviate Cloud (WCD) offers a scalable, managed environment. Learn how to set up a cloud cluster and get the API keys by following the Weaviate Cluster Setup Guide.

🐳 Docker Deployment Another local alternative is deploying Weaviate using Docker. For more details, follow the How to install Verba with Docker section.

Ollama

Verba supports Ollama models. Download and Install Ollama on your device (https://ollama.com/download). Make sure to install your preferred LLM using ollama run <model>.

Tested with llama3, llama3:70b and mistral. The bigger models generally perform better, but need more computational power.

Make sure Ollama Server runs in the background and that you don't ingest documents with different ollama models since their vector dimension can vary that will lead to errors

You can verify that by running the following command

ollama run llama3

Unstructured

Verba supports importing documents through Unstructured IO (e.g plain text, .pdf, .csv, and more). To use them you need the UNSTRUCTURED_API_KEY and UNSTRUCTURED_API_URL environment variable. You can get it from Unstructured

UNSTRUCTURED_API_URL is set to https://api.unstructuredapp.io/general/v0/general by default

AssemblyAI

Verba supports importing documents through AssemblyAI (audio files or audio from video files). To use them you need the ASSEMBLYAI_API_KEY environment variable. You can get it from AssemblyAI

OpenAI

Verba supports OpenAI Models such as Ada, GPT3, and GPT4. To use them, you need to specify the OPENAI_API_KEY environment variable. You can get it from OpenAI

You can also add a OPENAI_BASE_URL to use proxies such as LiteLLM (https://github.com/BerriAI/litellm)

OPENAI_BASE_URL=YOUR-OPENAI_BASE_URL

HuggingFace

If you want to use the HuggingFace Features, make sure to install the correct Verba package. It will install required packages to use the local embedding models. Please note that on startup, Verba will automatically download and install embedding models when used.

pip install goldenverba[huggingface]

or

pip install `.[huggingface]`

If you're using Docker, modify the Dockerfile accordingly

Groq

To use Groq LPUs as generation engine, you need to get an API key from Groq.

Although you can provide it in the graphical interface when Verba is up, it is recommended to specify it as GROQ_API_KEY environment variable before you launch the application.
It will allow you to choose the generation model in an up-to-date available models list.

How to deploy with pip

Python >=3.10.0

1. (Very Important) Initialize a new Python Environment

python3 -m virtualenv venv

2. Install Verba

pip install goldenverba

3. Launch Verba

verba start

You can specify the --port and --host via flags

4. Access Verba

Visit localhost:8000

5. (Optional)Create .env file and add environment variables

How to build from Source

1. Clone the Verba repos

git clone https://github.com/weaviate/Verba.git

2. Initialize a new Python Environment

python3 -m virtualenv venv

3. Install Verba

pip install -e .

4. Launch Verba

verba start

You can specify the --port and --host via flags

5. Access Verba

Visit localhost:8000

6. (Optional) Create .env file and add environment variables

How to install Verba with Docker

Docker is a set of platform-as-a-service products that use OS-level virtualization to deliver software in packages called containers. To get started with deploying Verba using Docker, follow the steps below. If you need more detailed instructions on Docker usage, check out the Docker Curriculum.

0. Clone the Verba repos Ensure you have Git installed on your system. Then, open a terminal or command prompt and run the following command to clone the Verba repository:

git clone https://github.com/weaviate/Verba.git

1. Set necessary environment variables Make sure to set your required environment variables in the .env file. You can read more about how to set them up in the API Keys Section

2. Adjust the docker-compose file You can use the docker-compose.yml to add required environment variables under the verba service and can also adjust the Weaviate Docker settings to enable Authentification or change other settings of your database instance. You can read more about the Weaviate configuration in our docker-compose documentation

Please make sure to only add environment variables that you really need.

2. Deploy using Docker With Docker installed and the Verba repository cloned, navigate to the directory containing the Docker Compose file in your terminal or command prompt. Run the following command to start the Verba application in detached mode, which allows it to run in the background:

docker compose up -d

docker compose --env-file goldenverba/.env up -d --build

This command will download the necessary Docker images, create containers, and start Verba. Remember, Docker must be installed on your system to use this method. For installation instructions and more details about Docker, visit the official Docker documentation.

4. Access Verba

• You can access your local Weaviate instance at localhost:8080

• You can access the Verba frontend at localhost:8000

If you want your Docker Instance to install a specific version of Verba you can edit the Dockerfile and change the installation line.

RUN pip install -e '.'

Verba Walkthrough

Select your Deployment

The first screen you'll see is the deployment screen. Here you can select between Local, Docker, Weaviate Cloud, or Custom deployment. The Local deployment is using Weaviate Embedded under the hood, which initializes a Weaviate instance behind the scenes. The Docker deployment is using a separate Weaviate instance that is running inside the same Docker network. The Weaviate Cloud deployment is using a Weaviate instance that is hosted on Weaviate Cloud Services (WCS). The Custom deployment allows you to specify your own Weaviate instance URL, PORT, and API key.

Import Your Data

First thing you need to do is to add your data. You can do this by clicking on Import Data and selecting either Add Files, Add Directory, or Add URL tab. Here you can add all your files that you want to ingest. You can then configure every file individually by selecting the file and clicking on Overview or Configure tab.

Query Your Data

With Data imported, you can use the Chat page to ask any related questions. You will receive relevant chunks that are semantically relevant to your question and an answer generated by your choosen model. You can configure the RAG pipeline under the Config tab.

Open Source Contribution

Your contributions are always welcome! Feel free to contribute ideas, feedback, or create issues and bug reports if you find any! Before contributing, please read the Contribution Guide. Visit our Weaviate Community Forum if you need any help!

Project Architecture

You can learn more about Verba's architecture and implementation in its technical documentation and frontend documentation. It's recommended to have a look at them before making any contributions.

Known Issues

• Weaviate Embeeded currently not working on Windows yet
  • Will be fixed in future versions, until then please use the Docker or WCS Deployment

FAQ

• Is Verba Multi-Lingual?

  • This depends on your choosen Embedding and Generation Model whether they support multi-lingual data.

• Can I use my Ollama Server with the Verba Docker?

  • Yes, you can! Make sure the URL is set to: OLLAMA_URL=http://host.docker.internal:11434
  • If you're running on Linux, you might need to get the IP Gateway of the Ollama server: OLLAMA_URL="http://YOUR-IP-OF-OLLAMA:11434"

• How to clear Weaviate Embedded Storage?

  • You'll find the stored data here: ~/.local/share/weaviate

• How can I specify the port?

  • You can use the port and host flag verba start --port 9000 --host 0.0.0.0