AI Starter Kit

A minimal code base for creating AI apps to do Question Answering (QA) over PDF documents, completely locally.

This project is inspired by local-ai-stack. However, their stack is entirely javascript based, and I needed a python based backend, so decided to create this project.

ai-starter-kit-demo_3.mp4

Stack

• 🦙 Inference: Ollama
• 💻 VectorDB: Qdrant
• 🧠 LLM Orchestration: Langchain
• 🖼️ App logic: FastAPI

How to get started

1. Clone this repo:

git clone https://github.com/mallahyari/ai-starter-kit.git

2. Install backend dependencies:

cd backend/app
pip install -r requirements.txt

3. Install frontend dependencies:

cd frontend
npm install

4. Start the Qdrant vector database (you need Docker). See here for other options information:

docker run -p 6333:6333 qdrant/qdrant

5. Install Ollama Instructions are here

6. Run the FastApi server (from inside backend directory):

python app/main.py

7. Open a new terminal and start the React development server (from inside frontend):

npm start

Change Configurations

You can change configurations in .config file, such as the embedding model, chunk size, and chunk overlap. If you plan to use Qdrant Cloud, you can or you can create your own .env file and set necessary api keys.

Additional Use Cases

Although current app only support pdf files, it's very straightforward to add other types of files such as text files, etc. Also, you can easily add the open-ended chat in addition to QA over document use case.

Discussion and Contribution

If you have any comment or feedback, please don't hesitate to use the Discussions section and open a new topic. You can also reach out directly via Linkedin or Twitter