Skip to content

Latest commit

 

History

History
150 lines (112 loc) · 5 KB

README.md

File metadata and controls

150 lines (112 loc) · 5 KB
Raw

Darkside

🚀 Welcome to Private-AI!

Private-AI is an innovative AI project designed for asking questions about your documents using powerful Large Language Models (LLMs). The unique feature? It works offline, ensuring 100% privacy with no data leaving your environment.

🌐 What does Private-AI offer?

  • High-level API: Abstracts the complexity of a Retrieval Augmented Generation (RAG) pipeline. Handles document ingestion, chat, and completions.

  • Low-level API: For advanced users to implement custom pipelines. Includes features like embeddings generation and contextual chunks retrieval.

🌟 Why Private-AI?

Privacy is the key motivator! Private-AI addresses concerns in data-sensitive domains like healthcare and legal, ensuring your data stays under your control.

🤖 installation

Private-Ai Installation Guide

  • Install Python 3.11 (or 3.12)

  • Using apt(Debian base linux like-kali,Ubantu etc. )

    sudo apt-get install python3.11
apt install python3.11-venv

  • Using pyenv:

    pyenv install 3.11
pyenv local 3.11

  • Install Poetry for dependency management.

   sudo apt install python3-poetry
  sudo apt install python3-pytest

Installation Whithout GPU:

  • Git clone Private-Ai repository:
git clone https://github.com/AryanVBW/Private-Ai
cd Private-Ai && \
python3.11 -m venv .venv && source .venv/bin/activate && \
pip install --upgrade pip poetry && poetry install --with ui,local && ./scripts/setup 
python3.11 -m private_gpt

Run of private Ai:

  • forRunAgain jutsGoTo Private Ai directoy anr run following comand:
  make run

👍👍All Done 👍👍

For GPU utilization and customization, follow the steps below:

  • For Private-Ai to run fully locally GPU acceleration is required (CPU execution is possible, but very slow)

clone repo

  • Git clone Private-Ai repository:
git clone https://github.com/AryanVBW/Private-Ai
cd Private-Ai

Dependencies Installation:

  • Install make (OSX: brew install make, Windows: choco install make).
  • Install dependencies: 
    poetry install --with ui

Local LLM Setup:

  • Install extra dependencies for local execution: 
    poetry install --with local
  • Use the setup script to download embedding and LLM models: 
    poetry run python scripts/setup

Finalize:

  • Installation of private Ai: 
    make

Verification and run :

  • Run make run or poetry run python -m private_gpt.
  • Open http://localhost:8001 to see Gradio UI with a mock LLM echoing input.

Customization:

  • Customize low-level parameters in private_gpt/components/llm/llm_component.py.
  • Configure LLM options in settings.yaml.

GPU Support:

  • OSX: Build llama.cpp with Metal support.

    CMAKE_ARGS="-DLLAMA_METAL=on" pip install --force-reinstall --no-cache-dir llama-cpp-python

  • Windows NVIDIA GPU: Install VS2022, CUDA toolkit, and run:

    $env:CMAKE_ARGS='-DLLAMA_CUBLAS=on'; poetry run pip install --force-reinstall --no-cache-dir llama-cpp-python

  • Linux NVIDIA GPU and Windows-WSL: Install CUDA toolkit and run:

    CMAKE_ARGS='-DLLAMA_CUBLAS=on' poetry run pip install --force-reinstall --no-cache-dir llama-cpp-python

Troubleshooting:

  • Check GPU support and dependencies for your platform.
  • For C++ compiler issues, follow troubleshooting steps.

Note: If any issues, retry in verbose mode with -vvv during installations.

Troubleshooting C++ Compiler:

  • Windows 10/11: Install Visual Studio 2022 and MinGW.
  • OSX: Ensure Xcode is installed or install clang/gcc with Homebrew.

🧩 Architecture Highlights:

  • FastAPI-Based API: Follows the OpenAI API standard, making it easy to integrate.

  • LlamaIndex Integration: Leverages LlamaIndex for the RAG pipeline, providing flexibility and extensibility.

  • Present and Future: Evolving into a gateway for generative AI models and primitives. Stay tuned for exciting new features!

💡 How to Contribute?

Contributions are welcome! Check the ProjectBoard for ideas. Ensure code quality with format and typing checks (run make check).

🤗Supporters:

Supported by Qdrant, Fern, and LlamaIndex. Influenced by projects like LangChain, GPT4All, LlamaCpp, Chroma, and SentenceTransformers.

👏 Thank you for contributing to the future of private and powerful AI with Private-AI! 📝 License: Apache-2.0

Copyright Notice

This is a modified version of PrivateGPT. All rights and licenses belong to the PrivateGPT team.

© 2023 PrivateGPT Developers. All rights reserved.