talk2robots

Complete and extensible human to AI interface

What is talk2robots?

talk2robots is an extensible human to AI interface. It is a framework that allows you to easily create your own chatbot, voice assistant, or any other kind of AI interface. It is designed to be easy to use and to extend.

Try it

in Telegram:

in Slack:

📝 Table of contents

• General Features
• Telegram Features
• Slack Features
• Admin Features (via Telegram System bot)
• Development in GitHub Codespaces
  • Requirements
  • Start local development environment
• Deploy and enjoy
  • DigitalOcean Requirements
  • Stripe Requirements
• Architecture overview

General Features

• Telegram bot
• Slack integration
• Discord bot
• Multiple models to choose from (Grok, GPT-4o, GPT-4o-mini, Claude Sonnet 3.5, Claude Haiku 3, Llama3 70b, Llama3 16b)
• Voice/Audio/Video prompt (OpenAI Wisper)
• Cache persistent layer (Redis)
• System health check and analytics (DataDog)
• Business analytics integration (DataDog)
• Alarm system/notifications integration (Telegram System Bot, DataDog)
• Support/feedback system (likes/dislikes)
• Payment system integration (Stripe)
• Video/Audio transcription
• Video/Audio summary
• Voice response (OpenAI TTS)
• Threads, i.e. context awareness and memory (OpenAI Assistant Threads, Mongo DB persistent threads)
• Image recognition
• Image generation (OpenAI DALL-E)
• Document/PDF reading and reasoning
• Shared links processing and reasoning (youtube, docs, etc)
• Web search and reasoning (Perplexity)

Telegram Features

• Chat with state of art LLM models /chatgpt. The bot remembers the context of the conversation until you say /clear.
• Voice support, just send a voice message in any popular language
• /voicegpt for full voice experience, i.e. voice prompt and voice reply (with OpenAI TTS)
• /translate [language code] mode to translate messages to English or a language of your choice
• /grammar mode just to correct grammar
• /teacher mode to correct and explain grammar
• /transcribe voice/audio/video messages
• /summarize text/voice/audio/video messages
• Upgrade subscription /upgrade. Three subscription plans are available:
  • Free - limits to $0.10/month of AI usage (text and audio)
  • Basic - $9.99/month, limits to $9.99/month AI usage
• Cancel subscription /downgrade
• Get user status (usage, limits etc) /status
• Get /support
• View terms /terms
• pin language for transcription and voice recognition by adding 'language' parameter to a command, e.g. /transcribe hebrew. Useful when translation of transcripts is needed or when studying a foreign language.

While in groups context:

• the bot will only reply when mentioned (so commands should be suffixed with @gienjibot, e.g. /upgrade@gienjibot)
• in /transcribe and /grammar modes, the bot will react to all messages to either transcribe audio or correct grammar.

Telegram bot in action

gienji-telegram.mov

Slack Features

• Chat in DM, private and public channels and/or threads
• Chat /chatgpt. The bot remembers the context of the conversation until you say /clear.
• Use /grammar mode just to correct grammar, you can also add 👓 emoji to a message to get grammar correction reply
• Summarize message thread by adding 📝 emoji to a message
• Use /teacher mode to correct and explain grammar
• Upgrade subscription /upgrade, only free subscription is available at the moment:
  • Free - limits to $0.10 AI usage
• Cancel subscription /downgrade
• Get user status (usage, limits etc) /mystatus
• Get /support
• View terms /terms

Slack bot in action

slack-gienji-720.mov

Admin Features (via Telegram System bot)

• Get system status /status
• Ban/Unban user /banuser /unbanuser
• Reset Stripe subscription for a user /stripereset
• Reset usage for a user /usagereset
• Get user info /user
• Get users count /userscount
• Get users count for a subscription /usersforsubscription
• /sendmessagetoauser to send a message to a user
• /sendmessagetousers to send a message to users (e.g. to notify about maintenance or marketing)

🏗️ Architecture overview

graph LR
    subgraph "User"
    O[User]
    X[Admin]
    end

    subgraph "Client"
    T[Telegram]
    S[Slack]
    end

    Client <-.-> User
    Client <--> A[Backend]

    subgraph "DigitalOcean"
    A[Backend] --> F[Redis]
    A[Backend] --> B[MongoDB]
    end

    subgraph "Dependencies"
    D[OpenAI API]
    E[FireworksAI API]
    P[DataDog]
    N[Stripe]
    end

    A[Backend] <--> D[OpenAI API]
    A[Backend] <--> E[FireworksAI API]
    A[Backend] <--> I[ClaudeAI API]
    A[Backend] -.-> P[DataDog] 
    A[Backend] <-.-> N[Stripe]

Loading

⛏️ Development in GitHub Codespaces

Requirements

• Setup Telegram bot - https://core.telegram.org/bots#6-botfather and save in the repo/Codespace secrets as TELEGRAM_BOT_TOKEN
• Setup OpenAI API key - https://platform.openai.com/account/api-keys and save in the repo/Codespace secrets as OPENAI_API_KEY
• Setup FireworksAI API key - https://fireworks.ai/api-keys and save in the repo/Codespace secrets as FIREWORKS_API_KEY

Start local development environment

• Fork the repository

• Start a Codespace

• Make port 80 of the Codespace public, guide.

• Run the following command inside the Codespace:

  $ make start

• Try accessing backend

  $ curl http://localhost/health

🚀 Deploy and enjoy

DigitalOcean Requirements

• Setup DO CLI - https://docs.digitalocean.com/reference/doctl/how-to/install/
• Setup DigitalOcean PAT - https://docs.digitalocean.com/reference/api/create-personal-access-token/ and save in the repo secrets as DO_PAT

Stripe Requirements

• Setup Stripe CLI - https://stripe.com/docs/stripe-cli#install