Our mission is to provide open-source tools that enable the effective deployment and use of LLMs and ML models in production settings for developers and MLOps/LLMOps. Our key projects include:
- Paddler: An open-source load balancer and reverse proxy optimized for servers running llama.cpp. Paddler ensures efficient request distribution with a stateful load balancer that monitors server slots and health, supporting dynamic scaling.
- LLMOps Handbook: (work in progress) Practical and advanced guide to LLMOps. It provides a solid understanding of large language models’ general concepts, deployment techniques, and software engineering practices.
- Resonance: A PHP framework for building IO-intensive web applications and services featuring AI capabilities, a built-in web server, and integration with llama.cpp. It leverages asynchronous PHP and Swoole. Learn more about Resonance Framework on the official website.
Join our community on Discord to collaborate and stay updated!
