Java Binding for LLaMA.cpp

Introduction

This is the javacpp preset, which provides a complete java API for the LLaMA.cpp project. The main goal of LLaMA.cpp is to run LLMs (i.e., Large Language Models) on consumer-grade devices such as your Mac or PC.

Important

This project is still under active development, please do not use it directly in production. PRs are always welcome!

Build Instructions

Important

Currently this binding only works on Mac with Apple Silicon, support for other major platforms (i.e., Linux, Windows etc.) is yet to come very soon! Please stay tuned.

In the project root, run the following commands to build and install the related jars to your local machine:

mvn clean install --projects .,llamacpp
mvn clean install -f platform --projects ../llamacpp/platform -Djavacpp.platform.host

Then, in order to use LLaMA APIs in your project, you should add the following dependencies in your pom.xml:

<dependencies>
    <dependency>
        <groupId>org.javallm</groupId>
        <artifactId>llamacpp</artifactId>
        <version>b1266</version>
    </dependency>

    <dependency>
        <groupId>org.javallm</groupId>
        <artifactId>llamacpp-platform</artifactId>
        <version>b1266</version>
    </dependency>
</dependencies>

Acknowledgements

I really appreciate bytedeco for their awesome project javacpp. 🥳 JavaLLM is built upon on the giant javacpp's shoulder, please give them a generous star if you found JavaLLM useful!