LLM inference with Rust

Inspired by [karpathy/llama2.c][https://github.com/karpathy/llama2.c].

Usage

cargo build --release
./target/release/llm_rs

History

• [v0.0.1][https://github.com/zhihuij/llm.rs/tree/v0.0.1]: Initial version (48.19 token/s)
• [v0.0.2][https://github.com/zhihuij/llm.rs/tree/main]: Data Parallelism with Rayon (164.08 token/s)

License

This project is licensed under the [MIT license][https://github.com/zhihuij/llm.rs/LICENSE].