No API Documentation

[josiahbryan]

This was billed as "ollama compatible", but when I run ramallama serve -p 11434 llama3.2 - my client code that works with ollama does NOT work (posting to /api/chat returns 404, and I see the POST hit ramallama in the console as well)

Where's API documentation for the actual API served by ramallama? 🙏

[ericcurtin]

We are using upstream llama.cpp server by default:

https://github.com/ggerganov/llama.cpp/tree/master/examples/server

and it does say on that page:

looking for feedback and contributors

But there's also a "--runtime" flag, where the intent is switchable servers, vllm is one that we will support in future, but it is only implemented in --nocontainer mode right now so one must set up vllm themselves

[josiahbryan]

Good to know! Thank you for sharing that. Be really helpful to have a link to the API documentation on the serve page or mentioned in the readme or something because that was not at all obvious that I need to go look for llama.cpp rest documentation when I thought it would be just like the ollama API haha. Thank you for straightening me out on this one, you can close this issue. I really appreciate it

[rhatdan]

Care to open a PR to make this point in the README.md and potentially in the ramalama-serve.1.md file.

[josiahbryan]

Finally had a free minute to fork and PR - sorry it took so long. PR is up at: #274

[josiahbryan]

Thanks for merging the PR! I'll close this issue now. Cheers!