[FEATURE_REQUEST] Custom (OpenAI-compatible) Text Completion API

[tnunamak]

Have you searched for similar requests?

Yes

Is your feature request related to a problem? If so, please describe.

Compiling a complete prompt from chat messages is difficult. SillyTavern already does it well for Text Completion APIs, it would be nice if I could take advantage of that by connecting SillyTavern to my custom text completion API rather than a custom chat completion API.

Describe the solution you'd like

It is already possible to connect to a Text Generation Web UI Text Completion API, which is Open-AI compatible. It would be great if we could do the same (reuse some of that), but with a custom endpoint, including setting extra parameters and headers (like in the custom Chat Completion API configuration).

Describe alternatives you've considered

I tried using pretending that my custom API was a Text Generation WebUI API, and it kind of worked, but I couldn't set any additional parameters or headers.

Additional context

The details of my specific use case:

I'm am connecting SillyTavern to my custom API, which acts as a kind of middleware before passing the text generation request through to Text Generation WebUI. I can only connect to my custom API's chat completion API.

SillyTavern sends multiple system messages in its Chat Completion API requests:

Unfortunately, Text Generation WebUI only includes one system message in the compiled prompt, and drops the others (code). This basically ruins the quality of the responses.

Priority

Medium (Would be very useful)

Is this something you would be keen to implement?

None

[Cohee1207]

Duplicate #1627
TL;DR: use Default (ooba) type, bypass status check and a type-in model (if needed)

[Cohee1207]

For custom headers: https://github.com/SillyTavern/SillyTavern/blob/release/default/config.yaml#L43

[tnunamak]

@Cohee1207 thanks for that link. Is there a way to replicate Include Body Parameters via config.yaml?

[houmie]

@tnunamak have you found a way? I have the exact same issue. My local API requires an API key that I need to pass in via headers. But how? Thanks