OpenAI-api Streaming: closing http socket does not cancel generation

[IkariDevGIT]

Describe the bug

Streaming: closing http socket does not cancel generation like when the old api was around

Is there an existing issue for this?

• I have searched the existing issues

Reproduction

enable openai-api extension, use api with streaming, close socket, continues generating

Screenshot

No response

Logs

no logs

System Info

win 10
msi
3070
local
frontend: SillyTavern

[oobabooga]

Afaik, SSE is uni-directional so the server doesn't know about the client disconnection. But it is possible to stop the generation by posting to /v1/internal/stop-generation:

  curl -X 'POST' \
    -H "Content-Type: application/json" \
    -d '' \
    'https://127.0.0.1:5000/v1/internal/stop-generation'