Support and document server-sent-events

[RobertLucian]

Motivation

This can be useful for APIs that want to stream results in real time, before returning from predict() (e.g. text generation models). This is an alternative to websockets if data only needs to be transferred to the client.

Additional context

This is blocked on #1637

Examples

These examples have been tested using quay.io/cortexlabsdev/python-predictor-cpu-async:0.23.0, which has #1637 merged in.

import asyncio
from sse_starlette.sse import EventSourceResponse

async def numbers(minimum, maximum):
    for i in range(minimum, maximum + 1):
        await asyncio.sleep(0.9)
        yield dict(data=i)

class PythonPredictor:
    def __init__(self, config):
        pass

    async def predict(self, payload):
        generator = numbers(1, 5)
        return EventSourceResponse(generator)

with a shell command:

import asyncio
from sse_starlette.sse import EventSourceResponse

async def run_command():
    command = "echo test1 && sleep 2 && echo test2 && sleep 2 && echo test3 && echo done"

    proc = await asyncio.create_subprocess_shell(
        command, stdout=asyncio.subprocess.PIPE, stderr=asyncio.subprocess.PIPE
    )

    while True:
        output = await proc.stdout.readline()
        if not output:
            return
        yield output.decode().strip()

class PythonPredictor:
    def __init__(self, config):
        pass

    async def predict(self, payload):
        return EventSourceResponse(run_command())