fastapi-streaming-example - perhaps creating a new thread can be avoided & instead use create_task

[ksachdeva]

Expected Behaviour

minimize or eliminate thread creation if the entire stack is async

Current Behaviour

multi-agent-orchestrator/examples/fast-api-streaming/main.py

Line 119 in da9563f

Thread(target=lambda: asyncio.run(start_generation(query, user_id, session_id, streamer_queue))).start()

Above line will spawn a new thread for every request.

Code snippet

https://github.com/awslabs/multi-agent-orchestrator/blob/da9563fe9e87343fcda8abe6303b1e1d4f36c943/examples/fast-api-streaming/main.py#L119

Possible Solution

What would be an issue if you do the following -

asyncio.create_task(start_generation(query, user_id, session_id, streamer_queue))

instead of spawning a new thread

Steps to Reproduce

See -

multi-agent-orchestrator/examples/fast-api-streaming/main.py

Line 119 in da9563f

Thread(target=lambda: asyncio.run(start_generation(query, user_id, session_id, streamer_queue))).start()

[brnaba-aws]

Hi @ksachdeva
Thanks for you contribution.
This is a code example, just to show dev how they might use it. People are free to take the code and modify it the way they want.

Thanks,
Anthony