Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

🔧#149: Streaming Chat Workflow #150

Merged
merged 27 commits into from
Mar 2, 2024

Conversation

roma-glushko
Copy link
Member

@roma-glushko roma-glushko commented Feb 18, 2024

A PoC of the GEP0008 implementation.

Changes

  • Initing a new type of workflow, a streaming (async) routing workflow using the Streaming Chat API as an example
  • Updated the Bruno collection
  • Updated the LanguageModel API to include ChatStream() and SupportChatStream() methods
  • Get the streaming router working
  • Implemented SSE event parsing to be able to work with OpenAI streaming chat API
  • Integrated OpenAI chat streaming into the Glide's streaming chat API
  • Covered the happy workflow by tests

Follow-up Work

Demo

https://vimeo.com/916483448?share=copy

Ref

@roma-glushko roma-glushko self-assigned this Feb 18, 2024
@roma-glushko roma-glushko marked this pull request as draft February 18, 2024 17:03
Copy link

codecov bot commented Feb 18, 2024

Codecov Report

Attention: Patch coverage is 53.75375% with 154 lines in your changes are missing coverage. Please review.

Project coverage is 68.82%. Comparing base (e9e7246) to head (f50ef15).

Files Patch % Lines
pkg/providers/openai/chat_stream.go 63.71% 32 Missing and 9 partials ⚠️
pkg/routers/router.go 29.62% 33 Missing and 5 partials ⚠️
pkg/providers/openai/chat.go 62.12% 23 Missing and 2 partials ⚠️
pkg/providers/clients/sse.go 48.71% 17 Missing and 3 partials ⚠️
pkg/providers/bedrock/chat.go 0.00% 6 Missing ⚠️
pkg/providers/anthropic/chat_stream.go 0.00% 4 Missing ⚠️
pkg/providers/azureopenai/chat_stream.go 0.00% 4 Missing ⚠️
pkg/providers/bedrock/chat_stream.go 0.00% 4 Missing ⚠️
pkg/providers/cohere/chat_stream.go 0.00% 4 Missing ⚠️
pkg/providers/octoml/chat_stream.go 0.00% 4 Missing ⚠️
... and 1 more
Additional details and impacted files
@@             Coverage Diff             @@
##           develop     #150      +/-   ##
===========================================
- Coverage    70.73%   68.82%   -1.92%     
===========================================
  Files           40       50      +10     
  Lines         2006     2268     +262     
===========================================
+ Hits          1419     1561     +142     
- Misses         512      615     +103     
- Partials        75       92      +17     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@roma-glushko roma-glushko marked this pull request as ready for review March 2, 2024 22:10
@roma-glushko roma-glushko merged commit 4edb644 into develop Mar 2, 2024
7 of 8 checks passed
@roma-glushko roma-glushko deleted the 149-streaming-chat-api-workflow branch March 2, 2024 22:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Lang Chat] Implement a PoC of a async router with streaming lang API
1 participant