🔧 Set router context for stream response chunks #195
Comments
roma-glushko
added a commit
that referenced
this issue
Apr 15, 2024
roma-glushko
added a commit
that referenced
this issue
Apr 15, 2024
…nd error message types & removed all unneeded context from stream objects & defined a set of possible error codes
roma-glushko
added a commit
that referenced
this issue
Apr 16, 2024
…n case of some errors (#203) - Passed RouterID and ModelID information in the chat stream messages - Introduced a new ChatStreamMessage type that joins both chunk and error messages. Removed unneeded context from provider chatStream structs - defined a set of possible error codes during chat streaming - started simplifying logging by using context-based loggers - Introduced finish_reason on the error schema
Merged
roma-glushko
added a commit
that referenced
this issue
Apr 16, 2024
Final major improvements to streaming chat workflow. Fixed issues with Cohere streaming chat. Expanded and revisited Cohere params in config. ## Changelog ### Added - 🔧 #195 #196: Set router ctx in stream chunks & handle end of stream in case of some errors (@roma-glushko) - 🐛🔧 #197: Handle max_tokens & content_filtered finish reasons across OpenAI, Azure and Cohere (@roma-glushko) ### Changed - 🔧 💥 #198: Expose more Cohere params & fixing validation of provider params in config (breaking change) (@roma-glushko) - 🔧 #186: Rendering Durations in a human-friendly way (@roma-glushko) ### Fixed - 🐛 #209: Embed Swagger specs into binary to fix panics caused by missing swagger.yaml file (@roma-glushko) - 🐛 #200: Implemented a custom json per line stream reader to read Cohere chat streams correctly (@roma-glushko)
Merged
roma-glushko
added a commit
that referenced
this issue
Apr 16, 2024
## Summary ✨ Bringing support for streaming chat in Glide (integrated with OpenAI, Azure OpenAI and Cohere) ✨ Started handling 401 errors to mark models as premaritally unavailable (e.g. when API key was not correct) 🐛 Fixing the panic related to swagger.yaml file 🐛 Fixing Anthropic chat workflow by passing API key correctly 🔧 Improved Cohere param config and validation ## Changelog ### Added - ✨Streaming Chat Workflow #149 #163 #161 (@roma-glushko) - ✨Streaming Support for Azure OpenAI #173 (@mkrueger12) - ✨Cohere Streaming Chat Support #171 (@mkrueger12) - ✨Start counting token usage in Anthropic Chat #183 (@roma-glushko) - ✨Handle unauthorized error in health tracker #170 (@roma-glushko) - 🔧 #195 #196: Set router ctx in stream chunks & handle end of stream in case of some errors (@roma-glushko) - 🐛🔧 #197: Handle max_tokens & content_filtered finish reasons across OpenAI, Azure and Cohere (@roma-glushko) ## Changed - 🔧 💥 #198: Expose more Cohere params & fixing validation of provider params in config (breaking change) (@roma-glushko) - 🔧 #186: Rendering Durations in a human-friendly way (@roma-glushko) ### Fixed - 🐛 Fix Anthropic API key header #183 (@roma-glushko) - 🐛 #209: Embed Swagger specs into binary to fix panics caused by missing swagger.yaml file (@roma-glushko) - 🐛 #200: Implemented a custom json per line stream reader to read Cohere chat streams correctly (@roma-glushko) ### Security - 🔓 Update crypto lib, golang, fiber #148 (@roma-glushko) ### Miscellaneous - 🐛 Update README.md to fix helm chart location #167 (@arjunnair22) - 🔧 Updated .go-version (@roma-glushko) - ✅ Covered the telemetry by tests #146 (@roma-glushko) - 📝 Separate and list all supported capabilities per provider #190 (@roma-glushko)
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Each response chunk should have information about routerID, modelID:
glide/pkg/routers/router.go
Line 193 in dc7d42f
The text was updated successfully, but these errors were encountered: