Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CC: Improve performance of responses to user #1689

Closed
4 of 5 tasks
matthewbolanos opened this issue Jun 23, 2023 · 1 comment
Closed
4 of 5 tasks

CC: Improve performance of responses to user #1689

matthewbolanos opened this issue Jun 23, 2023 · 1 comment
Assignees
Labels
sk team issue A tag to denote issues that where created by the Semantic Kernel team (i.e., not the community)

Comments

@matthewbolanos
Copy link
Member

matthewbolanos commented Jun 23, 2023

The goal of this task is to speed up the initial time for feedback after the user has sent a request. Below is a list of optimizations that could be made to the application. We should continue making improvements to the experience until time-to-first message is under 5 seconds.

  • Parallelizing LLM calls
  • Stream responses from the bot
  • Provide a "bot is typing" message
  • Provide messaging describing what the bot is doing (e.g., creating plan, performing step 1, step 2, etc.)
  • Print an initial message back to the user before the plan is fully crated.
@matthewbolanos matthewbolanos converted this from a draft issue Jun 23, 2023
@matthewbolanos matthewbolanos added copilot chat sk team issue A tag to denote issues that where created by the Semantic Kernel team (i.e., not the community) labels Jun 23, 2023
@matthewbolanos
Copy link
Member Author

Is there a reason why we're using prompts for ExtractAudienceAsync? Doing that with native code could help speed things up and avoid using tokens.

@dehoward dehoward moved this from Ready to In review in Apps & Services Semantic Kernel Jul 24, 2023
@github-project-automation github-project-automation bot moved this from In review to Done in Apps & Services Semantic Kernel Aug 3, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
sk team issue A tag to denote issues that where created by the Semantic Kernel team (i.e., not the community)
Projects
No open projects
Development

No branches or pull requests

3 participants