Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add an option to stream for Gemini. #8652

Closed
5 tasks done
Hisir0909 opened this issue Sep 23, 2024 · 4 comments · Fixed by #8678
Closed
5 tasks done

Add an option to stream for Gemini. #8652

Hisir0909 opened this issue Sep 23, 2024 · 4 comments · Fixed by #8678
Labels
💪 enhancement New feature or request

Comments

@Hisir0909
Copy link
Contributor

Self Checks

  • I have searched for existing issues search for existing issues, including closed ones.
  • I confirm that I am using English to submit this report (我已阅读并同意 Language Policy).
  • [FOR CHINESE USERS] 请务必使用英文提交 Issue,否则会被关闭。谢谢!:)
  • Please do not modify this template :) and fill in all the required fields.

1. Is this request related to a challenge you're experiencing? Tell me about your story.

image
It is possible that the gemini interface's review mechanism causes the stream interface to have a duration that is three times longer than the no stream interface.
Model: gemini-1.5-flash

2. Additional context or comments

Someone told me how to add whether to use stream configuration for LLM nodes.

3. Can you help us with this feature?

  • I am interested in contributing to this feature.
@dosubot dosubot bot added the 💪 enhancement New feature or request label Sep 23, 2024
@AAEE86 AAEE86 mentioned this issue Sep 23, 2024
12 tasks
@AAEE86
Copy link
Contributor

AAEE86 commented Sep 24, 2024

Sorry, I didn't notice that you checked the third item. If you submit a pull request, I can close it.

@Hisir0909
Copy link
Contributor Author

Sorry, I didn't notice that you checked the third item. If you submit a pull request, I can close it.抱歉,我没注意到您勾选了第三项。如果您提交拉取请求,我可以关闭它。

Thank you. If what you submitted can be used, that’s fine. I'm just not clear on how the stream parameter is passed in, so I'm unsure if adding it directly in the YAML is effective.

@Hisir0909
Copy link
Contributor Author

image
image
In fact, @AAEE86 ‘s pr is incorrect. Simply adding the stream flag in the YAML file is ineffective; it's passed to the model's internal parameters, not to the stream flag itself.

@AAEE86
Copy link
Contributor

AAEE86 commented Oct 8, 2024

I found this problem and created a new issue #8998
Can you solve this problem?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
💪 enhancement New feature or request
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants