-
Notifications
You must be signed in to change notification settings - Fork 2.7k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add dev-mode setting for forcing the use of C2 #43988
Conversation
This comment was marked as resolved.
This comment was marked as resolved.
I think we might need to balance what the current default brings vs what it costs in some cases. If it brings just a tiny bit more speed in standard cases and makes some use cases really really worse, it might make sense to just use C2 all the time. Now if the current default actually makes things a lot better in most cases then it makes sense to keep it. |
C2 should not be used by default as it does come with a non-trivial startup penalty. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
OK, thanks for having a look. I was asking because from time to time, we are micro-optimizing because it doesn't cost us anything. It's worth revisiting when we actually identify a cost.
LGTM, then!
Yeah, I'm glad you brought it up as we haven't checked in a long time! |
Status for workflow
|
This setting allows users to opt-in to C2 when it makes sense (for example when running LLM inference via Jlama or Llama3.java)