You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Reasoning and Description:
In order to prevent high spendings and hitting AI core rate limiting, we need to introduce our own rate limiting per cluster which uses our companion.
Tasks:
Make sure our traces in Langfuse have identifiers of clusters which uses our companion
Whenever we receive a request we need to:
Pull number of tokens used by cluster from 00:00UTC to 23:59UTC (possible through Langfuse API) or within 24h period(whatever is easier).
If the number of tokens is higher than a constant we set, return a message to the user that they over consumed their token usage and should come back after 23:59UTC or after X minutes(if we go with 24h approach)
Acceptance criteria:
Traces are being identified per cluster
Total token usage is being pulled within agreed time range and compared against the constant
The text was updated successfully, but these errors were encountered:
Reasoning and Description:
In order to prevent high spendings and hitting AI core rate limiting, we need to introduce our own rate limiting per cluster which uses our companion.
Tasks:
Whenever we receive a request we need to:
Acceptance criteria:
The text was updated successfully, but these errors were encountered: