Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Frontend] Add max_tokens prometheus metric #9881

Merged
merged 3 commits into from
Nov 4, 2024
Merged

[Frontend] Add max_tokens prometheus metric #9881

merged 3 commits into from
Nov 4, 2024

Commits on Oct 31, 2024

  1. Add max_tokens to prometheus metrics. This is the number of output to… 

    …kens requested by the user, which is generally different than the number of tokens actually generated

Signed-off-by: Tomer Asida <[email protected]>
    @tomeras91
    tomeras91 committed Oct 31, 2024
    Configuration menu
    Copy the full SHA
    981efbf View commit details
    Browse the repository at this point in the history

  2. Add vllm:request_params_max_tokens metric to tests 

    Signed-off-by: Tomer Asida <[email protected]>
    @tomeras91
    tomeras91 committed Oct 31, 2024
    Configuration menu
    Copy the full SHA
    9986120 View commit details
    Browse the repository at this point in the history

  3. format 

    Signed-off-by: Tomer Asida <[email protected]>
    @tomeras91
    tomeras91 committed Oct 31, 2024
    Configuration menu
    Copy the full SHA
    9267cd0 View commit details
    Browse the repository at this point in the history