Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

ci: bench: support sse and fix prompt processing time / server: add tokens usage in stream OAI response #6495

Merged
merged 6 commits into from
Apr 6, 2024
Merged

ci: bench: support sse and fix prompt processing time / server: add tokens usage in stream OAI response #6495

merged 6 commits into from
Apr 6, 2024

Commits on Apr 4, 2024

  1. ci: bench: support sse and fix prompt processing time 

    server: add tokens usage in stream mode
    @phymbert
    phymbert committed Apr 4, 2024
    Configuration menu
    Copy the full SHA
    713fa98 View commit details
    Browse the repository at this point in the history

  2. ci: bench: README.md EOL

    @phymbert
    phymbert committed Apr 4, 2024
    Configuration menu
    Copy the full SHA
    1534d90 View commit details
    Browse the repository at this point in the history

  3. ci: bench: remove total pp and tg as it is not accurate

    @phymbert
    phymbert committed Apr 4, 2024
    Configuration menu
    Copy the full SHA
    3694026 View commit details
    Browse the repository at this point in the history

  4. ci: bench: fix case when there is no token generated

    @phymbert
    phymbert committed Apr 4, 2024
    Configuration menu
    Copy the full SHA
    59dc4bb View commit details
    Browse the repository at this point in the history

  5. ci: bench: change to the 95 percentile for pp and tg as it is closer … 

    …to what the server exports in metrics
    @phymbert
    phymbert committed Apr 4, 2024
    Configuration menu
    Copy the full SHA
    b6b50b1 View commit details
    Browse the repository at this point in the history

  6. ci: bench: fix finish reason rate

    @phymbert
    phymbert committed Apr 4, 2024
    Configuration menu
    Copy the full SHA
    8789e17 View commit details
    Browse the repository at this point in the history