Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Quantized KV Cache #30483

Merged
merged 21 commits into from
May 23, 2024
Merged

Quantized KV Cache #30483

merged 21 commits into from
May 23, 2024

Commits on May 10, 2024

  1. clean-up

    @zucchini-nlp
    zucchini-nlp committed May 10, 2024
    Configuration menu
    Copy the full SHA
    16731be View commit details
    Browse the repository at this point in the history

  2. Update src/transformers/cache_utils.py 

    Co-authored-by: Arthur <[email protected]>
    @zucchini-nlp @ArthurZucker
    zucchini-nlp and ArthurZucker authored May 10, 2024
    Configuration menu
    Copy the full SHA
    cf00de6 View commit details
    Browse the repository at this point in the history

  3. Update src/transformers/cache_utils.py 

    Co-authored-by: Arthur <[email protected]>
    @zucchini-nlp @ArthurZucker
    zucchini-nlp and ArthurZucker authored May 10, 2024
    Configuration menu
    Copy the full SHA
    6de0d8a View commit details
    Browse the repository at this point in the history

  4. Update src/transformers/cache_utils.py 

    Co-authored-by: Arthur <[email protected]>
    @zucchini-nlp @ArthurZucker
    zucchini-nlp and ArthurZucker authored May 10, 2024
    Configuration menu
    Copy the full SHA
    5a87cbb View commit details
    Browse the repository at this point in the history

  5. fixup

    @zucchini-nlp
    zucchini-nlp committed May 10, 2024
    Configuration menu
    Copy the full SHA
    bfe0804 View commit details
    Browse the repository at this point in the history

Commits on May 13, 2024

  1. Update tests/quantization/quanto_integration/test_quanto.py 

    Co-authored-by: Younes Belkada <[email protected]>
    @zucchini-nlp @younesbelkada
    zucchini-nlp and younesbelkada authored May 13, 2024
    Configuration menu
    Copy the full SHA
    62abc33 View commit details
    Browse the repository at this point in the history

  2. Update src/transformers/generation/configuration_utils.py 

    Co-authored-by: Arthur <[email protected]>
    @zucchini-nlp @ArthurZucker
    zucchini-nlp and ArthurZucker authored May 13, 2024
    Configuration menu
    Copy the full SHA
    519682c View commit details
    Browse the repository at this point in the history

  3. more suggestions

    @zucchini-nlp
    zucchini-nlp committed May 13, 2024
    Configuration menu
    Copy the full SHA
    6f19cee View commit details
    Browse the repository at this point in the history

  4. mapping if torch available

    @zucchini-nlp
    zucchini-nlp committed May 13, 2024
    Configuration menu
    Copy the full SHA
    2b8e042 View commit details
    Browse the repository at this point in the history

Commits on May 14, 2024

  1. Merge branch 'main' into quant

    @zucchini-nlp
    zucchini-nlp authored May 14, 2024
    Configuration menu
    Copy the full SHA
    4c65b5b View commit details
    Browse the repository at this point in the history

  2. run tests & add 'support_quantized' flag

    @zucchini-nlp
    zucchini-nlp committed May 14, 2024
    Configuration menu
    Copy the full SHA
    1d9cf15 View commit details
    Browse the repository at this point in the history

  3. Merge branch 'main' into quant

    @zucchini-nlp
    zucchini-nlp authored May 14, 2024
    Configuration menu
    Copy the full SHA
    a58aa9d View commit details
    Browse the repository at this point in the history

  4. fix jamba test

    @zucchini-nlp
    zucchini-nlp committed May 14, 2024
    Configuration menu
    Copy the full SHA
    d181339 View commit details
    Browse the repository at this point in the history

  5. revert, will be fixed by another PR

    @zucchini-nlp
    zucchini-nlp committed May 14, 2024
    Configuration menu
    Copy the full SHA
    e658c9f View commit details
    Browse the repository at this point in the history

Commits on May 18, 2024

  1. Merge branch 'main' into quant

    @zucchini-nlp
    zucchini-nlp authored May 18, 2024
    Configuration menu
    Copy the full SHA
    9397dda View commit details
    Browse the repository at this point in the history

  2. codestyle

    @zucchini-nlp
    zucchini-nlp committed May 18, 2024
    Configuration menu
    Copy the full SHA
    045e406 View commit details
    Browse the repository at this point in the history

Commits on May 20, 2024

  1. HQQ and versatile cache classes

    @zucchini-nlp
    zucchini-nlp committed May 20, 2024
    Configuration menu
    Copy the full SHA
    3193b43 View commit details
    Browse the repository at this point in the history

Commits on May 21, 2024

  1. final update

    @zucchini-nlp
    zucchini-nlp committed May 21, 2024
    Configuration menu
    Copy the full SHA
    126ce84 View commit details
    Browse the repository at this point in the history

Commits on May 23, 2024

  1. Merge "main"

    @zucchini-nlp
    zucchini-nlp committed May 23, 2024
    Configuration menu
    Copy the full SHA
    0d1df5f View commit details
    Browse the repository at this point in the history

  2. typo

    @zucchini-nlp
    zucchini-nlp committed May 23, 2024
    Configuration menu
    Copy the full SHA
    89413d3 View commit details
    Browse the repository at this point in the history

  3. make tests happy

    @zucchini-nlp
    zucchini-nlp committed May 23, 2024
    Configuration menu
    Copy the full SHA
    9d0f668 View commit details
    Browse the repository at this point in the history