Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Speculative Decoding 2/2 ] Integrate typical acceptance sampler into Spec Decode Worker #5348

Merged
merged 51 commits into from
Jul 1, 2024
Merged

[Speculative Decoding 2/2 ] Integrate typical acceptance sampler into Spec Decode Worker #5348

merged 51 commits into from
Jul 1, 2024

Commits on May 29, 2024

  1. Merge pull request #1 from vllm-project/main 

    Pull from head
    @sroy745
    sroy745 authored May 29, 2024
    Configuration menu
    Copy the full SHA
    5650b95 View commit details
    Browse the repository at this point in the history

Commits on Jun 3, 2024

  1. Merge branch 'vllm-project:main' into main

    @sroy745
    sroy745 authored Jun 3, 2024
    Configuration menu
    Copy the full SHA
    8f36146 View commit details
    Browse the repository at this point in the history

  2. Merge branch 'vllm-project:main' into main

    @sroy745
    sroy745 authored Jun 3, 2024
    Configuration menu
    Copy the full SHA
    9e75057 View commit details
    Browse the repository at this point in the history

Commits on Jun 7, 2024

  1. Integrate Typical Acceptance Sampler into spec decode worker

    @sroy745
    sroy745 committed Jun 7, 2024
    Configuration menu
    Copy the full SHA
    bbf1484 View commit details
    Browse the repository at this point in the history

  2. Merge branch 'vllm-project:main' into main

    @sroy745
    sroy745 authored Jun 7, 2024
    Configuration menu
    Copy the full SHA
    db2c679 View commit details
    Browse the repository at this point in the history

Commits on Jun 9, 2024

  1. Fixing tests

    @sroy745
    sroy745 committed Jun 9, 2024
    Configuration menu
    Copy the full SHA
    3495673 View commit details
    Browse the repository at this point in the history

Commits on Jun 10, 2024

  1. adding missing commit

    @sroy745
    sroy745 committed Jun 10, 2024
    Configuration menu
    Copy the full SHA
    26c7c57 View commit details
    Browse the repository at this point in the history

  2. reverting changes to conftest

    @sroy745
    sroy745 committed Jun 10, 2024
    Configuration menu
    Copy the full SHA
    090f0bf View commit details
    Browse the repository at this point in the history

  3. reverting changes to conftest

    @sroy745
    sroy745 committed Jun 10, 2024
    Configuration menu
    Copy the full SHA
    733cc6e View commit details
    Browse the repository at this point in the history

  4. Merge branch 'main' into spec_decode_integrate_accpetance_sampler

    @sroy745
    sroy745 authored Jun 10, 2024
    Configuration menu
    Copy the full SHA
    19ca0c9 View commit details
    Browse the repository at this point in the history

  5. Dummy commit

    @sroy745
    sroy745 committed Jun 10, 2024
    Configuration menu
    Copy the full SHA
    acf8d2c View commit details
    Browse the repository at this point in the history

  6. Merge branch 'spec_decode_integrate_accpetance_sampler' of https://gi… 

    …thub.com/sroy745/vllm into spec_decode_integrate_accpetance_sampler
    @sroy745
    sroy745 committed Jun 10, 2024
    Configuration menu
    Copy the full SHA
    2d2b02b View commit details
    Browse the repository at this point in the history

  7. Revert unnecessary commits

    @sroy745
    sroy745 committed Jun 10, 2024
    Configuration menu
    Copy the full SHA
    2010b35 View commit details
    Browse the repository at this point in the history

  8. Merge branch 'vllm-project:main' into main

    @sroy745
    sroy745 authored Jun 10, 2024
    Configuration menu
    Copy the full SHA
    8d7512c View commit details
    Browse the repository at this point in the history

  9. Merge remote-tracking branch 'origin/main' into spec_decode_integrate… 

    …_accpetance_sampler
    @sroy745
    sroy745 committed Jun 10, 2024
    Configuration menu
    Copy the full SHA
    7fa64b6 View commit details
    Browse the repository at this point in the history

  10. Pass only one sampler which can either be the RejectionSampler of the… 

    … TypicalAcceptanceSampler
    @sroy745
    sroy745 committed Jun 10, 2024
    Configuration menu
    Copy the full SHA
    dea6fbd View commit details
    Browse the repository at this point in the history

  11. Fix test scripture

    @sroy745
    sroy745 committed Jun 10, 2024
    Configuration menu
    Copy the full SHA
    c3383db View commit details
    Browse the repository at this point in the history

Commits on Jun 11, 2024

  1. Fix tests

    @sroy745
    sroy745 committed Jun 11, 2024
    Configuration menu
    Copy the full SHA
    b15abba View commit details
    Browse the repository at this point in the history

  2. Fix tests

    @sroy745
    sroy745 committed Jun 11, 2024
    Configuration menu
    Copy the full SHA
    6ca731c View commit details
    Browse the repository at this point in the history

  3. Pass only 1 verification_sampler which can either be rejectionSampler… 

    … of TypicalAcceptanceSampler
    @sroy745
    sroy745 committed Jun 11, 2024
    Configuration menu
    Copy the full SHA
    483c671 View commit details
    Browse the repository at this point in the history

  4. Update metrics.py to take the base sampler class

    @sroy745
    sroy745 committed Jun 11, 2024
    Configuration menu
    Copy the full SHA
    2c6d06c View commit details
    Browse the repository at this point in the history

  5. Fix tests and comments

    @sroy745
    sroy745 committed Jun 11, 2024
    Configuration menu
    Copy the full SHA
    027b485 View commit details
    Browse the repository at this point in the history

  6. Fix test fixture and default values of args

    @sroy745
    sroy745 committed Jun 11, 2024
    Configuration menu
    Copy the full SHA
    ded92ac View commit details
    Browse the repository at this point in the history

  7. Small misc fixes

    @sroy745
    sroy745 committed Jun 11, 2024
    Configuration menu
    Copy the full SHA
    738871e View commit details
    Browse the repository at this point in the history

  8. Fix spec_decode/test_metrics.py

    @sroy745
    sroy745 committed Jun 11, 2024
    Configuration menu
    Copy the full SHA
    50e8771 View commit details
    Browse the repository at this point in the history

  9. Merge branch 'main' into spec_decode_integrate_accpetance_sampler

    @sroy745
    sroy745 authored Jun 11, 2024
    Configuration menu
    Copy the full SHA
    101611e View commit details
    Browse the repository at this point in the history

Commits on Jun 25, 2024

  1. Merge branch 'main' into spec_decode_integrate_accpetance_sampler

    @sroy745
    sroy745 authored Jun 25, 2024
    Configuration menu
    Copy the full SHA
    5e6638b View commit details
    Browse the repository at this point in the history

  2. Make rejection_sampler.py and typical_acceptance_sampler.py implement… 

    … the same interface
    @sroy745
    sroy745 committed Jun 25, 2024
    Configuration menu
    Copy the full SHA
    cc760a0 View commit details
    Browse the repository at this point in the history

  3. Raise exception instead of returning None for invalid sampler name

    @sroy745
    sroy745 committed Jun 25, 2024
    Configuration menu
    Copy the full SHA
    360ce0b View commit details
    Browse the repository at this point in the history

  4. Adding log about type of sampler

    @sroy745
    sroy745 committed Jun 25, 2024
    Configuration menu
    Copy the full SHA
    6572ba4 View commit details
    Browse the repository at this point in the history

Commits on Jun 26, 2024

  1. Misc comment fixes

    @sroy745
    sroy745 committed Jun 26, 2024
    Configuration menu
    Copy the full SHA
    be85f07 View commit details
    Browse the repository at this point in the history

  2. Misc fixes

    @sroy745
    sroy745 committed Jun 26, 2024
    Configuration menu
    Copy the full SHA
    6dc9efe View commit details
    Browse the repository at this point in the history

  3. Misc fixes

    @sroy745
    sroy745 committed Jun 26, 2024
    Configuration menu
    Copy the full SHA
    512fad9 View commit details
    Browse the repository at this point in the history

  4. Misc fixes

    @sroy745
    sroy745 committed Jun 26, 2024
    Configuration menu
    Copy the full SHA
    b1d510c View commit details
    Browse the repository at this point in the history

  5. Misc fixes

    @sroy745
    sroy745 committed Jun 26, 2024
    Configuration menu
    Copy the full SHA
    f4b9e4d View commit details
    Browse the repository at this point in the history

  6. Documentation

    @sroy745
    sroy745 committed Jun 26, 2024
    Configuration menu
    Copy the full SHA
    0ea9408 View commit details
    Browse the repository at this point in the history

  7. Fix comments

    @sroy745
    sroy745 committed Jun 26, 2024
    Configuration menu
    Copy the full SHA
    5772d04 View commit details
    Browse the repository at this point in the history

  8. Fix arg name

    @sroy745
    sroy745 committed Jun 26, 2024
    Configuration menu
    Copy the full SHA
    b7254e7 View commit details
    Browse the repository at this point in the history

  9. Fixing a test

    @sroy745
    sroy745 committed Jun 26, 2024
    Configuration menu
    Copy the full SHA
    ef93081 View commit details
    Browse the repository at this point in the history

  10. Fix comment

    @sroy745
    sroy745 committed Jun 26, 2024
    Configuration menu
    Copy the full SHA
    0165842 View commit details
    Browse the repository at this point in the history

  11. Fix formatting

    @sroy745
    sroy745 committed Jun 26, 2024
    Configuration menu
    Copy the full SHA
    510974b View commit details
    Browse the repository at this point in the history

  12. Fixing tests and lint failures

    @sroy745
    sroy745 committed Jun 26, 2024
    Configuration menu
    Copy the full SHA
    396fa54 View commit details
    Browse the repository at this point in the history

Commits on Jun 27, 2024

  1. Removing e2e test for TypicalAcceptanceSampler from test_ngram_correc… 

    …tness.py
    @sroy745
    sroy745 committed Jun 27, 2024
    Configuration menu
    Copy the full SHA
    f8cc895 View commit details
    Browse the repository at this point in the history

  2. Fix a comment

    @sroy745
    sroy745 committed Jun 27, 2024
    Configuration menu
    Copy the full SHA
    439117d View commit details
    Browse the repository at this point in the history

  3. Dummy commit

    @sroy745
    sroy745 committed Jun 27, 2024
    Configuration menu
    Copy the full SHA
    75f034f View commit details
    Browse the repository at this point in the history

  4. Merge pull request #2 from vllm-project/main 

    Merge changes.
    @sroy745
    sroy745 authored Jun 27, 2024
    Configuration menu
    Copy the full SHA
    a0f5ade View commit details
    Browse the repository at this point in the history

  5. Fix format error

    @sroy745
    sroy745 committed Jun 27, 2024
    Configuration menu
    Copy the full SHA
    3082255 View commit details
    Browse the repository at this point in the history

Commits on Jun 28, 2024

  1. Merge pull request #3 from vllm-project/main 

    Merging new commits
    @sroy745
    sroy745 authored Jun 28, 2024
    Configuration menu
    Copy the full SHA
    4e7f51a View commit details
    Browse the repository at this point in the history

Commits on Jun 29, 2024

  1. Dummy fix

    @sroy745
    sroy745 committed Jun 29, 2024
    Configuration menu
    Copy the full SHA
    d26c624 View commit details
    Browse the repository at this point in the history

  2. Merge branch 'main' into spec_decode_integrate_accpetance_sampler

    @sroy745
    sroy745 authored Jun 29, 2024
    Configuration menu
    Copy the full SHA
    98d5f92 View commit details
    Browse the repository at this point in the history

  3. Update test_multistep_correctness.py

    @sroy745
    sroy745 authored Jun 29, 2024
    Configuration menu
    Copy the full SHA
    f186844 View commit details
    Browse the repository at this point in the history