Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Kernel][Model] Improve continuous batching for Jamba and Mamba #9189

Merged
merged 27 commits into from
Oct 16, 2024
Merged

[Kernel][Model] Improve continuous batching for Jamba and Mamba #9189

merged 27 commits into from
Oct 16, 2024

Commits on Oct 9, 2024

  1. Do not read or write to padding

    @mzusman
    mzusman committed Oct 9, 2024
    Configuration menu
    Copy the full SHA
    2109fbc View commit details
    Browse the repository at this point in the history

  2. Padding support to mamba_ssm

    @mzusman
    mzusman committed Oct 9, 2024
    Configuration menu
    Copy the full SHA
    8545ec4 View commit details
    Browse the repository at this point in the history

  3. continuous batching jamba

    @mzusman
    mzusman committed Oct 9, 2024
    Configuration menu
    Copy the full SHA
    24758ed View commit details
    Browse the repository at this point in the history

  4. format

    @mzusman
    mzusman committed Oct 9, 2024
    Configuration menu
    Copy the full SHA
    385c257 View commit details
    Browse the repository at this point in the history

  5. format

    @mzusman
    mzusman committed Oct 9, 2024
    Configuration menu
    Copy the full SHA
    927be2c View commit details
    Browse the repository at this point in the history

Commits on Oct 10, 2024

  1. Add pad_slot_id to kernels params

    @mzusman
    mzusman committed Oct 10, 2024
    Configuration menu
    Copy the full SHA
    5bc07fb View commit details
    Browse the repository at this point in the history

  2. revert merged column parallel

    @mzusman
    mzusman committed Oct 10, 2024
    Configuration menu
    Copy the full SHA
    6c9b043 View commit details
    Browse the repository at this point in the history

  3. Add TP=2 test

    @mzusman
    mzusman committed Oct 10, 2024
    Configuration menu
    Copy the full SHA
    3a4d02b View commit details
    Browse the repository at this point in the history

  4. Add with padding params to mamba tests

    @mzusman
    mzusman committed Oct 10, 2024
    Configuration menu
    Copy the full SHA
    c934c30 View commit details
    Browse the repository at this point in the history

  5. Format

    @mzusman
    mzusman committed Oct 10, 2024
    Configuration menu
    Copy the full SHA
    8a6626c View commit details
    Browse the repository at this point in the history

  6. causal_conv1d outputs to x inplace

    @mzusman
    mzusman committed Oct 10, 2024
    Configuration menu
    Copy the full SHA
    a6eab1b View commit details
    Browse the repository at this point in the history

  7. Fix tests and add with_padding test

    @mzusman
    mzusman committed Oct 10, 2024
    Configuration menu
    Copy the full SHA
    f6d3a05 View commit details
    Browse the repository at this point in the history

  8. Fix tests

    @mzusman
    mzusman committed Oct 10, 2024
    Configuration menu
    Copy the full SHA
    d96bd01 View commit details
    Browse the repository at this point in the history

  9. Return none

    @mzusman
    mzusman committed Oct 10, 2024
    Configuration menu
    Copy the full SHA
    a663fa8 View commit details
    Browse the repository at this point in the history

  10. Merge remote-tracking branch 'github/main' into continous_batching_ma… 

    …mba_from_scratch
    @mzusman
    mzusman committed Oct 10, 2024
    Configuration menu
    Copy the full SHA
    39776d5 View commit details
    Browse the repository at this point in the history

  11. fix typo

    @mzusman
    mzusman committed Oct 10, 2024
    Configuration menu
    Copy the full SHA
    69ebab8 View commit details
    Browse the repository at this point in the history

  12. remove diffs and use pad_slot_id as var in tests

    @mzusman
    mzusman committed Oct 10, 2024
    Configuration menu
    Copy the full SHA
    906379d View commit details
    Browse the repository at this point in the history

Commits on Oct 13, 2024

  1. Adaptiations to vllm-project#6484 and Merge remote-tracking branch 'g… 

    …ithub/main' into continous_batching_mamba_from_scratch
    @mzusman
    mzusman committed Oct 13, 2024
    Configuration menu
    Copy the full SHA
    fa1162e View commit details
    Browse the repository at this point in the history

  2. Fix tests

    @mzusman
    mzusman committed Oct 13, 2024
    Configuration menu
    Copy the full SHA
    94fe819 View commit details
    Browse the repository at this point in the history

  3. format

    @mzusman
    mzusman committed Oct 13, 2024
    Configuration menu
    Copy the full SHA
    1544231 View commit details
    Browse the repository at this point in the history

  4. Format

    @mzusman
    mzusman committed Oct 13, 2024
    Configuration menu
    Copy the full SHA
    158f22d View commit details
    Browse the repository at this point in the history

Commits on Oct 15, 2024

  1. Merge remote-tracking branch 'github/main' into continous_batching_ma… 

    …mba_from_scratch
    @mzusman
    mzusman committed Oct 15, 2024
    Configuration menu
    Copy the full SHA
    c893c68 View commit details
    Browse the repository at this point in the history

Commits on Oct 16, 2024

  1. Address review comments

    @mzusman
    mzusman committed Oct 16, 2024
    Configuration menu
    Copy the full SHA
    40d14ee View commit details
    Browse the repository at this point in the history

  2. Format

    @mzusman
    mzusman committed Oct 16, 2024
    Configuration menu
    Copy the full SHA
    7325254 View commit details
    Browse the repository at this point in the history

  3. Fix mamba

    @mzusman
    mzusman committed Oct 16, 2024
    Configuration menu
    Copy the full SHA
    f4c1198 View commit details
    Browse the repository at this point in the history

  4. add cache empty for consistency

    @mzusman
    mzusman committed Oct 16, 2024
    Configuration menu
    Copy the full SHA
    d1c5c32 View commit details
    Browse the repository at this point in the history

  5. Format

    @mzusman
    mzusman committed Oct 16, 2024
    Configuration menu
    Copy the full SHA
    9905319 View commit details
    Browse the repository at this point in the history