Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Model] Support Mamba2 (Codestral Mamba) #9292

Draft
wants to merge 7 commits into
base: main
Choose a base branch
from
Draft

[Model] Support Mamba2 (Codestral Mamba) #9292

wants to merge 7 commits into from

Commits on Oct 11, 2024

  1. Initial mamba2 support

    @tlrmchlsmth
    tlrmchlsmth committed Oct 11, 2024
    Configuration menu
    Copy the full SHA
    09a30d5 View commit details
    Browse the repository at this point in the history

  2. Some fixes but TP is broken

    @tlrmchlsmth
    tlrmchlsmth committed Oct 11, 2024
    Configuration menu
    Copy the full SHA
    ed3cc3a View commit details
    Browse the repository at this point in the history

  3. format

    @tlrmchlsmth
    tlrmchlsmth committed Oct 11, 2024
    Configuration menu
    Copy the full SHA
    58941dc View commit details
    Browse the repository at this point in the history

Commits on Oct 15, 2024

  1. Move ssd_chunk_combined triton kernels into vLLM

    @tlrmchlsmth
    tlrmchlsmth committed Oct 15, 2024
    Configuration menu
    Copy the full SHA
    0735328 View commit details
    Browse the repository at this point in the history

Commits on Oct 16, 2024

  1. fixups

    @tlrmchlsmth
    tlrmchlsmth committed Oct 16, 2024
    Configuration menu
    Copy the full SHA
    d2bd1ac View commit details
    Browse the repository at this point in the history

Commits on Oct 17, 2024

  1. format and small tweaks to mamba2.py 

    1. Format triton kernels
2. Tweak mamba2.py so models converted using transformers util
   src/transformers/models/mamba2/convert_mamba2_ssm_checkpoint_to_pytorch.py
   will run. However they have garbage output.
    @tlrmchlsmth
    tlrmchlsmth committed Oct 17, 2024
    Configuration menu
    Copy the full SHA
    5f7f67d View commit details
    Browse the repository at this point in the history

  2. Fixes for mamba2-ssm. Need to rethink TP.

    @tlrmchlsmth
    tlrmchlsmth committed Oct 17, 2024
    Configuration menu
    Copy the full SHA
    552d02a View commit details
    Browse the repository at this point in the history