Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Feature] Auto scaling factor tuning for FP8 collective communication #140

Open
wants to merge 15 commits into
base: main
Choose a base branch
from
Open

[Feature] Auto scaling factor tuning for FP8 collective communication #140

wants to merge 15 commits into from

Commits on Dec 7, 2023

  1. wgrad auto scaling in optimizer

    @wkcn
    wkcn committed Dec 7, 2023
    Configuration menu
    Copy the full SHA
    6352d6f View commit details
    Browse the repository at this point in the history

  2. pre_scale support

    @wkcn
    wkcn committed Dec 7, 2023
    Configuration menu
    Copy the full SHA
    57d4a42 View commit details
    Browse the repository at this point in the history

  3. fp8e4m3 for wgrad

    @wkcn
    wkcn committed Dec 7, 2023
    Configuration menu
    Copy the full SHA
    69f73ce View commit details
    Browse the repository at this point in the history

  4. refine code

    @wkcn
    wkcn committed Dec 7, 2023
    Configuration menu
    Copy the full SHA
    55735b7 View commit details
    Browse the repository at this point in the history

  5. comment

    @wkcn
    wkcn committed Dec 7, 2023
    Configuration menu
    Copy the full SHA
    bcd0f74 View commit details
    Browse the repository at this point in the history

  6. pre_scale support in add_to_fp8

    @wkcn
    wkcn committed Dec 7, 2023
    Configuration menu
    Copy the full SHA
    d248505 View commit details
    Browse the repository at this point in the history

Commits on Dec 8, 2023

  1. pre_scale as Tensor

    @wkcn
    wkcn committed Dec 8, 2023
    Configuration menu
    Copy the full SHA
    4a7fd1f View commit details
    Browse the repository at this point in the history

  2. pre_scale *fp32

    @wkcn
    wkcn committed Dec 8, 2023
    Configuration menu
    Copy the full SHA
    c850d63 View commit details
    Browse the repository at this point in the history

  3. pre_scale ut

    @wkcn
    wkcn committed Dec 8, 2023
    Configuration menu
    Copy the full SHA
    617ee0b View commit details
    Browse the repository at this point in the history

  4. test meta

    @wkcn
    wkcn committed Dec 8, 2023
    Configuration menu
    Copy the full SHA
    199eb3a View commit details
    Browse the repository at this point in the history

Commits on Dec 10, 2023

  1. auto scaling freq

    @wkcn
    wkcn committed Dec 10, 2023
    Configuration menu
    Copy the full SHA
    1be092c View commit details
    Browse the repository at this point in the history

  2. Merge branch 'wgrad_auto_scaling' of github.com:wkcn/ms-amp into wgra… 

    …d_auto_scaling
    @wkcn
    wkcn committed Dec 10, 2023
    Configuration menu
    Copy the full SHA
    c08bc0d View commit details
    Browse the repository at this point in the history

Commits on Dec 11, 2023

  1. unittest

    @wkcn
    wkcn committed Dec 11, 2023
    Configuration menu
    Copy the full SHA
    6e58e1e View commit details
    Browse the repository at this point in the history

  2. bug fixed when stacking pre_scales

    @wkcn
    wkcn committed Dec 11, 2023
    Configuration menu
    Copy the full SHA
    df68631 View commit details
    Browse the repository at this point in the history

Commits on Dec 14, 2023

  1. Merge branch 'main' into wgrad_auto_scaling

    @wkcn
    wkcn authored Dec 14, 2023
    Configuration menu
    Copy the full SHA
    664025d View commit details
    Browse the repository at this point in the history