Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Quickfix: Accelerate YAML and LoRA Fused Ops #92

Merged
merged 11 commits into from
Oct 14, 2024
Merged

Quickfix: Accelerate YAML and LoRA Fused Ops #92

merged 11 commits into from
Oct 14, 2024

Conversation

fabianlim
Copy link
Contributor

@fabianlim fabianlim commented Oct 11, 2024

This PR fixes:

Restoration of Benches for Full FT and Regular Peft

We have removed mixed precision for these benches and restored performance, see
outliers.csv

Updated Compatibility Matrix

Compatibility Matrix with Mixed Precision

torch_dtype Mixed Precision Full-FT-FOAK PEFT-FOAK QPEFT-FOAK
FLOAT16 - Compatible Compatible
FLOAT16 FP16 ValueError:
Attempting to
unscale FP16 gradients.
See here
Compatible Compatible
BFLOAT16 - Compatible Compatible
BFLOAT16 BF16 Compatible Compatible Less Performant

@fabianlim fabianlim requested a review from anhuong October 11, 2024 17:20
Signed-off-by: Yu Chin Fabian Lim <[email protected]>
Signed-off-by: Yu Chin Fabian Lim <[email protected]>
Copy link
Collaborator

@anhuong anhuong left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Signed-off-by: Yu Chin Fabian Lim <[email protected]>
Signed-off-by: Yu Chin Fabian Lim <[email protected]>
Signed-off-by: Yu Chin Fabian Lim <[email protected]>
Signed-off-by: Yu Chin Fabian Lim <[email protected]>
Signed-off-by: Yu Chin Fabian Lim <[email protected]>
Signed-off-by: Yu Chin Fabian Lim <[email protected]>
Signed-off-by: Yu Chin Fabian Lim <[email protected]>
@fabianlim fabianlim merged commit 97fc3c1 into main Oct 14, 2024
6 checks passed
@fabianlim fabianlim deleted the quickfix branch October 14, 2024 03:13
fabianlim added a commit to willmj/fms-acceleration that referenced this pull request Oct 29, 2024
fabianlim added a commit that referenced this pull request Oct 31, 2024
* feat: Add granite GPTQ model

Signed-off-by: Will Johnson <[email protected]>

* fmt + lint

Signed-off-by: Yu Chin Fabian Lim <[email protected]>

* update granite benches to be in line with #92

Signed-off-by: Yu Chin Fabian Lim <[email protected]>

---------

Signed-off-by: Will Johnson <[email protected]>
Signed-off-by: Yu Chin Fabian Lim <[email protected]>
Co-authored-by: Yu Chin Fabian Lim <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants