New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

FLUX latency optimizations #20

Merged

atiorh merged 2 commits into main from atila/flux_latency_opt

Aug 19, 2024

Contributor

atiorh commented Aug 18, 2024 •

edited

Loading

Remove unnecssary mx.eval calls
Remove float32 casting for nn.RMSNorm inputs (has internal high precision accumulators)
Remove hardcoded float32 cast for VAE Decoder


          Remove unnecessary casts and mx.eval calls for a 30 percent speed-up.…

bacd70f

… Remove hardcoded float32 cast for VAE

mgierschdev commented Aug 18, 2024

Any ETA for Flux.dev ?


          Fuse layer norm and affine transform

fc551f3

atiorh merged commit 3d1b7b7 into main

1 check passed

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet