Flux Speed Up with PipeFusion

[mali-afridi]

Hello, you guys implemented Sequence parallel approach for Flux. But, the inference speed for 3 and 4gpus is slower as compared to 1 GPU. Do you have ideas to speed it up using distrifuser or pipefuser? Why is it not possible in pipefusion?

[Eigensystem]

Can you give us your command? Flux is 2x faster than 1 card when tested on our L40 4-card device with ulysses_degree = 2, ring_degree = 2

[feifeibear]

https://github.com/xdit-project/xDiT/blob/main/docs/performance/flux.md
@mali-afridi we have a detailed performance report regarding to Flux.1, and they can be reproduced by the scripts in the example.

Could you please provide more information about your devices and environments?

Regarding PipeFusion, we are planing to support it. Would like to help us on this project?