[Question] The inference speed of MAR and MAR-Diffusion

[EPIC-Lab-sjtu]

Thank you for your most valuable contribution in the paper! Currently, our lab are trying to follow this work, but find some problems in the inference costs of MAR and MAR-Diffusion Models.

1. We find that in the inference period, the MLP for diffusion tasks about 70% inference costs, much more than the encoder and the decoder. This may suggest that MAR-Diffusion should have much more computation costs (at least 2 times more) compared with the MAR in the same model size.
2. However, we find that in Figure 6 of the paper, the computation costs of MAR-Diffusion are just slightly higher than MAR (it seems that there are only 10% inference cost improvements), which conflicts with the first observation.

Hence, we are curious about why this happens, why the inference costs of MAR-Diffusion are similar to MAR when the additional computation of MAR-Diffusion (i.e., the diffusion MLP with 100 steps) are much more than the original encoder-decoder costs.

Thanks for your reading!

[LTH14]

Thank you for your interest! We tested the inference speed using a batch size of 256, generating 256 images simultaneously. In this case, the MLP component accounts for only 10-20% of the inference time.

On the other hand, with a smaller batch size, the small MLP in DiffLoss cannot utilize the GPU optimally. In this scenario, the MLP occupies a larger portion of the inference time.

In terms of FLOPs, the MLP's inference FLOPs is ~10% of the encode-decoder.