About NV3D measurement data

[WuJH2001]

Hello authors, thanks to your work. I would like to ask a few questions here. I found that your NV3D measurement data is lower than that in other authors' papers. I want to see where the specific differences are. Can you make them public?

[weify627]

Hi thank you for your questions. Could you help specify what the "NV3D measurement data" means? Are they the metrics (e.g., PSNR) of baseline methods reported in our paper?

[jerry-ryu]

@weify627 ,@WuJH2001
First of all, thank you for your excellent paper.
I have the same question. I'd appreciate it if you could help me.

In your paper, Deformable4DGS(CVPR2024) and RealTime4DGS(ICLR2024) record below PSNR 30.

However, in those papers (the above table cites Spacetime Gaussian (CVPR 2024)), the PSNR values are 31 or 32.
4DGaussians[93] refers to Deformable4DGS (CVPR 2024), and 4DGS[100] refers to RealTime4DGS (ICLR 2024).

If the evaluation methods or metrics used are different, I would appreciate it if you could let me know.

[Sheng-Qi]

@weify627 Thank you for your excellent paper. I have diligently followed the guidelines provided in the README.md to train and test the cook_spinach scene of NV3D, and I am pleased to report the following results:

• PSNR: 32.56
• DSSIM: 0.026
• LPIPS-alex: 0.051
• Average processing time per image: 108 ms

These timing results were derived by measuring the execution time of the get_outputs() function during the ns-render process. The PSNR and other metrics you have reported in the paper are commendably high and represent a significant benchmark in the field.

However, I noticed a considerable discrepancy between the ‘time per image’ that I recorded and the figures mentioned in your paper. Could you please elucidate on how the FPS were calculated to achieve the results as stated?

I am grateful for your contributions and for making this valuable research available to the community.

Thank you once again for your time and assistance.

[Sheng-Qi]

The aforementioned timing results were tested on a single A100 GPU.

[weify627]

@jerry-ryu @WuJH2001
We directly ran the official released code of baseline methods (no modification). The PSNRs for all methods (including baselines and ours) are calculated with this script for fair comparison.

We are also curious why we could not reproduce the PSNRs of some baselines (e.g., Deformable4DGS and RealTime4DGS)-- I guess this would be a question to the authors of these baseline methods. If you are able to reproduce the PSNRs reported in their papers with their official codebase, we would also be happy to learn how you achieve that!

[weify627]

Hi @Sheng-Qi, thank you for sharing your results! The rendering metrics look reasonable to me!
As explained in README, this repository contains our PyTorch implementation to support related research. The FPS reported in the paper is measured using our highly optimized CUDA framework, which we plan to commercialize and are not releasing at this time. For inquiries regarding the CUDA-based implementation, please contact Yuanxing Duan at [email protected].