Recent advancements in radiance field rendering show promising results in 3D scene representation, where Gaussian splatting-based techniques emerge as state-of-the-art due to their quality and efficiency. Gaussian splatting is widely used for various applications, including 3D human representation. However, previous 3D Gaussian splatting methods either use parametric body models as additional information or fail to provide any underlying structure, like human biomechanical features, which are essential for different applications. In this paper, we present a novel approach called HFGaussian that can estimate novel views and human features, such as the 3D skeleton, 3D key points, and dense pose, from sparse input images in real time at 25 FPS. The proposed method leverages generalizable Gaussian splatting technique to represent the human subject and its associated features, enabling efficient and generalizable reconstruction. By incorporating a pose regression network and the feature splatting technique with Gaussian splatting, HFGaussian demonstrates improved capabilities over existing 3D human methods, showcasing the potential of 3D human representations with integrated biomechanics. We thoroughly evaluate our HFGaussian method against the latest state-of-the-art techniques in human Gaussian splatting and pose estimation, demonstrating its real-time, state-of-the-art performance.
在辐射场渲染的最新进展中，3D场景表示取得了令人瞩目的成果，其中基于高斯散点的技术凭借其质量和效率成为先进方法。这种技术已广泛应用于包括3D人体表示在内的多种应用。然而，现有的3D高斯散点方法要么依赖于参数化人体模型作为额外信息，要么未能提供底层结构（如人体生物力学特征），这些特征对于不同应用至关重要。本文提出了一种新方法，称为HFGaussian，能够从稀疏输入图像实时估计新视图和人体特征（如3D骨架、3D关键点和密集姿态），帧率达25 FPS。该方法利用通用高斯散点技术来表示人体对象及其相关特征，实现高效且具有泛化能力的重建。通过结合姿态回归网络和特征散点技术与高斯散点，HFGaussian在现有的3D人体方法之上展示了增强的能力，体现了与生物力学集成的人体3D表示的潜力。我们对HFGaussian方法与最新的先进人体高斯散点和姿态估计技术进行了全面评估，证明其在实时性能和先进性方面的表现。