See In Detail: Enhancing Sparse-view 3D Gaussian Splatting with Local Depth and Semantic Regularization
3D Gaussian Splatting (3DGS) has shown remarkable performance in novel view synthesis. However, its rendering quality deteriorates with sparse inphut views, leading to distorted content and reduced details. This limitation hinders its practical application. To address this issue, we propose a sparse-view 3DGS method. Given the inherently ill-posed nature of sparse-view rendering, incorporating prior information is crucial. We propose a semantic regularization technique, using features extracted from the pretrained DINO-ViT model, to ensure multi-view semantic consistency. Additionally, we propose local depth regularization, which constrains depth values to improve generalization on unseen views. Our method outperforms state-of-the-art novel view synthesis approaches, achieving up to 0.4dB improvement in terms of PSNR on the LLFF dataset, with reduced distortion and enhanced visual quality.
3D 高斯点渲染 (3DGS) 在新视角合成中表现出色，但在稀疏输入视角情况下，其渲染质量显著下降，导致内容失真和细节丢失。这一局限性阻碍了其实际应用。为了解决此问题，我们提出了一种稀疏视角的 3DGS 方法。由于稀疏视角渲染本质上是不适定问题，引入先验信息至关重要。 我们提出了一种语义正则化技术，利用预训练的 DINO-ViT 模型提取的特征，确保多视角的语义一致性。此外，我们引入了局部深度正则化，通过约束深度值来提高对未见视角的泛化能力。 实验表明，我们的方法优于当前最先进的新视角合成方法，在 LLFF 数据集上的 PSNR 提升高达 0.4dB，同时显著降低失真并增强视觉质量。