计算机科学 ›› 2026, Vol. 53 ›› Issue (8): 127-138.doi: 10.11896/jsjkx.260200106

• 计算机图形学 & 多媒体 • 上一篇    下一篇

融合掩膜感知生成式先验的稀疏视角高斯溅射一致性重建

郭宸珲, 陈泽彬, 谭光   

  1. 中山大学智能工程学院 广东 深圳 518107
  • 收稿日期:2026-02-27 修回日期:2026-05-30 出版日期:2026-08-15 发布日期:2026-08-17
  • 通讯作者: 谭光(tanguang@mail.sysu.edu.cn)
  • 作者简介:(guochh25@mail2.sysu.edu.cn)
  • 基金资助:
    深圳市自然科学基金(JCYJ20241202130025030)

Sparse-view Gaussian Splatting Consistent Reconstruction with Mask-guided Generative Prior

GUO Chenhui, CHEN Zebin, TAN Guang   

  1. School of Intelligent Systems Engineering, Sun Yat-sen University, Shenzhen, Guangdong 518107, China
  • Received:2026-02-27 Revised:2026-05-30 Published:2026-08-15 Online:2026-08-17
  • About author:GUO Chenhui,born in 2001,postgra-duate,is a member of CCF(No.A47415G).His main research interests include 3D reconstruction and deep learning.
    TAN Guang,born in 1978,Ph.D,professor,is a member of CCF(No.19464M).His main research interests include mobile computing,distributed computing,and networking.
  • Supported by:
    Shenzhen Science and Technology Program(JCYJ20241202130025030).

摘要: 高斯溅射重建在效率与质量之间具有良好权衡,但在稀疏视角条件下,由于观测不足,几何与外观约束不充分,重建结果容易出现结构缺失,并在新视角渲染时表现为未观测区域内容缺失和伪影等问题,渲染真实感与多视角一致性下降。为缓解上述欠约束退化,提出一种生成先验引导的稀疏视角高斯溅射重建优化框架。首先,以预训练去退化扩散模型DiFix为基础构建掩膜引导补全修复扩散模型(Mask-guided Restoration Diffusion,MRD),并在所构建的训练数据上进行微调。MRD通过引入轻量级掩膜预测分支(Mask Head)定位以缺失区域为主的待修复区域,并在训练阶段采用掩膜引导的区域选择性注噪,使随机噪声主要作用于缺失区域,以增强补全效果,同时兼顾对渲染伪影的修复。随后,将MRD生成结果以伪观测形式周期性引入高斯溅射训练,并采用由易到难的渐进式融合策略逐步扩大伪观测视角跨度,以可控方式增强未观测区域约束并提升多视角一致性。同时,引入区域加权约束,以降低伪观测不确定性对优化过程的影响。此外,在DL3DV与Mip-NeRF 360数据集上进行了实验验证,结果表明,所提方法可适配2DGS与3DGS重建管线,在多项评价指标上均取得更优结果。定性分析结果进一步表明,所提方法能够使稀疏视角下的高斯溅射重建更加完整、稳定,并显著提升新视角渲染的视觉质量与多视角一致性。

关键词: 稀疏视角, 高斯溅射, 生成先验, 掩膜引导补全修复扩散模型, 渐进式融合

Abstract: Gaussian splatting achieves a favorable balance between efficiency and reconstruction quality.However,under sparse-view conditions,insufficient observations lead to under-constrained geometry and appearance,resulting in structural incompleteness.Consequently,novel view synthesis often suffers from missing content in unobserved regions and rendering artifacts,degra-ding visual realism and multi-view consistency.To address these issues,this paper proposes a generative prior-guided optimization framework for sparse-view Gaussian splatting reconstruction.Specifically,this paper builds a mask-guided restoration diffusion(MRD) model based on the pretrained degradation removal diffusion model DiFix,and further fine-tunes it on establishedtraining dataset.MRD introduces a lightweight mask prediction branch(Mask Head) to localize restoration regions primarily correspon-ding to missing areas,while also responding to certain rendering artifacts.During training,a mask-guided selective noise injection strategy is adopted,where random noise is mainly applied to the missing regions to enhance completion performance,while preserving the ability to correct rendering artifacts.The generated results from MRD are then periodically incorporated into the Gaussian splatting training as pseudo-observations,and a progressive fusion strategy is employed to gradually expand the viewpoint span of pseudo-observations in a controlled manner,thereby strengthening constraints in unobserved regions and improving multi-view consistency.In addition,a region-weighted constraint is introduced to reduce the impact of uncertainty in pseudo-observations on the optimization process.Experiments on the DL3DV and Mip-NeRF 360 datasets demonstrate that the proposed method is compatible with both 2DGS and 3DGS pipelines and achieves superior performance across multiple evaluation metrics.Qualitative results further show that the proposed method produces more complete and stable reconstructions under sparse-view settings,significantly improving visual quality and multi-view consistency in novel view synthesis.

Key words: Sparse-view, Gaussian splatting, Generative prior, Mask-guided restoration diffusion model, Progressive fusion

中图分类号: 

  • TP391
[1] KERBL B,KOPANAS G,LEIMKÜHLER T,et al.3D Gaussian splatting for real-time radiance field rendering[J].ACM Transactions on Graphics,2023,42(4):139:1-139:14.
[2] ZHU Z,FAN Z,JIANG Y,et al.Fsgs:Real-time few-shot view synthesis using gaussian splatting[C]//European Conference on Computer Vision.Cham:Springer,2024:145-163.
[3] LI J,ZHANG J,BAI X,et al.Dngaussian:Optimizing sparse-view 3d gaussian radiance fields with global-local depth normalization[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2024:20775-20785.
[4] HOU Z X,LI B C,CAI B Y,et al.High Quality Image Generation Method Based on Improved Diffusion Model[J].Computer Science,2025,52(S1):461-469.
[5] WU J Z,ZHANG Y,TURKI H,et al.Difix3d+:Improving 3d reconstructions with single-step diffusion models[C]//Procee-dings of the Computer Vision and Pattern Recognition Confe-rence.2025:26024-26035.
[6] KONG H,YANG X,WANG X.Generative sparse-view gaussian splatting[C]//Proceedings of the Computer Vision and Pattern Recognition Conference.2025:26745-26755.
[7] ZHANG J,LI J,YU X,et al.Cor-gs:sparse-view 3d gaussian splatting via co-regularization[C]//European Conference on Computer Vision.Cham:Springer,2024:335-352.
[8] PARK H,RYU G,KIM W.Dropgaussian:Structural regularization for sparse-view gaussian splatting[C]//Proceedings of the Computer Vision and Pattern Recognition Conference.2025:21600-21609.
[9] XU Y,WANG L,CHEN M,et al.DropoutGS:Dropping OutGaussians for Better Sparse-view Rendering[C]//Proceedings of the Computer Vision and Pattern Recognition Conference.2025:701-710.
[10] WAN Y,SHAO M,CHENG Y,et al.S2Gaussian:Sparse-View Super-Resolution 3D Gaussian Splatting[C]//Proceedings of the Computer Vision and Pattern Recognition Conference.2025:711-721.
[11] PENG R,XU W,TANG L,et al.Structure consistent gaussian splatting with matching prior for few-shot novel view synthesis[J].Advances in Neural Information Processing Systems,2024,37:97328-97352.
[12] YIN R,YUGAY V,LI Y,et al.FewViewGS:Gaussian splatting with few view matching and multi-stage training[J].Advances in Neural Information Processing Systems,2024,37:127204-127225.
[13] XU H,PENG S,WANG F,et al.Depthsplat:Connecting gaussian splatting and depth[C]//Proceedings of the Computer Vision and Pattern Recognition Conference.2025:16453-16463.
[14] WU J,LI R,ZHU Y,et al.Sparse2dgs:Geometry-prioritizedgaussian splatting for surface reconstruction from sparse views[C]//Proceedings of the Computer Vision and Pattern Recognition Conference.2025:11307-11316.
[15] XU W,GAO H,SHEN S,et al.Mvpgs:Excavating multi-view priors for gaussian splatting from sparse input views[C]//European Conference on Computer Vision.Cham:Springer,2024:203-220.
[16] CHEN Y,XU H,ZHENG C,et al.Mvsplat:Efficient 3d gaussian splatting from sparse multi-view images[C]//European Conference on Computer Vision.Cham:Springer,2024:370-386.
[17] LIU T,WANG G,HU S,et al.Mvsgaussian:Fast generalizable gaussian splatting reconstruction from multi-view stereo[C]//European Conference on Computer Vision.Cham:Springer,2024:37-53.
[18] CHARATAN D,LI S L,TAGLIASACCHI A,et al.pixelsplat:3d gaussian splats from image pairs for scalable generalizable 3d reconstruction[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2024:19457-19467.
[19] CHEN Y,ZHENG C,XU H,et al.Mvsplat360:Feed-forward360 scene synthesis from sparse views[J].Advances in Neural Information Processing Systems,2024,37:107064-107086.
[20] YANG H,HUI L,QIAN J,et al.GSRecon:Efficient Generalizable Gaussian Splatting for Surface Reconstruction from Sparse Views[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision.2025:25346-25356.
[21] GAO Z,BIAN J W,LIN G,et al.SurfaceSplat:Connecting Surface Reconstruction and Gaussian Splatting[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision.2025:28525-28534.
[22] XU J,GAO S,SHAN Y.Freesplatter:Pose-free gaussian splatting for sparse-view 3d reconstruction[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision.2025:25442-25452.
[23] LIU X,ZHOU C,HUANG S.3dgs-enhancer:Enhancing un-bounded 3d gaussian splatting with view-consistent 2d diffusion priors[J].Advances in Neural Information Processing Systems,2024,37:133305-133327.
[24] ZHONG Y,LI Z,CHEN D Z,et al.Taming Video Diffusion Prior with Scene-Grounding Guidance for 3D Gaussian Splatting from Sparse Inputs[C]//Proceedings of the Computer Vision and Pattern Recognition Conference.2025:6133-6143.
[25] KONG H,YANG X,WANG X.Rogsplat:Robust gaussiansplatting via generative priors[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision.2025:25735-25745.
[26] YU Z,WANG H,YANG J,et al.Sgd:Street view synthesis with gaussian splatting and diffusion prior[C]//2025 IEEE/CVF Winter Conference on Applications of Computer Vision(WACV).IEEE,2025:3812-3822.
[27] LEE S,LEE G H.DiET-GS:Diffusion Prior and Event Stream-Assisted Motion Deblurring 3D Gaussian Splatting[C]//Proceedings of the Computer Vision and Pattern Recognition Conference.2025:21739-21749.
[28] DU K,LIANG Z,SHEN Y,et al.Gs-id:Illumination decomposition on gaussian splatting via adaptive light aggregation and diffusion-guided material priors[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision.2025:26220-26229.
[29] TANG Z,ZHANG J,CHENG X,et al.Cycle3d:High-quality and consistent image-to-3d generation via generation-reconstruction cycle[C]//Proceedings of the AAAI Conference on Artificial Intelligence.2025:7320-7328.
[30] YANG X,CHEN Y,CHEN C,et al.Learn to Optimize Denoi-sing Scores:A Unified and Improved Diffusion Prior for 3D Ge-neration[C]//European Conference on Computer Vision.Cham:Springer,2024:136-152.
[31] ROMBACH R,BLATTMANN A,LORENZ D,et al.High-resolution image synthesis with latent diffusion models[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2022:10684-10695.
[32] RADFORD A,KIM J W,HALLACY C,et al.Learning transferable visual models from natural language supervision[C]//International Conference on Machine Learning.PMLR,2021:8748-8763.
[33] SAUER A,LORENZ D,BLATTMANN A,et al.Adversarial dif-fusion distillation[C]//European Conference on Computer Vision.Cham:Springer,2024:87-103.
[34] LING L,SHENG Y,TU Z,et al.Dl3dv-10k:A large-scale scene dataset for deep learning-based 3d vision[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2024:22160-22169.
[35] FAN Z,CONG W,WEN K,et al.Instantsplat:Sparse-viewgaussian splatting in seconds[J].arXiv:2403.20309,2024.
[36] BARRON J T,MILDENHALL B,VERBIN D,et al.Mip-nerf 360:Unbounded anti-aliased neural radiance fields[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2022:5470-5479.
[37] WANG J,CHEN M,KARAEV N,et al.Vggt:Visual geometry grounded transformer[C]//Proceedings of the Computer Vision and Pattern Recognition Conference.2025:5294-5306.
[38] SCHONBERGER J L,FRAHM J M.Structure-from-motion revisited[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2016:4104-4113.
[39] HUANG B,YU Z,CHEN A,et al.2D gaussian splatting forgeometrically accurate radiance fields[C]//ACM SIGGRAPH 2024 Conference Papers.2024:1-11.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!