Weakly-Supervised Single-view Dense 3D Point Cloud Reconstruction via Differentiable Renderer

• 作者: Peng Jin Shaoli Liu Jianhua Liu Hao Huang Linlin Yang Michael Weinmann Reinhard Klein
• 来源期刊: 中国机械工程学报
• 年，卷（期）: 2021,（5）
• 所属期刊栏目: Intelligent Manufacturing Technology
• 页码范围: 195-205
• 页数: 11页
• 语种: 英文

摘要

In recent years,addressing ill-posed problems by leveraging prior knowledge contained in databases on learning techniques has gained much attention.In this paper,we focus on complete three-dimensional(3D)point cloud reconstruction based on a single red-green-blue(RGB)image,a task that cannot be approached using classical recon-struction techniques.For this purpose,we used an encoder-decoder framework to encode the RGB information in latent space,and to predict the 3D structure of the considered object from different viewpoints.The individual predic-tions are combined to yield a common representation that is used in a module combining camera pose estimation and rendering,thereby achieving differentiability with respect to imaging process and the camera pose,and optimi-zation of the two-dimensional prediction error of novel viewpoints.Thus,our method allows end-to-end training and does not require supervision based on additional ground-truth(GT)mask annotations or ground-truth camera pose annotations.Our evaluation of synthetic and real-world data demonstrates the robustness of our approach to appear-ance changes and self-occlusions,through outperformance of current state-of-the-art methods in terms of accuracy,density,and model completeness.