Princeton > CS Dept > PIXL > Publications Local Access 

PlaneMatch: Patch Coplanarity Prediction for Robust RGB-D Reconstruction
European Conference on Computer Vision (ECCV) oral presentation, September 2018

Yifei Shi, Kai Xu, Matthias Nießner,
Szymon Rusinkiewicz, Thomas Funkhouser


Abstract

We introduce a novel RGB-D patch descriptor designed for detecting coplanar surfaces in SLAM reconstruction. The core of our method is a deep convolutional neural network that takes in RGB, depth, and normal information of a planar patch in an image and outputs a descriptor that can be used to find coplanar patches from other images. We train the network on 10 million triplets of coplanar and non-coplanar patches, and evaluate on a new coplanarity benchmark created from commodity RGB-D scans. Experiments show that our learned descriptor outperforms alternatives extended for this new task by a significant margin. In addition, we demonstrate the benefits of coplanarity matching in a robust RGBD reconstruction formulation. We find that coplanarity constraints detected with our method are sufficient to get reconstruction results comparable to state-of-the-art frameworks on most scenes, but outperform other methods on established benchmarks when combined with traditional keypoint matching.

Citation (BibTeX)

Yifei Shi, Kai Xu, Matthias Nießner, Szymon Rusinkiewicz, and Thomas Funkhouser. PlaneMatch: Patch Coplanarity Prediction for Robust RGB-D Reconstruction. European Conference on Computer Vision (ECCV) oral presentation, September 2018.

Link
  Paper page on arXiv