PrincetonComputer SciencePIXL GroupPublications → [Sitzmann et al. 2019] Local Access
DeepVoxels: Learning Persistent 3D Feature Embeddings

IEEE Conference on Computer Vision and Pattern Recognition (CVPR) oral presentation, July 2019

Vincent Sitzmann, Justus Thies, Felix Heide,
Matthias Nießner, Gordon Wetzstein, Michael Zollhöfer
Abstract

In this work, we address the lack of 3D understanding of generative neural networks by introducing a persistent 3D feature embedding for view synthesis. To this end, we propose DeepVoxels, a learned representation that encodes the view-dependent appearance of a 3D scene without having to explicitly model its geometry. At its core, our approach is based on a Cartesian 3D grid of persistent embedded features that learn to make use of the underlying 3D scene structure. Our approach combines insights from 3D geometric computer vision with recent advances in learning image-to-image mappings based on adversarial loss functions. DeepVoxels is supervised, without requiring a 3D reconstruction of the scene, using a 2D re-rendering loss and enforces perspective and multi-view geometry in a principled manner. We apply our persistent 3D scene representation to the problem of novel view synthesis demonstrating high-quality results for a variety of challenging scenes.
Citation

Vincent Sitzmann, Justus Thies, Felix Heide, Matthias Nießner, Gordon Wetzstein, and Michael Zollhöfer.
"DeepVoxels: Learning Persistent 3D Feature Embeddings."
IEEE Conference on Computer Vision and Pattern Recognition (CVPR) oral presentation, June 2019.

BibTeX

@inproceedings{Sitzmann:2019:DLP,
   author = "Vincent Sitzmann and Justus Thies and Felix Heide and Matthias
      Nie{\ss}ner and Gordon Wetzstein and Michael Zollh{\"o}fer",
   title = "{DeepVoxels}: Learning Persistent {3D} Feature Embeddings",
   booktitle = "IEEE Conference on Computer Vision and Pattern Recognition (CVPR) oral
      presentation",
   year = "2019",
   month = jun
}