Robot warehouse automation has attracted significant interest in recent years, perhaps most visibly in the Amazon Picking Challenge (APC). A fully autonomous warehouse pick-and-place system requires robust vision that reliably recognizes and locates objects amid cluttered environments, self-occlusions, sensor noise, and a large variety of objects. In this paper we present an approach that leverages multi-view RGB-D data and self-supervised, data-driven learning to overcome those difficulties. The approach was part of the MIT-Princeton Team system that took 3rd- and 4th- place in the stowing and picking tasks, respectively at APC 2016. In the proposed approach, we segment and label multiple views of a scene with a fully convolutional neural network, and then fit pre-scanned 3D object models to the resulting segmentation to get the 6D object pose. Training a deep neural network for segmentation typically requires a large amount of training data. We propose a self-supervised method to generate a large labeled dataset without tedious manual segmentation. We demonstrate that our system can reliably estimate the 6D pose of objects under a variety of scenarios. All code, data, and benchmarks are available at http://apc.cs.princeton.edu/

@inproceedings{Zeng:2017:MSD,
   author = "Andy Zeng and Kuan-Ting Yu and Shuran Song and Daniel Suo and Ed Walker
      Jr. and Alberto Rodriguez and Jianxiong Xiao",
   title = "Multi-view Self-supervised Deep Learning for {6D} Pose Estimation in
      the Amazon Picking Challenge",
   booktitle = "IEEE International Conference on Robotics and Automation (ICRA)",
   year = "2017",
   month = may
}