Princeton > CS Dept > PIXL > Publications Local Access 

The more you look, the more you see: towards general object understanding through recursive refinement
Winter Conference on Applications of Computer Vision (WACV), March 2018

Jingyan Wang, Olga Russakovsky, Deva Ramanan


Abstract

Comprehensive object understanding is a central challenge in visual recognition, yet most advances with deep neural networks reason about each aspect in isolation. In this work, we present a unified framework to tackle this broader object understanding problem. We formalize a refinement module that recursively develops understanding across space and semantics — “the more it looks, the more it sees.” More concretely, we cluster the objects within each semantic category into fine-grained subcategories; our recursive model extracts features for each region of interest, recursively predicts the location and the content of the region, and selectively chooses a small subset of the regions to process in the next step. Our model can quickly determine if an object is present, followed by its class (“Is this a person?”), and finally report fine-grained predictions (“Is this person standing?”). Our experiments demonstrate the advantages of joint reasoning about spatial layout and fine-grained semantics. On the PASCAL VOC dataset, our proposed model simultaneously achieves strong performance on instance segmentation, part segmentation and keypoint detection in a single efficient pipeline that does not require explicit training for each task. One of the reasons for our strong performance is the ability to naturally leverage highly-engineered architectures, such as Faster-RCNN, within our pipeline.

Citation (BibTeX)

Jingyan Wang, Olga Russakovsky, and Deva Ramanan. The more you look, the more you see: towards general object understanding through recursive refinement. Winter Conference on Applications of Computer Vision (WACV), March 2018.