Sea of Images




Daniel G. Aliaga, Thomas Funkhouser, Dimah Yanovsky, Ingrid Carlbom


A long-standing research problem in computer graphics is to reproduce the visual experience of walking through a large photorealistic environment interactively. On one hand, traditional geometry-based rendering systems fall short of simulating the visual realism of a complex environment. On the other hand, image-based rendering systems have to date been unable to capture and store a sampled representation of a large environment with complex lighting and visibility effects.

We present a "Sea of Images", a practical approach to dense sampling, storage, and reconstruction of the plenoptic function in large, complex indoor environments. We use a motorized cart to capture omnidirectional images every few inches on a eye-height plane throughout an environment. The captured images are compressed and stored in a multi-resolution hierarchy suitable for real-time pre-fetching during an interactive walkthrough. Later, novel images are reconstructed for a simulated observer by re-sampling nearby captured images.

Our system acquires 15,254 images over 1,050 square feet at an average image spacing of 1.5 inches. The average capture and processing time is 7 hours. We demonstrate realistic walkthroughs of real-world environments reproducing specular reflections and occlusion effects while rendering 15-25 frames per second.


Related Publications:

[Daniel G. Aliaga, Dimah Yanovsky, Thomas Funkhouser, Ingrid Carlbom, "Interactive Image-Based Rendering Using Feature Globalization", Proceedings of ACM Symposium on Interactive 3D Graphics, to appear, 2003]

[Daniel G. Aliaga, Thomas Funkhouser, Dimah Yanovsky, Ingrid Carlbom, "Sea of Images", Proceedings of IEEE Visualization, pp. 331-338, 2002]


[Daniel G. Aliaga, Ingrid Carlbom, "Plenoptic Stitching: A Scalable Method for Reconstructing 3D Interactive Walkthroughs", Proceedings of ACM SIGGRAPH, pp. 443-450, 2001]



[Daniel G. Aliaga, "Accurate Catadioptric Calibration for Real-time Pose Estimation in Room-size Environments", Proceedings of International Conference on Computer Vision (ICCV), pp. 127-134, 2001]


[Daniel G. Aliaga, Ingrid Carlbom, "Fiducial Planning for Error-Bounded Pose Estimation of a Panoramic Camera in Large Environments", Special Issue of IEEE Robotics and Automation Magazine: Panoramic Robotics, June, 2003] .


Related Videos:


         Sea of Images (186MB .avi or 24MB .avi)


         Feature Globalization (166MB .avi or 15MB .avi)


         Plenoptic Stitching (158MB .avi or 13MB .avi)





Princeton Graphics Group Projects Page