Abstract

We present a system for generating "infinite" images from large collections of photos by means of transformed image retrieval. Given a query image, we first transform it to simulate how it would look if the camera moved sideways and then perform image retrieval based on the transformed image. We then blend the query and retrieved images to create a larger panorama. Repeating this process will produce an "infinite" image. The transformed image retrieval model is not limited to simple 2-D left/right image translation, however, and we show how to approximate other camera motions like rotation and forward motion/zoom-in using simple 2-D image transforms. We represent images in the database as a graph where each node is an image and different types of edges correspond to different types of geometric transformations simulating different camera motions. Generating infinite images is thus reduced to following paths in the image graph. Given this data structure we can also generate a panorama that connects two query images, simply by finding the shortest path between the two in the image graph. We call this option the "mage taxi." Our approach does not assume photographs are of a single real 3-D location, nor that they were taken at the same time. Instead, we organize the photos in themes, such as city streets or skylines and synthesize new virtual scenes by combining images from distinct but visually similar locations. There are a number of potential applications to this technology. It can be used to generate long panoramas as well as content aware transitions between reference images or video shots. Finally, the image graph allows users to interactively explore large photo collections for ideation, games, social interaction, and artistic purposes.

Panorama Examples


Video


Papers

Biliana Kaneva, Josef Sivic, Antonio Torralba, Shai Avidan, William Freeman. Infinite Images: Creating and Exploring a Large Photorealistic Virtual Space Proceedings of the IEEE, 2010

Josef Sivic, Biliana Kaneva, Antonio Torralba, Shai Avidan, William Freeman. Creating and Exploring a Large Photorealistic Virtual Space First IEEE Workshop on Internet Vision, CVPR 2008

BibTex 

@InProceedings{Kaneva_2010,
  author    = {Biliana Kaneva and Josef Sivic and Antonio Torralba and Shai Avidan and William T. Freeman},
  title     = {Infinite Images: Creating and Exploring a Large Photorealistic Virtual Space},
  booktitle = {Proceedings of the IEEE},
  year      = {2010}
}

@InProceedings{Sivic_2008,
  author    = {Josef Sivic and Biliana Kaneva and Antonio Torralba and Shai Avidan and William T. Freeman},
  title     = {Creating and Exploring a Large Photorealistic Virtual Space},
  booktitle = {First IEEE Workshop on Internet Vision},
  year      = {2008}
}

Acknowledgments

Funding for this research was provided by ARDA VACE, by NGA NEGI-1582-04-0004, by MURI Grant N00014-06-1- 0734, and by ANR project HFIBMR (ANR-07-BLAN-0331-01).