Accidental pinhole and antipinhole cameras

There are many ways in which pictures are formed around us. The most efficient mechanisms are to use lenses or narrow apertures to focus light into a picture of what is in front. So a set of occluders (to form a pinhole camera) or a mirror surface (to capture only a subset of the reflected rays) will let us see an image as we view a surface. For those cases, an image is formed by intentionally building a particular arrangement of surfaces that will result in a camera. However, similar arrangements appear naturally by accidental arrangements of surfaces in many places. Often the observer is not aware of the faint images produced by those accidental cameras.

A shadow is also a form of accidental image. The shadow of an object is all the light that is missing because of the object presence in the scene. If we were able to extract the light that is missing (that is the difference between when the object is absent from the scene and when the object is present) we would get an image. That image would be the negative of the shadow and it will be approximatively equivalent to the image produced by a pinhole camera with a pinhole with the shape of the occluder. Therefore, a shadow is not just a dark region around an object. A shadow is the negative picture of the environment around the object producing it.

When we walk under the Sun we project a sharp dark shadow on the ground, and there seems to be nothing special about it. The shadow seems to disapear as soon as we enter under the shadow of a building. However, even when there is no apparent shadow around us, we are still blocking some of the light that fills the space producing a very faint penumbra on the ground all around us. That shadow is colorful, and even if we can not see it, it reveals the scene around us. This effect is what is shown in the two images on the right side. On the image on top, a person is jumping. As there is no direct sun light there seems to be no shadow. However, if we subtract that picture from a picture taken without the person, we will be able to extract a faint shadow as shown in the image bellow. The shadow shows the blue of the sky above, and the yellow color of the buildings on the opposite side to the wall. In fact, an occluder is an accidental antipinhole camera and its shadow is the picture.

In this work we identify and study two types of accidental cameras (pinholes and antipinholes) that can be formed in scenes revealing the scene outside the picture.

Paper: A. Torralba and W. T. Freeman. Accidental pinhole and pinspeck cameras: revealing the scene outside the picture. Proceedings of 25th IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2012).

  • Paper.pdf
  • Sumplementary material

    CVPR talk:

  • Slides.pdf
  • Video of the CVPR talk available at Techtalks

    Data: Here there are a few of the raw videos I used in the paper and the talk. You can play with them to see if you can extract better images than what I got.

  • Input: person walking.MOV, output: person walking output.avi
  • Input: ball.MOV, output: ball output.avi
  • Input: living, output: living room output.avi

  • A person moving inside the room projects a faint change of illumination in the wall. The illumination change can be used to reveal the scene outside the room.

    When there is no direct Sun light, a person walking seems not to project any shadow. But the shadow is still there even if it is so faint that can not be easily seen. We can reveal this shadow by increasing the contrast as shown in the bottom images.