Moving Gradients: A Path-Based Method for Plausible Image Interpolation

 

Dhruv Mahajan Fu-Chung Huang Wojciech Matusik Ravi Ramamoorthi Peter Belhumeur
Columbia University UC Berkeley Adobe Systems, Inc. UC Berkeley Columbia University
 
Siggraph 2009

 

This paper describes an image interpolation technique to produce a sequence of intermediate frames between two input Images A and B (a,c), as seen in our result in (b). While we show only three frames above, our method moves gradients in the original images to solve simultaneously for any number of frames of a temporally consistent interpolation, producing an animation sequence that preserves the frequency content of the original images, and looks quite natural. Note that this example is very challenging and under-constrained, involving complex, non-rigid and local deformations. Moreover, in contrast to conventional morphing-based techniques, our method is fully automatic -- we require only the input images, with no further user assistance.

 

Abstract

 

We describe a method for plausible interpolation of images, with a wide range of applications like temporal up-sampling for smooth playback of lower frame rate video, smooth view interpolation, and animation of still images. The method is based on the intuitive idea, that a given pixel in the interpolated frames traces out a path in the source images. Therefore, we simply move and copy pixel gradients from the input images along this path. A key innovation is to allow arbitrary (asymmetric) transition points, where the path moves from one image to the other. This flexible transition preserves the frequency content of the originals without ghosting or blurring, and maintains temporal coherence. Perhaps most importantly, our framework makes occlusion handling particularly simple. The transition points allow for matches away from the occluded regions, at any suitable point along the path. Indeed, occlusions do not need to be handled explicitly at all in our initial graph-cut optimization. Moreover, a simple comparison of computed path lengths after the optimization, allows us to robustly identify occluded regions, and compute the most plausible interpolation in those areas. Finally, we show that significant improvements are obtained by moving gradients and using Poisson reconstruction.

 

Citation

Dhruv Mahajan, Fu-Chung Huang, Wojciech Matusik, Ravi Ramamoorthi, Peter Belhumeur. Moving Gradients: A Path-Based Method for Plausible Image Interpolation. ACM Transactions on Graphics. 28(3) 2009.

Paper

PDF

Presentation

PPT

Video

MOV