Moving Gradients: A PathBased Method for Plausible Image Interpolation


Dhruv Mahajan  FuChung Huang  Wojciech Matusik  Ravi Ramamoorthi  Peter Belhumeur 
Columbia University  UC Berkeley  Adobe Systems, Inc.  UC Berkeley  Columbia University 
Siggraph 2009 

This paper describes an image interpolation technique to produce a sequence of intermediate frames between two input Images A and B (a,c), as seen in our result in (b). While we show only three frames above, our method moves gradients in the original images to solve simultaneously for any number of frames of a temporally consistent interpolation, producing an animation sequence that preserves the frequency content of the original images, and looks quite natural. Note that this example is very challenging and underconstrained, involving complex, nonrigid and local deformations. Moreover, in contrast to conventional morphingbased techniques, our method is fully automatic  we require only the input images, with no further user assistance. 
Abstract We describe a method for plausible interpolation of images, with a wide range of applications like temporal upsampling for smooth playback of lower frame rate video, smooth view interpolation, and animation of still images. The method is based on the intuitive idea, that a given pixel in the interpolated frames traces out a path in the source images. Therefore, we simply move and copy pixel gradients from the input images along this path. A key innovation is to allow arbitrary (asymmetric) transition points, where the path moves from one image to the other. This flexible transition preserves the frequency content of the originals without ghosting or blurring, and maintains temporal coherence. Perhaps most importantly, our framework makes occlusion handling particularly simple. The transition points allow for matches away from the occluded regions, at any suitable point along the path. Indeed, occlusions do not need to be handled explicitly at all in our initial graphcut optimization. Moreover, a simple comparison of computed path lengths after the optimization, allows us to robustly identify occluded regions, and compute the most plausible interpolation in those areas. Finally, we show that significant improvements are obtained by moving gradients and using Poisson reconstruction. 
Citation Dhruv Mahajan, FuChung Huang, Wojciech Matusik, Ravi Ramamoorthi, Peter Belhumeur.
Moving Gradients: A PathBased Method for Plausible Image Interpolation.
ACM Transactions on Graphics. 28(3) 2009.
Paper Presentation Video 