ICCP 2014 Riesz Pyramids for Fast Phase-Based Video Magnification
Neal Wadhwa1 Michael Rubinstein2,1 Frédo Durand1 William T. Freeman1
1MIT Computer Science and Artificial Intelligence Laboratory 2Microsoft Research

Best Demo at CVPR 2014!


A recording of our real-time phase-based video magnification interface, in which an excited wine glass's oscillations are made visible. We play a 439 Hz note, the fundamental frequency of the wine glass at it, causing it to subtly move. In the strobed video, on the left, the motions are too small to be seen. They become visible, on the right, when amplified 250 times with the Riesz Pyramid. Click on the image to play the video.

Abstract

We present a new compact image pyramid representation, the Riesz pyramid, that can be used for real-time phase-based motion magnification. Our new representation is less overcomplete than even the smallest two orientation, octave-bandwidth complex steerable pyramid, and can be implemented using compact, efficient linear filters in the spatial domain. Motion-magnified videos produced with this new representation are of comparable quality to those produced with the complex steerable pyramid. When used with phase-based video magnification, the Riesz pyramid phase-shifts image features along only their dominant orientation rather than every orientation like the complex steerable pyramid.

@article{Wadhwa2014RieszPyramid,
  author = {Neal Wadhwa and Michael Rubinstein and Fr\'{e}do Durand and William T. Freeman},
  title = {Riesz Pyramids for Fast Phase-Based Video Magnification},
  booktitle = {Computational Photography (ICCP), 2014 IEEE International Conference on},
  year = {2014},
  organization = {IEEE}
}

Paper: pdf

Quaternion Technical Report: pdf

ICCP 2014 Presentation: With Videos (zip, 121 MB), Without videos (pptx, 23MB)

Supplemental Material: zip (4.5 MB)
Data: zip (159 MB)
Results and Comparisons: zip (216 MB)

 

Real-time Video Magnification

The techniques in this paper drive the real-time video magnification system that we demoed at CVPR 2014. We won the best demo award!

We presented our real-time demo at the 2014 Computer Vision and Pattern Recognition conference (CVPR) in Columbus, Ohio.

Related Publications

Analysis and Visualization of Temporal Variations in Video, Michael Rubinstein, PhD Thesis, MIT Feb 2014

Phase-Based Video Motion Processing, SIGGRAPH 2013

Eulerian Video Magnification for Revealing Subtle Changes in the World, SIGGRAPH 2012

 

Selected Results

Offline results:

result (mp4)

result (mp4)

Real-time results:

result (mp4)

result (mp4)

result (mp4)

 

Acknowledgements

We would like to thank the ICCP reviewers for their comments. We acknowledge funding support from: Quanta Computer, Shell Research and NSF CGV-1111415. Michael Rubinstein was supported by the Microsoft Research PhD Fellowship. Neal Wadhwa was supported by the MIT Department of Mathematics and the NSF Graduate Research Fellowship Program under Grant No. 1122374.

 

 

Last updated: August 2014