SIGGRAPH 2013 Phase-Based Video Motion Processing
Neal Wadhwa Michael Rubinstein Frédo Durand William T. Freeman
MIT Computer Science and Artificial Intelligence Laboratory

(a) Input (b) Linear [Wu et al. 2012] (c) Phase-based (this work)
Motion magnification of a crane imperceptibly swaying in the wind. (a) Top: a zoom-in onto a patch in the original sequence (crane) shown on the left. Bottom: a spatiotemporal XT slice of the video along the profile marked on the zoomed-in patch. (b-c) Linear [Wu et al. 2012] and phase-based motion magnification results, respectively, shown for the corresponding patch and spatiotemporal slice as in (a). The previous, linear method visualizes the crane's motion, but amplifies both signal and noise and introduces artifacts for higher spatial frequencies and larger motions, shown by the clipped intensities (bright pixels) in (b). In comparison, our new phase-based method supports larger magnification factors with significantly fewer artifacts and less noise (c).

 

Abstract

We introduce a technique to manipulate small movements in videos based on an analysis of motion in complex-valued image pyramids. Phase variations of the coefficients of a complex-valued steerable pyramid over time correspond to motion, and can be temporally processed and amplified to reveal imperceptible motions, or attenuated to remove distracting changes. This processing does not involve the computation of optical flow, and in comparison to the previous Eulerian Video Magnification method it supports larger amplification factors and is significantly less sensitive to noise. These improved capabilities broaden the set of applications for motion processing in videos. We demonstrate the advantages of this approach on synthetic and natural video sequences, and explore applications in scientific analysis, visualization and video enhancement.

@article{Wadhwa2013PhaseBased,
  author = {Neal Wadhwa and Michael Rubinstein and Fr\'{e}do Durand and William T. Freeman},
  title = {Phase-Based Video Motion Processing},
  journal = {ACM Trans. Graph. (Proceedings SIGGRAPH 2013)},
  year = {2013},
  volume = {32},
  number={4}
}

Paper: pdf

Web Application: Magnify your videos with phase-based motion processing with Videoscope by Quanta Research!

SIGGRAPH 2013 Presentation: With videos (zip, 609 MB), Without videos (pptx, 49 MB)

Supplemental Video:


Download: mp4 (292 MB)

Supplemental material: Comparison with Wu et al. [2012] pre- and post-processed by video denoising (zip, 354MB)

Code: zip

Data and Results: All source and result videos (zip, 1.3GB)

 

Related Publications

Riesz Pyramids for Fast Phase-Based Video Magnification, ICCP 2014

Analysis and Visualization of Temporal Variations in Video, Michael Rubinstein, PhD Thesis, MIT Feb 2014

Eulerian Video Magnification for Revealing Subtle Changes in the World, SIGGRAPH 2012

 

Selected Results

Engine vibrations (mp4)

result (mp4)

result (mp4)

result (mp4)

result (mp4)

result (mp4)

 

Acknowledgements

We would like to thank the SIGGRAPH reviewers for their comments. We thank Justin Chen for his assistance with the controlled metal structure experiment. We acknowledge funding support from: Quanta Computer, Shell Research, the DARPA SCENICC program, NSF CGV-1111415 and a gift from Cognex. Michael Rubinstein was supported by the Microsoft Research PhD Fellowship. Neal Wadhwa was supported by the DoD through the NDSEG fellowship program.

 

 

Last updated: February 2015