Video Magnification

banner slider

Many seemingly static scenes contain subtle changes that are invisible to the naked human eye. However, it is possible to pull out these small changes from videos through the use of algorithms we have developed. We give a way to visualize these small changes by amplifying them and we present algorithms to pull out interesting signals from these videos, such as the human pulse, sound from vibrating objects and the motion of hot air.

 


Videos
Revealing Invisible Changes In The World
NSF Science and Engineering Visualization Challenge 2012
Finding the Visible in the Invisible
Story in NY Times, Feb 2013
Eulerian Video Magnification
SIGGRAPH'12 supplemental video
Detecting Pulse from Head Motions in Videos
CVPR'13 Supplemental Video
Phase-based Video Motion Processing
SIGGRAPH'13 supplemental video
The Visual Microphone: Passive Recovery of Sound from Video
SIGGRAPH'14 Supplemental Video

 


Publications (Magnifying Motion and Color Changes)

Neal Wadhwa, Michael Rubinstein, Frédo Durand, William T. Freeman
Riesz Pyramids for Fast Phase-Based Video Magnification
Computational Photography (ICCP), 2014 IEEE International Conference on
[Paper] [Webpage]
Provides the quality of the previous phase-based technique with the real-time speed of the original linear technique.

Neal Wadhwa, Michael Rubinstein, Frédo Durand, William T. Freeman
Phase-based Video Motion Processing
ACM Transactions on Graphics, Volume 32, Number 4 (Proc. SIGGRAPH), 2013.
[Paper] [Webpage] [BibTeX]
A new technique to amplify small motions that solves the noise amplification and intensity clipping artifacts of the previous linear method by manipulating the phase in sub-bands of videos.

Michael Rubinstein, Neal Wadhwa, Frédo Durand, William T. Freeman
Revealing Invisible Changes In The World
Science Vol. 339 No. 6119, Feb 1 2013
[Article in Science] [Video] [NSF SciVis 2012] [BibTeX]
An expository video showcasing our results and explaining our technique.

Hao-Yu Wu, Michael Rubinstein, Eugene Shih, John Guttag, Frédo Durand, William T. Freeman
Eulerian Video Magnification for Revealing Subtle Changes in the World
ACM Transactions on Graphics, Volume 31, Number 4 (Proc. SIGGRAPH), 2012
[Paper] [Webpage] [BibTeX]
The first Eulerian method to amplify small motions and color variations in videos.

Ce Liu, Antonio Torralba, William T. Freeman, Frédo Durand, Edward H. Adelson
Motion Magnification
ACM Transactions on Graphics, Volume 24, Number 3 (Proc. SIGGRAPH), 2005
[Paper] [Webpage]
The original Lagrangian method to amplify small motions by explicitly estimating them and then warping the frame by amplified motion amounts.


Publications (Analysis of Small Motions)

Tianfan Xue, Michael Rubinstein, Neal Wadhwa, Anat Levin, Frédo Durand, William T. Freeman
Refraction Wiggles for Measuring Fluid Depth and Velocity from Video
Proc. of European Conference on Computer Vision (ECCV), 2014.
[Paper] [Webpage]
A method to recover the velocity and depth of hot air from video.

Abe Davis, Michael Rubinstein, Neal Wadhwa, Gautham Mysore, Frédo Durand, William T. Freeman
The Visual Microphone: Passive Recovery of Sound from Video
ACM Transactions on Graphics, Volume 33, Number 4 (Proc. SIGGRAPH), 2014.
[Paper] [Webpage]
A technique to recover sound from videos of objects subtly vibrating in response to sound.

Justin G. Chen, Neal Wadhwa, Young-Jin Cha, Frédo Durand, William T. Freeman, Oral Buyukozturk
Structural Modal Identification through High Speed Camera Video: Motion Magnification
Proceedings of the 32nd International Modal Analysis Conference (2014)
[Paper]
A validation that the motion magnified motions are indeed real and a way to compute the mode shapes of a cantilevered beam from video.

Guha Balakrishnan, Frédo Durand, John Guttag
Detecting Pulse from Head Motions in Video
Computer Vision and Pattern Recognition (CVPR), 2013 IEEE Conference on
[Paper] [Video]
A method to measure pulse from the Newtonian motion of the head as blood flows into it.


People


Frédo Durand William T. Freeman

Students and Lab Members:
Guha Balakrishnan Justin G. Chen Abe Davis Hossein Mobahi Michael Rubinstein Neal Wadhwa Hao-Yu Wu Tianfan Xue

Collaborators: Edward H. Adelson, Oral Buyukozturk, Anat Levin, Ce Liu, Gautham J. Mysore, Eugene Shih, Antonio Torralba

Related Work and Other Applications

Pregnant belly motion magnified by YouTube user Bellyfeed
World's first motion magnified guinea pig by YouTube user SuperCreaturefan
Hold Still, video magnification portraits of apparently still people by Erin Knutson

Visible Imaging of Global MHD on MAST
Ryan, D. A.
Plasma Science, IEEE Transactions on , vol.42, no.10, pp.2556,2557, Oct. 2014
[Paper]
Video magnification applied to plasma physics.

Auto Localization and Segmentation of Occluded Vessels in Robot-Assisted Partial Nephrectomy.
Amir-Khalili, A., Peyrat, J. M., Abinahed, J., Al-Alao, O., Al-Ansari, A., Hamarneh, G., and Abugharbieh, R.
In Medical Image Computing and Computer-Assisted Intervention–MICCAI 2014 (pp. 407-414). Springer International Publishing.
[Paper]
The temporal frequency of subtle vibrations is used to segment blood vessels in medical imagery.


Talks

A Big World of Small Motions, Michael Rubinstein
Invited talk at 2013 International Conference in Computational Photography (April 20, 2013)
Invited talk at TEDx @ Beacon Street 2013 (November 15, 2014)

Revealing What's Hidden in Videos, William T. Freeman
Plenary Talk at 2014 International Conference on Image Processing (October 29, 2014)


Last updated: December 2014