Many seemingly static scenes contain subtle changes that are invisible to the naked human eye. However, it is possible to pull out these small changes from videos through the use of algorithms we have developed. We give a way to visualize these small changes by amplifying them and we present algorithms to pull out interesting signals from these videos, such as the human pulse, sound from vibrating objects and the motion of hot air.
Videos
|
|
|
|
New video technology that reveals an object's hidden properties
Abe Davis @ TED, March 2015 |
See invisible motion, hear silent sounds
Michael Rubinstein @ TEDxBeaconStreet, Nov 2014 |
Revealing Invisible Changes In The World
NSF Science and Engineering Visualization Challenge 2012 |
Visual Vibrometry: Estimating Material Properties from Small Motions in Video CVPR'15 Supplemental Video
|
|
|
|
Finding the Visible in the Invisible
Story in NY Times, Feb 2013 |
Eulerian Video Magnification
SIGGRAPH'12 Supplemental Video |
Detecting Pulse from Head Motions in Videos
CVPR'13 Supplemental Video |
|
|
|
Phase-based Video Motion Processing
SIGGRAPH'13 Supplemental Video |
The Visual Microphone: Passive Recovery of Sound from Video
SIGGRAPH'14 Supplemental Video |
Amplifying Tiny Movements to Visualize the Invisible
Reuters Video, Jan 2015 |
Software and Code
Eulerian Video Magnification code
Matlab code and executables implementing Eulerian video processing for amplifying color and motion changes.
Phase Based Video Motion Processing code
Matlab code implementing the new and improved phase-based motion magnification pipeline.
Learning-based Video Motion Magnification code
Tensorflow implementation of the learning-based motion magnification pipeline.
Videoscope
Web interface for motion and color magnification. Upload your videos and have them magnified!
Publications (Magnifying Motion and Color Changes)
|
Tae-Hyun Oh*, Ronnachai Jaroensri*, Changil Kim, Mohamed Elgharib, Frédo Durand, William T. Freeman, Wojciech Matusik
Learning-based Video Motion Magnification
European Conference on Computer Vision (ECCV), 2018
[Paper] [Webpage]
A learning-based approach to motion magnification with reduced artifact and better noise handling. |
|
Mohamed A. Elgharib, Mohamed Hefeeda, Frédo Durand, William T. Freeman
Video Magnification in Presence of Large Motions
IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), 2015
[Paper] [Webpage]
A new video technique to motion magnify interesting small motions that are combined with large motions. |
|
Frédo Durand, William T. Freeman, Michael Rubinstein
A World of Movement
Scientific American, Volume 312, Number 1, January 2015
[Article in SciAm] [Videos]
An expository article describing several of the motion magnification techniques and applications we have worked on. |
|
Neal Wadhwa, Michael Rubinstein, Frédo Durand, William T. Freeman
Riesz Pyramids for Fast Phase-Based Video Magnification
Computational Photography (ICCP), 2014 IEEE International Conference on
[Paper] [Webpage]
Provides the quality of the previous phase-based technique with the real-time speed of the original linear technique. |
|
Neal Wadhwa, Michael Rubinstein, Frédo Durand, William T. Freeman
Phase-based Video Motion Processing
ACM Transactions on Graphics, Volume 32, Number 4 (Proc. SIGGRAPH), 2013.
[Paper] [Webpage]
[BibTeX]
A new technique to amplify small motions that solves the noise amplification and intensity clipping artifacts of the previous linear method by manipulating the phase in sub-bands of videos.
|
|
Michael Rubinstein, Neal Wadhwa, Frédo Durand, William T. Freeman
Revealing Invisible Changes In The World
Science Vol. 339 No. 6119, Feb 1 2013
[Article in Science] [Video]
[NSF SciVis 2012] [BibTeX]
An expository video showcasing our results and explaining our technique.
|
|
Hao-Yu Wu, Michael Rubinstein, Eugene Shih, John Guttag, Frédo Durand, William T. Freeman
Eulerian Video Magnification for Revealing Subtle Changes in the World
ACM Transactions on Graphics, Volume 31, Number 4 (Proc. SIGGRAPH), 2012
[Paper] [Webpage]
[BibTeX]
The first Eulerian method to amplify small motions and color variations in videos.
|
|
Ce Liu, Antonio Torralba, William T. Freeman, Frédo Durand, Edward H. Adelson
Motion Magnification
ACM Transactions on Graphics, Volume 24, Number 3 (Proc. SIGGRAPH), 2005
[Paper] [Webpage]
The original Lagrangian method to amplify small motions by explicitly estimating them and then warping the frame by amplified motion amounts.
|
Publications (Analysis of Small Motions)
|
Abe Davis, Katherine L. Bouman, Justin G. Chen, Michael Rubinstein, Frédo Durand, William T. Freeman
Visual Vibrometry: Estimating Material Properties from Small Motions in Video
IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), 2015
[Paper] [Webpage]
A method to estimate the material properties of from small motions in videos.
|
|
Tianfan Xue, Michael Rubinstein, Neal Wadhwa, Anat Levin, Frédo Durand, William T. Freeman
Refraction Wiggles for Measuring Fluid Depth and Velocity from Video
Proc. of European Conference on Computer Vision (ECCV), 2014.
[Paper] [Webpage]
A method to recover the velocity and depth of hot air from video.
|
|
Abe Davis, Michael Rubinstein, Neal Wadhwa, Gautham Mysore, Frédo Durand, William T. Freeman
The Visual Microphone: Passive Recovery of Sound from Video
ACM Transactions on Graphics, Volume 33, Number 4 (Proc. SIGGRAPH), 2014.
[Paper] [Webpage]
A technique to recover sound from videos of objects subtly vibrating in response to sound.
|
|
Justin G. Chen, Neal Wadhwa, Young-Jin Cha, Frédo Durand, William T. Freeman, Oral Buyukozturk
Structural Modal Identification through High Speed Camera Video: Motion Magnification
Proceedings of the 32nd International Modal Analysis Conference (2014)
[Paper]
A validation that the motion magnified motions are indeed real and a way to compute the mode shapes of a cantilevered beam from video.
|
|
Guha Balakrishnan, Frédo Durand, John Guttag
Detecting Pulse from Head Motions in Video
Computer Vision and Pattern Recognition (CVPR), 2013 IEEE Conference on
[Paper] [Video]
A method to measure pulse from the Newtonian motion of the head as blood flows into it.
|
People
Faculty:
|
|
Frédo Durand |
William T. Freeman |
Students, Postdocs and Affiliates:
|
|
|
|
|
|
|
|
|
|
Guha Balakrishnan |
Katherine L. Bouman |
Justin G. Chen |
Abe Davis |
Hossein Mobahi |
Michael Rubinstein |
Neal Wadhwa |
Hao-Yu Wu |
Tianfan Xue |
Ronnachai 'Tiam' Jaroensri |
Collaborators:
Edward H. Adelson,
Oral Buyukozturk,
Anat Levin,
Ce Liu,
John Guttag,
Gautham J. Mysore,
Eugene Shih,
Antonio Torralba
Related Work and Other Applications
Visible Imaging of Global MHD on MAST
Ryan, D. A.
Plasma Science, IEEE Transactions on , vol.42, no.10, pp.2556,2557, Oct. 2014
[Paper]
Video magnification applied to plasma physics.
|
Auto Localization and Segmentation of Occluded Vessels in Robot-Assisted Partial Nephrectomy.
Amir-Khalili, A., Peyrat, J. M., Abinahed, J., Al-Alao, O., Al-Ansari, A., Hamarneh, G., and Abugharbieh, R.
In Medical Image Computing and Computer-Assisted Intervention–MICCAI 2014 (pp. 407-414). Springer International Publishing.
[Paper]
The temporal frequency of subtle vibrations is used to segment blood vessels in medical imagery.
|
Talks
A Big World of Small Motions, Michael Rubinstein
Invited talk at 2013 International Conference in Computational Photography (April 20, 2013)
Invited talk at TEDx @ Beacon Street 2014 (November 15, 2014)
|
Revealing What's Hidden in Videos, William T. Freeman
Plenary Talk at 2014 International Conference on Image Processing (October 29, 2014)
|