Neal Wadhwa1	Justin G. Chen1,2	Jonathan B. Sellon3,4	Donglai Wei1
Michael Rubinstein5	Roozbeh Ghaffari4	Dennis M. Freeman3,4,6	Oral Büyüköztürk2
Pai Wang7	Sijie Sun7	Sung Hoon Kang7,8	Katia Bertoldi7
	Frédo Durand1,6	William T. Freeman1,5,6

1MIT Computer Science and Artificial Intelligence Laboratory	2MIT Department of Civil and Environmental Engineering
3Harvard-MIT Program in Health Sciences and Technology	4MIT Research Laboratory of Electronics
5Google Research	6Department of Electrical Engineering and Computer Science
7Harvard School of Engineering and Applied Sciences	8Johns Hopkins Department of Mechanical Engineering

One example video from our paper. Traveling waves of the tectorial membrane revealed. The displacement from mean location of the membrane in the input video on the left was amplified 20 times to produce the motion magnified video shown on the right. The original video consists of eight frames. The included video repeats these eight frames 10 times for 80 frames and plays the result at 10 FPS.

Although the human visual system is remarkable at perceiving and interpreting motions, it has limited sensitivity, and we cannot see motions that are smaller than some threshold. Although difficult to visualize, tiny motions below this threshold are important and can reveal physical mechanisms, or be precursors to large motions in the case of mechanical failure. Here, we present a “motion microscope,” a computational tool that quantifies tiny motions in videos and then visualizes them by producing a new video in which the motions are made large enough to see. Three scientific visualizations are shown, spanning macroscopic to nanoscopic length scales. They are the resonant vibrations of a bridge demonstrating simultaneous spatial and temporal modal analysis, micrometer vibrations of a metamaterial demonstrating wave propagation through an elastic matrix with embedded resonating units, and nanometer motions of an extracellular tissue found in the inner ear demonstrating a mechanism of frequency separation in hearing. In these instances, the motion microscope uncovers hidden dynamics over a variety of length scales, leading to the discovery of previously unknown phenomena.

@article {Wadhwa2017MotionMicroscopy,
  author = {Wadhwa, Neal and Chen, Justin G. and Sellon, Jonathan B. and Wei, Donglai and Rubinstein, Michael and
            Ghaffari, Roozbeh and Freeman, Dennis M. and B{\"u}y{\"u}k{\"o}zt{\"u}rk, Oral and Wang, Pai and
            Sun, Sijie and Kang, Sung Hoon and Bertoldi, Katia and Durand, Fr{\'e}do and Freeman, William T.},
  title = {Motion microscopy for visualizing and quantifying small motions},
  volume = {114},
  number = {44},
  pages = {11639--11644},
  year = {2017},
  doi = {10.1073/pnas.1703715114},
  publisher = {National Academy of Sciences},
  issn = {0027-8424},
  URL = {http://www.pnas.org/content/114/44/11639},
  journal = {Proceedings of the National Academy of Sciences}
}

Paper: pdf

Supplemental Material: pdf

Data: zip (1.5 GB)

Code: zip

 

Related Publications

Revealing and Analyzing Imperceptible Deviations in Images and Videos, Neal Wadhwa, PhD Thesis, MIT Feb 2016

Riesz Pyramids for Fast Phase-Based Video Magnification , ICCP 2014

Analysis and Visualization of Temporal Variations in Video, Michael Rubinstein, PhD Thesis, MIT Feb 2014

Phase-Based Video Motion Processing, SIGGRAPH 2013

Eulerian Video Magnification for Revealing Subtle Changes in the World, SIGGRAPH 2012

 

Supplemental Videos

Movie S1 (mov)

Movie S2 (mov)

Movie S3 (mov)

Movie S4 (mov)

Movie S5 (mov)

 

Acknowledgements

We thank Professor Erin Bell and Travis Adams at University of New Hampshire and New Hampshire Department of Transportation for their assistance with filming the Portsmouth lift bridge. This work was supported, in part, by Shell Research, Quanta Computer, National Science Foundation Grants CGV-1111415 and CGV-1122374, and National Institutes of Health Grant R01-DC00238.