Analysis and Visualization of Temporal Variations in Video

PhD Thesis

Michael Rubinstein

Massachusetts Institute of Technology

February 2014

 

George M. Sprowls Award for outstanding doctoral thesis in Computer Science at MIT

 

Abstract

Our world is constantly changing, and it is important for us to understand how our environment changes and evolves over time. A common method for capturing and communicating such changes is imagery -- whether captured by consumer cameras, microscopes or satellites, images and videos provide an invaluable source of information about the time-varying nature of our world. Due to the great progress in digital photography, such images and videos are now widespread and easy to capture, yet computational models and tools for understanding and analyzing time-varying processes and trends in visual data are scarce and undeveloped.

In this dissertation, we propose new computational techniques to efficiently represent, analyze and visualize both short-term and long-term temporal variation in videos and image sequences. Small-amplitude changes that are difficult or impossible to see with the naked eye, such as variation in human skin color due to blood circulation and small mechanical movements, can be extracted for further analysis, or exaggerated to become visible to an observer. Our techniques can also attenuate motions and changes to remove variation that distracts from the main temporal events of interest.

The main contribution of this thesis is in advancing our knowledge on how to process spatiotemporal imagery and extract information that may not be immediately seen, so as to better understand our dynamic world through images and videos.

Thesis Committee: William T. Freeman (advisor), Fredo Durand, Ce Liu, Richard Szeliski

@phdthesis{RubinsteinPhDThesis2014,
  author = {Michael Rubinstein},
  title = {Analysis and Visualization of Temporal Variations in Video},
  school = {Massachusetts Institute of Technology},
  year = {2014},
  month = {Feb}
}

 

Thesis: PDF

Defense presentation: [PDF (10MB)] [PPT only (80MB)] [PPT with videos (1.3GB)]

For code and demos see the project web pages under Publications below.

 

Videos

Check out the following videos for a quick overview of this work:


Revealing Invisible Changes In The World
NSF Science and Engineering Visualization Challenge 2012		 Finding the Visible in the Invisible
Story in NY Times, Feb 2013		 Eulerian Video Magnification
SIGGRAPH'12 supplemental video		 Phase-based Video Motion Processing
SIGGRAPH'13 supplemental video

 

Publications

Phase-based Video Motion Processing
ACM Transactions on Graphics, Volume 32, Number 4 (Proc. SIGGRAPH), 2013.
[Paper] [Webpage] [BibTeX]
Patent pending

Revealing Invisible Changes In The World
Science Vol. 339 No. 6119, Feb 1 2013
[Article in Science] [Video] [NSF SciVis 2012] [BibTeX]

Eulerian Video Magnification for Revealing Subtle Changes in the World
ACM Transactions on Graphics, Volume 31, Number 4 (Proc. SIGGRAPH), 2012
[Paper] [Webpage] [BibTeX]
Patent pending

Motion Denoising with Application to Time-lapse Photography
IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), 2011
[Paper] [Webpage] [BibTeX]

 

Press

 

 

 

Last updated: Nov 2014