Rob Fergus - Research

Overview

I am interested in modeling the statistics of images, from high-level representations of scenes and objects to low-level cues such as image gradients. Such models may be used for a range of applications within Computer Vision and Computational Photography. These include object recognition or removing blur from photos corrupted by camera shake. Below are brief descriptions of my research, along with links to pages giving more details.

Image Deblurring Link

Many photos are spoiled by the user's hand moving while the camera shutter is open. Points in the scene are smeared out over the exposure interval, resulting in a blurry photo. My co-authors and I pose the problem as a blind deconvolution: we assume the blur function is constant over the image and so aim to recover the blur kernel (the motion of the user's hand) together with the underlying sharp image. We make use of heavy-tailed image priors on the image gradients in conjunction with some sophisticated machine learning tools to solve the blind deconvolution problem. We apply the algorithm to real photos, obtaining what we believe are the first convincing results on this difficult problem.

Object Recognition Link

One of my main areas of research is Object Recognition. Here the goal is to give computers the ability to "see" just as humans do.  A computer should be able to know where it is and what surrounds it just by looking. Today, cameras are ubiquitous but we lack the computational algorithms to process the images into more useful representations.

I have focused on the problem of recognizing object categories. While there are now viable methods for finding specific objects (e.g. a can of Coke) in images, the more general problem of finding categories of objects (e.g. all cans of soda) is harder. My co-authors and I have proposed various probabilistic representations which can be used in conjunction with machine learning methods to learn object models from a set of images containing the desired class of object. This model can then be used to recognize instances of the class in novel images. 

Leveraging the Internet for Object Recognition Link

The Internet is an incredibly rich resource of information that I am interested in using in conjunction with object recognition algorithms. It can provide a diverse set of images from which object category models may be trained. In turn, these models may be applied to collections of images from the Internet or elsewhere, enabling search by visual content (known as content-based image retrieval, CBIR), rather than the text-based searches that are currently employed.