About me

I am a Co-Founder of GelSight, Inc. — a startup company which is commercializing a technology that I helped develop while I was a postdoc at MIT. Prior to starting the company, I was a Research Scientist in CSAIL working with Bill Freeman, and before that a postdoc working with Ted Adelson in the department of Brain and Cognitive Sciences. My main research interest is computer vision, with an emphasis on understanding the various properties of image appearance. My research is interdisciplinary and the tools I have developed are being used in a variety of domains, including scientific measurement, art analysis, forensic analysis, cultural heritage, and industrial inspection.

Research summary

Sophisticated image analysis and editing is challenging due to the complexities of image appearance. I have worked on a variety of projects that measure or manipulate different aspects of appearance. These projects can be grouped into three broad themes: measuring shape and shading, improving the realism of fake images, and image forensics.

Measuring shape and shading

Shape, illumination and albedo are three factors that contribute to image appearance. As part of my Ph.D., I developed tools for measuring illumination in images (TIFS 2007). These tools could be used to detect tampering in images. Since then, I have worked on a method for measuring high-resolution surface shape (CVPR 2009) and am currently exploring how these detailed measurements contribute to appearance. I have also created a dataset of high-quality shading and reflectance images that can be used as training or testing data for the intrinsic image problem (ICCV 2009). I am currently working on methods for estimating shape under natural illumination and exploring how illumination and material affect shape estimation.

Improving realism of fake images

Often, when a factor of appearance can be measured, it can be manipulated to produce new or improved images. Together with collaborators from Adobe Systems and Harvard University, I have developed methods for improving the realism of image composites and computer-generated images. At SIGGRAPH 2010, we presented a framework that matches contrast, texture, noise, and blur when compositing images. I also helped with a method for reducing artifacts that occur during gradient-domain image processing (ECCV 2010). In related work, I have explored methods for enhancing the realism of computer-generated images using textures and colors from natural images (TVCG 2011) and have helped to adapt these methods for automatic image restoration (ICCV 2009). I am currently extending a few of these methods to image sequences and video.

Image forensics

With a digital camera, personal computer, and image editing software, almost anyone can make a digital forgery. It is not surprising, then, that tampered images have been discovered everywhere from magazine covers to scientific publications. For my Ph.D., I helped develop digital forensic techniques to address this growing problem. In addition to illumination estimation, I explored methods for exposing forgeries by analyzing specular highlights and geometry of the eye (IH 2007, IWDW 2007) and through local measurements of chromatic aberration (ACM MM&Sec 2006). I am currently exploring how information gathered from large image databases can be used for forensic tasks.

Collaborators

My current collaborators include: