Object Detection for Large-Scale Image Search
Internship at Microsoft Research New England - working with Ce Liu and Alex Berg
Today, most results on image search sites such as Bing and Google use meta-data primarily from the surrounding text and crowd voting to the rank results for a search query. Only simple filters - such as face detectors - are ever applied to the images to improve ranking results. However, it is often the case that images contain a lot of information that is never expressed in the text. This missing information could potentially help users in pruning their search query so that they can find what type of image they are looking for much more quickly. In this project we are trying to design scalable and efficient mulit-class detection algorithms to try to further understand the images. This information can then be used as meta-data for image search.
Estimating the Material Properties of Fabric from Observed Motion
MIT - working with Bill Freeman, Bei Xiao, and Peter Battaglia
This project focuses on learning the physical material properities of deformable objects by observing their movement under unknown forces. This work is motivated by the observation that humans seem to be able to understand the physical properties of the objects around them without ever directly interacting with them. Currently, we are focusing on learning the stiffness and mass of different types of cloth by observing their motion in 2D video.
Automatic Geo-Registration of Continuous Video using 3D Point Cloud Alignment
Internship at MIT Lincoln Laboratory - work with Karl Ni and Nadya Bliss
This project focused on creating a system that automatically geo-registers the content of 2D images from continuos video data. The system works by constructing a model of the 3D structure of a scene from identified corresponding points in stereo images. These stereo images are extracted from consecutive frames of the video data. Once the model has been created, the system is able to identify the GPS coordinates of objects within the scene by aligning the model’s point cloud to a previously collected reference point cloud. In our experimental data we choose to align our model point clouds with LIDAR data taken of the areas imaged.
Anomoly Detection in Carbon Dioxide Monitoring
University of Michigan - work with Clayton Scott and Long Nguyen
This project focused on detecting anomalies in spatio-temporal environmental data, such as carbon-dioxide flux maps. The challenge in this work that there are no labeled examples of anomolies in the dataset. Thus, our goal was to learn a spatial and temporal model that would follow trends in the carbon-dioxide flux map over the seasons. This model would allow us to detect anomolies in the data.
The Rosetta Phone
Purdue University - work with Edward J. Delp, Mireille Boutin, and Golnaz Abdollahian
The Rosetta Phone project focused on developing an application for handheld mobile imaging devices that captures an image of text in a selected language and then translates the text into written and spoken English. While text-to-text translation with current technology is relatively easy for Latin characters, translation from languages such as Chinese and Urdu is very difficult without a knowledgeable user and a sophisticated keyboard. The automatic localization of text within a natural image is an important problem for this application. Once identified, the text can be analyzed, recognized, and interpreted. However, many objects in natural images, such as tree branches or electrical wires, are easily confused for text by existing optical character recognition (OCR) algorithms. For this reason, applying OCR on an unprocessed natural image is computationally expensive and may produce erroneous results. Hence, robust and efficient methods are needed to identify the text-containing regions within natural images before performing OCR. I worked on developing a low complexity method for sign detection and text localization in natural images. This method is designed for mobile applications (e.g., unmanned or handheld devices) in which computational and energy resources are limited. No prior assumption is made regarding the text size, font, language, or character set. However, the text is assumed to be located on a homogeneous background using a contrasting color.
Identifying Nanoparticles Through Light Microscopy
University of Michigan - work with Jingyong Ye, and Chuck Divin
This project focused on reducing the limitations in optical microscopy through the use of light interference patterns. By illuminating a sample with two angled beams of coherent light, a nanoparticle’s signal can be identified and extracted from the combined signal. The developed algorithm proved to have a higher performance than other methods in detecting nanoparticles smaller than a wavelength of light.
Digital Image Forensics
Purdue University - work with Edward J. Delp
As result of the growing use of digital cameras, methods to identify an image's origin quickly, reliably, and inexpensively are needed. By designing and implementing methods for mitigating or "attacking" forensic features in the images, the robustness of various forensic identification techniques can be analyzed. The objective of this research was to see if drastically modified images could still be identified using a camera’s defects in the form of a reference noise pattern.
