Research Overview
During my undergraduate and master's, I have had the opportunity of exploring research in machine learning and it's applications to Computer Vision/Graphics, NLP and Biology. Below are listed some of my works in these areas.
During my undergraduate and master's, I have had the opportunity of exploring research in machine learning and it's applications to Computer Vision/Graphics, NLP and Biology. Below are listed some of my works in these areas.
We have designed a procedural graphics pipeline using GIS data and 3D geometry to provide a controlled environment for vision, robotics and machine learning research. Our pipeline allows generating large scale data with fine control over parameters like object viewpoints, poses, materials, photometric parameters like light source distribution and intensity and other scene attributes. Currently, we are using this pipeline to quantify generalization behaviour of different neural net architectures and optimization strategies. By controlling the variation in the above mentioned parameters seen by a network during training, we can quantitiatively comment on how it generalizes beyond the training range when tested on greater variation.
We have developed an algorithm for non-parametric reservoir sampling: Given a stream of data, our algorithm samples points which match a seed, non parametric distribution (based on the MMD criterion). Using this algorithm, we can create conversational agents which build their own dataset. A user can begin by listing a set of words/phrases, and their word2vec representations serve as a non parametric distribution. Then, our system samples lines from internet, using our algorithm to retain lines which match the distribution. Microsoft Xbox is interested in using this system to create more immersive, interactive environments.
We Introduce a new dataset of infographics, and a bunch of tools for automated understanding of these images. We start by running topic models on text extracted from these infographics, to pin down the major concepts discussed in the image. Then, using these concepts as weak supervision, we used a multiple instance learning based CNN to extract regions in inforgraphics most representative of their content. In this manner, we can provide a textual and a visual summary of an infographic's content. Following up, we designed a synthetic data augmentation strategy for infographics, and used it to train a model to detect icons in infographics using the synthetic annotations only. Another journal paper from this work is currently under preparation and is soon to be submitted to IJCV.
We designed an automated system that can predict the relative importance of different elements in data visualizations and graphic designs. Our models are neural networks trained on human clicks and importance annotations on hundreds of designs. We collected a new dataset of crowdsourced importance, and analyzed the predictions of our models with respect to ground truth importance and human eye movements. Finally, we also demonstrated how such a system can help graphic designers create better designs by providing them real time feedback as they create.
We performed an analysis of how changing words in the title of a line graph can alter people's memory of the underlying trends in these graphs, concluding that title wording can indeed bias memory of a graph. For this, we designed a UI where the same graph was presented to m-Turkers with titles with slightly altered wordings. For ex: "Substantial increase" vs "steady increase" vs "subtle increase". Then the visual memory task asks users to select one of three trends with different slopes. People tended to pick steeper trends when more intense adjectives were used in the titles.
Cleaving DNA at a desired location is a fundamental problem in targeted gene editing. However, molecular scissors like zinc finger proteins are highly specific, and designing a zinc finger protein to cleave at a desired DNA sequence is slow and expensive experimental process. While chemical simulation and molecular docking methods exist, these are extremely slow to compute. We designed a novel ensemble micro neural network model, which are order of magnitude faster at this task. We achieve this by using an ensemble of 100 shallow neural networks, each with a different hidden architecture to capture different features from the small available dataset.
ATM security is a big challenge, given the widespread use and the large amounts of money involved. Instances of robberies and violence in ATM vestibules is a major safety concern, and traditionally the only solution is to have a security guard designated to the ATM. We designed a fast, ARM CPU compatible anomaly detection algorithm, which can detect anomalies from CCTV camera footage. Our algorithm runs on board on smart CCTVs, and can be used to automate ATM security.