Jeremy Scott
| phone: | 617-758-9579 |
| email: | jscott@csail.mit.edu |
| office: | 32-239 |
| resume/CV |
about
I'm a second-year graduate student in computer science at MIT. I'm researching the intersection of AI and HCI with Randall Davis in the Multimodal Understanding Group in the Computer Science and Artificial Intelligence Laboratory (CSAIL). I earned my Bachelor of Applied Science degree in the Engineering Science program at the University of Toronto. For my undergraduate thesis, I explored foot-based interaction techniques working with Khai Truong in the Toronto Ubiquitous Computing Research Group (TorURG).
research
My Master's thesis is to build a system that understands sketch-and-speech descriptions of physical machines. From an HCI perspective, this is motivated by the fact that despite advances in both interaction technologies (Kinect, multi-touch) and software (CAD, physics simulators, Flash), there is no easy way to describe behavior, as well as structure and appearance, without getting buried in parameters and explicit models. We still (literally) go back to the drawing board as a communication and learning tool.
From a human intelligence perspective, it is interesting to ask how we store and retrieve knowledge of physical behavior. For example, consider the event: the ball bounces off the backboard and into the hoop. What representations are necessary for storing both visual data (sketch) and language (speech) that describe behavior? How can we test that the system understands the physical constraints of objects involved (the ball can move, but the hoop cannot)? How do we evaluate its understanding of language semantics (through a hoop) and causal relations between events (a bounce causing a change in trajectory)?
By building an intelligent whiteboard that interprets sketch-and-speech descriptions of behavior, I hope to tackle some of the usability concerns and AI-related questions associated with the task. The system demonstrates its understanding of constraints on sketched objects by making them manipulable in a constrained way. The user can then give examples of physical behavior by moving objects in the sketch. The system then demonstrates its understanding of events and their causal relations by adapting to new scenarios presented by the user: What happens if I move the hoop and backboard farther away from the ball? What happens if I take it away entirely?
publications
Sensing Foot Gestures From The Pocket (in TorURG at University of Toronto)
Scott, J., Dearman, D., Yatani, K. and Truong, K.N. 2010. Sensing Foot Gestures from the Pocket. In Proceedings of UIST 2010: The 23rd ACM Symposium on User Interface Software and Technology. (New York, NY, USA, October 3 - 6, 2010). ACM, New York, NY. [18.4% acceptance rate] PDF
past work
SkinMetrics (in the Artificial Perception Lab at University of Toronto)
Advisor: Parham Aarabi
While working in the Artificial Perception Lab (APL) in the summer of 2007, I developed early versions of an image processing algorithm to analyze skin quality from the image of a face. Using edge detection, the software detected anomalies, such as wrinkles or lesions. The algorithm has since been integrated into other APL research and a ModiFace application called SkinMetrics.
Autonomous Inventory and Liquid Level Detection Robot (in AER201, Engineering Science)
In EngSci's AER201 Engineering Design course, I worked with a world-class team of engineers to build an autonomous robot that could detect the presence of oil barrels and liquid levels inside them.
Jeremy Scott (jks@mit.edu)
