Natural Language Understanding of Robot Instructions
After several years of studying how best to search the continuum of actions for the best possible motions, I became interested in how planning problems emerge from interactions and observations between the robot and the environment. Rather than rely on intermediate layers of autonomy architectures to translate instructions into constraints and cost functions, I am seeking to develop techniques that identify the formulation of trajectory planning problems directly from unstructured natural language. Recently I developed a technique that uses probabilistic graphical models trained with a corpus of labeled examples to determine the constraint set that best represents a given command, thereby avoiding computational bottlenecks imposed by search in a state-action space for the most probable path.
Selected Publications
F. Duvallet, M. Walter, T.M. Howard, S. Hemachandra, J. Oh, S. Teller, N. Roy, and A. Stentz, "A Probabilistic Framework for Inferring Maps and Behaviors from Natural Language", to appear in the Proceedings of the 14th International Symposium on Experimental Robotics, 2014.
T.M. Howard, S. Tellex, and N. Roy, "A Natural Language Planner Interface for Mobile Manipulators," to appear in the Proceedings of the 2014 International Conference on Robotics and Automation, 2014. [paper]
Several challenges for dexterous manipulation include coordination of many degrees of freedom, modeling of interactions between objects and end-effectors, and effective techniques for human-robot interaction. I applied the concepts behind model-predictive trajectory generation of mobile robots for mobile manipulators by augmenting the space of parameterized action sequences with degrees of freedom of the manipulator. This was used as part of a supervised teleoperation system for a mobile manipulator designed to improve the efficiency and reduce the cognitive load of human operators. I later extended this technique for coordinated planning of a robot arm, hand, and neck as motion planning lead of the JPL/Caltech team during Phase I of the DARPA Autonomous Robotic Manipulation program.
The fundamental challenge of robot navigation involves making optimal decisions under temporal and computational resource constraints. I began work in the topic by developing efficient algorithms for navigation search space design through intelligent sampling of boundary states in both constrained and unstructured environments. I applied this technique for autonomous in-lane navigation as a member of 
Robots need to understand the consequences of their own actions in order to make informed decisions about safety and optimality of candidate motions. My work in this area has primarily focused on the development and application of techniques that invert arbitrarily complex models of motion and environmental interaction, with particular attention to terrain shape and wheel-terrain interaction. I later demonstrated that this method could be applied for predictive compensation of impaired mobility through experiments of a planetary rover with a disabled drive motor. My thesis work presented an algorithm that adapts coarse recombinant motion planning search spaces to the local environment to improve the relative optimality and computational efficiency of planning in complex environments.