M.Eng - Monitoring Robotic Plan Execution
My Master of Engineering (M.Eng) thesis is concerned with robotic plan execution. I develop a plan execution and monitoring system known as Pike (Star Trek reference!) that, when given a plan for a robot, is able to automatically extract what features or states must hold true over time for the plan to succeed. These conditions are then monitored online, so that if some disturbance occurs that threatens the robot’s plan, it can be detected immediately.
As a concerete example, consider the case of a robot that is cooking dinner. The robot may be given a plan (i.e., a bunch of actions with timing information) for how to do this; an example might be something like “Preheat the oven to 350 degrees, and while that’s happening, take the chicken out of the fridge and put it in a pan, and then add spices. Next once the oven is hot, place the chicken in the oven for 1 hour. The entire task should take no longer than 1 hour and 30 minutes.” Please note that this plan is specified at a very high, cognitive level - we use actions as if we were speaking to a human. We don’t need to say things like “move your arm 90 degrees” or “close your gripper” - these are all low level commands. Given such a high-level plan that contains actions and temporal constraints, and also given a model of the world, the set of algorithms I developed for my thesis extract sets of conditions that must hold true at various times in the plan and what other actions made those conditions true. For example, one such condition may be that, while the robot is adding spices, the robot must remain in the pan on the counter. If the chicken falls on the floor, this presents an obvious problem. The set of algorithms I develop is able to reason over the temporal plan and world model to conclude that the chicken should stay in the pan until it is put in the oven, because the action of taking the chicken out of the refrigerator put it there. During execution, if this condition is violated, the robot’s plan is no longer valid - a new one is needed and can be obtained via a generative planner.
Algorithm Development
The algorithms I develop have two pieces: an offline and an online component. The offline component runs before the robot starts cooking dinner. This is what extracts the rationale, or “causal links” for the actions in the plan by reasoning about the temporal constraints and the world model (which in this thesis is a PDDL model for all actions). The offline algorithms hand this information to the online set of algorithms, which actively monitor the conditions to make sure they hold true during run-time by observing state estimates constructed (for example, by a hybrid estimation module) from sensor data.
A key idea introduced in this thesis is that, given any arbitrary temporal plan, there may not be a unique set of causal links that explain the rationale behind all actions. Rather, it may at times be ambiguous as to which action is the cause of which other actions. The approach we take is to extract sets of candidate causal links. At run time, once more information is known, decisions are made that activate exactly one of these candidate causal link for every required condition in the plan. These are then monitored online.
Please see my thesis for more details, proofs, and background information.
Experimental Validation via Simulator and Hardware Testbed
As you may have surmised from looking at some of my other projects, I am a very applied, hardware-loving person at heart. As such, a big part of my M.Eng was the implementation of these algorithms in a simulation environment built off of the OpenRAVE platform, as well as a hardware demonstration using a Barrett WAM arm. Please see my thesis for more details.
Videos
My lab’s collaborative manufacturing project has gotten some fun publicity! Please see the following links, each of which have videos:
- Discovery Channel segment (Daily Planet, a show aired in Canada)
- Engadget.com article
- Bloomberg BusinessWeek video
The work shown in these videos was done with several other grad students in my lab. The robotic execution monitor, integration with the WAM robot, and work with the OpenRAVE 3D simulator were my biggest contributions.
Paper
For more information, please see my thesis.
