Human-in-the-Loop Machine Learning

Robots and AI-enabled systems maintain an understanding of the world through mathematical models. Manual specification of these models is incomplete and error-prone, while existing learning algorithms demand a large amount of labeled data. Despite the emphasis on big data, in many settings (including human-machine interaction) data is scarce. To enable learning in such settings, we must shift to a hybrid paradigm, where learning is done both from data and human feedback.

Building on this insight, I am developing interactive approaches that allow robots/algorithms to ask questions and humans to provide feedback while learning models. In addition to enabling human-machine interaction in new settings, this research has broad applications for learning generative models, transferring agent experience from simulations to the real world, and for enabling the use of learning algorithms by non-programmers.

Related Publications

Unhelkar*, V. V., S. Li*, and J. A. Shah, Semi-Supervised Learning of Decision-Making Models for Human-Robot Collaboration , Conference on Robot Learning (CoRL), 2019.
Unhelkar, V. V., and J. A. Shah, Learning Models of Sequential Decision-Making with Partial Specification of Agent Behavior, AAAI Conference on Artificial Intelligence (AAAI), Honolulu, Hawaii, 2019.
Unhelkar, V. V., and J. A. Shah, Learning Models of Sequential Decision-Making without Complete State Specification using Bayesian Nonparametric Inference and Active Querying, MIT CSAIL Tech Report, 2018.




Decision-Making for Interaction

Along with models, robots and AI-enabled systems need algorithms to decide how to act and communicate while interacting with humans. Timely decisions are critical, and agents cannot directly observe human’s mental states. Gauging the right level of communication is challenging. Thus, interaction necessitates novel algorithms that can reason with incomplete models, partial observability, and limited planning time. I have developed execution-time algorithms that enable collaborative robots (a) to make communication decisions, and (b) efficiently use predictive information (available from machine learning models) to modify plans.

Related Publications

Unhelkar*, V. V., P. A. Lasota*, Q. Tyroller, R-D. Buhai, L. Marceau, B. Deml, and J. A. Shah, Human-Aware Robotic Assistant for Collaborative Assembly: Integrating Human Motion Prediction With Planning in Time, IEEE Robotics and Automation Letters (RA-L), vol. 3, issue 3, pp. 8, 2018.
Unhelkar, V. V., X. Jessie Yang, and J. A. Shah, Challenges for Communication Decision-Making in Sequential Human-Robot Collaborative Tasks, Robotics: Science and Systems (RSS), Workshop on Mathematical Models, Algorithms, and Human-Robot Interaction, 2017.
Unhelkar, V. V., and J. A. Shah, ConTaCT : Deciding to Communicate During Time-Critical Collaborative Tasks in Unknown, Deterministic Domains, AAAI Conference on Artificial Intelligence (AAAI), 2016.
Unhelkar, V. V., R-D. Buhai, and J. A. Shah, Planning in Dynamic Environments using Evolving Time-Indexed Graphs, Robotics: Science and Systems (RSS), Workshop on Planning for Human-Robot Interaction, 2016.




Introducing Robots among Humans

I believe introducing systems among human users is essential to understand and solve the (often unforeseen) challenges of human-machine interaction. In my research, I have also designed robotic systems, introduced them among humans, and evaluted their performance through human subject experiments. For instance, I developed a mobile robot capable of safely sharing space with humans, completing assembly tasks, and navigating moving floors of automotive factories. This system is the first mobile robot be introduced alongside humans in a live automotive assembly line.

Related Publications

Unhelkar, V. V., S. Dörr, A. Bubeck, P. A. Lasota, J. Perez, H. Chit Siu, J B. C. Jr., Q. Tyroller, J. Bix, S. Bartscher, et al., Mobile Robots for Moving-Floor Assembly Lines: Design, Evaluation, and Deployment, IEEE Robotics and Automation Magazine (RAM), vol. 25, issue 2, 2018.
Unhelkar*, V. V., P. A. Lasota*, Q. Tyroller, R-D. Buhai, L. Marceau, B. Deml, and J. A. Shah, Human-Aware Robotic Assistant for Collaborative Assembly: Integrating Human Motion Prediction With Planning in Time, IEEE Robotics and Automation Letters (RA-L), vol. 3, issue 3, pp. 8, 2018.
Yang, X. Jessie, V. V. Unhelkar, K. Li, and J. A. Shah, Evaluating Effects of User Experience and System Transparency on Trust in Automation, ACM/IEEE International Conference on Human Robot Interaction (HRI), 2017.
Unhelkar, V. V., H. Chit Siu, and J. A. Shah, Comparative Performance of Human and Mobile Robotic Assistants in Collaborative Fetch-and-Deliver Tasks, ACM/IEEE International Conference on Human Robot Interaction (HRI), 2014.