Learning from demonstrations has been shown to be a successful method for non-experts to teach manipulation tasks to robots. These methods typically build generative models from demonstrations and then use regression to reproduce skills. However, this approach has limitations to capture hard geometric constraints imposed by the task. On the other hand, while sampling and optimization-based motion planners exist that reason about geometric constraints, these are typically carefully hand-crafted by an expert. To address this technical gap, we contribute with C-LEARN, a method that learns multi- step manipulation tasks from demonstrations as a sequence of keyframes and a set of geometric constraints. The system builds a knowledge base for reaching and grasping objects, which is then leveraged to learn multi-step tasks from a single demonstration. C-LEARN supports multi-step tasks with multiple end effectors; reasons about SE(3) volumetric and CAD constraints, such as the need for two axes to be parallel; and offers a principled way to transfer skills between robots with different kinematics. We embed the execution of the learned tasks within a shared autonomy framework, and evaluate our approach by analyzing the success rate when performing physical tasks with a dual-arm Optimus robot, comparing the contribution of different constraints models, and demonstrating the ability of C-LEARN to transfer learned tasks by performing them with a legged dual-arm Atlas robot in simulation.

Claudia Pérez-D'Arpino, Julie A. Shah. C-LEARN: Learning Geometric Constraints from Demonstrations for Multi-Step Manipulation in Shared Autonomy. 2017 IEEE International Conference on Robotics and Automation (IEEE ICRA 2017), 2017.

Interest in human-robot coexistence, in which humans and robots share a common work volume, is increasing in manufacturing environments. Efficient work coordination requires both awareness of the human pose and a plan of action for both human and robot agents in order to compute robot motion trajectories that synchronize naturally with human motion. In this paper, we present a data-driven approach that synthesizes anticipatory knowledge of both human motions and subsequent action steps in order to predict in real-time the intended target of a human performing a reaching motion. Motion-level anticipatory models are constructed using multiple demonstrations of human reaching motions. We produce a library of motions from human demonstrations, based on a statistical representation of the degrees of freedom of the human arm, using time series analysis, wherein each time step is encoded as a multivariate Gaussian distribution. We demonstrate the benefits of this approach through offline statistical analysis of human motion data. The results indicate a considerable improvement over prior techniques in early prediction, achieving 70% or higher correct classification on average for the first third of the trajectory. We also indicate proof-of-concept through the demonstration of a human-robot cooperative manipulation task performed with a PR2 robot. We analyze the quality of task-level anticipatory knowledge required to improve prediction performance early in the motion trajectory.

Claudia Pérez-D'Arpino, Julie A. Shah. Fast Target Prediction of Human Reaching Motion for Cooperative Human-Robot Manipulation Tasks Using Time Series Classification. 2015 IEEE International Conference on Robotics and Automation (ICRA 2015).

Mobile, interactive robots that operate in human-centric environments need the capability to safely and efficiently navigate around humans. This requires the ability to sense and predict human motion trajectories and to plan around them. In this paper, we present a study that supports the existence of statistically significant biomechanical turn indicators of human walking motions. Further, we demonstrate the effectiveness of these turn indicators as features in the prediction of human motion trajectories. Human motion capture data is collected with predefined goals to train and test a prediction algorithm. Use of anticipatory features results in improved performance of the prediction algorithm. Lastly, we demonstrate the closed- loop performance of the prediction algorithm using an existing algorithm for motion planning within dynamic environments. The anticipatory indicators of human walking motion can be used with different prediction and/or planning algorithms for robotics; the chosen planning and prediction algorithm demonstrates one such implementation for human-robot co-navigation.

Vaibhav V. Unhelkar, Claudia Pérez-D'Arpino, Leia Stirling, Julie A. Shah. Human-Robot Co-Navigation Using Anticipatory Indicators of Human Walking Motion. 2015 IEEE International Conference on Robotics and Automation (ICRA 2015).

This paper presents the development of an inertial measurement unit (IMU) specially designed for unmanned aerial vehicles (UAV) applications. The design was intended to be a low cost solution of high performance for robotic applications using 3-axis accelerometers and 3-axis gyroscopes MEMS sensors. We present simultaneous sampling to avoid the loss of orthogonality of the inertial measurements due to multiplexed data acquisition commonly used in low cost IMUs, as well as anti-aliasing processing. The IMU was implemented in two boards to separate the sensors from the processing hardware in order to be able to use it with different sets of sensors. The sensor fusion algorithm for attitude determination is based on the Kalman Filter. As testing process, the IMU was installed in a 2-DOF helicopter and the results were compared with those obtained from the encoders for the pitch and roll angles. We also present the results of the IMU installed in a T-REX 450 scale helicopter inside a motion analysis laboratory, using a custom-design safety stand that supports the helicopter allowing only its rotational 3-DOF (roll, pitch and yaw movements). Those demanding experiences tested the IMU performance under true UAV conditions and the results exhibited a maximum RMS error of 4°.

@INPROCEEDINGS{5753503,
author={Pérez-D'Arpino, C. and Vigouroux, D. and Medina-Meléndez, W. and Fermín, L. and Torrealba, R.R. and Grieco, J.C. and Fernández-López, G.},
booktitle={Technologies for Practical Robot Applications (TePRA), 2011 IEEE Conference on}, title={Development of a low cost inertial measurement unit for UAV applications with Kalman Filter based attitude determination},
year={2011},
pages={178-183} }

Bilateral Control by States Convergence is a novel and little exploited control strategy that has been successfully applied to the teleoperation of robotic manipulators using SISO control. Based on the state space representation, the main philosophy of this control strategy consists in achieving convergence of states between the master and the slave, by setting the dynamical behavior of the master-slave error as a states-independent autonomous system. This paper presents a generalization of this strategy to MIMO systems, with time delay in the master-slave communication channels. It is demonstrated how the feedback gains required by the state convergence control schema can be found by solving a set of ((m × n) + (m × m) + 2n) nonlinear equations for a system with m inputs, n states and m outputs. Unlike previous research, which has been applied only to manipulators considering 1-DOF for the states-convergence control loop, the extension to the general MIMO case has allowed to apply the technique to the teleoperation of a 2-DOF helicopter. A decoupling network and states-feedback are used for local control, while the statesconvergence control manages the bilateral issues. Simulation results are presented, showing a satisfactory performance of the control strategy.

@INPROCEEDINGS{5509910,
author={Pérez-D'Arpino, C. and Medina-Meléndez, W. and Fermín-León, L. and Bogado, J.M. and Torrealba, R.R. and Fernández-López, G.},
booktitle={Robotics and Automation (ICRA), 2010 IEEE International Conference on},
title={Generalized Bilateral MIMO Control by States Convergence with time delay and application for the teleoperation of a 2-DOF helicopter},
year={2010},
pages={5328-5333},
doi={10.1109/ROBOT.2010.5509910},
ISSN={1050-4729},}

This paper presents the development of a biomechatronic knee prosthesis for transfemoral amputees. This kind of prostheses are considered ‘intelligent’ because they are able to automatically adapt their response at the knee axis, as a natural knee does. This behavior is achieved by characterizing the amputee’s gait through the signals captured with instrumentation of a prosthesis, which provides feedback about its current state along the gait cycle and therefore responds with the corresponding control action. In this case, unlike other commercially available intelligent knee prostheses, gait cycle characterization is based on accelerometers signals processed by an events detection algorithm. Two intelligent control strategies are presented: a bio-inspired approach, that consists of using a central pattern generator to generate a knee angle reference to be followed by the prosthesis during walking, and an adaptive scheme, that applies a control action proportional to the knee angle according to an auto-adaptive parameter dependant on gait speed. The mechanical design of the prosthesis is also presented, showing the knee joint mechanism and part of the manufacturing process. Results obtained from walking tests with both able body and amputees are shown, demonstrating the positive performance of the prosthesis in several aspects. Future works aimed at a finished product are also stated.

@INPROCEEDINGS{5509674,
author={Torrealba, R.R. and Pérez-D'Arpino, C. and Cappelletto, J. and Fermín-León, L. and Fernández-López, G. and Grieco, J.C.},
booktitle={Robotics and Automation (ICRA), 2010 IEEE International Conference on},
title={Through the development of a biomechatronic knee prosthesis for transfemoral amputees: Mechanical design and manufacture, human gait characterization, intelligent control strategies and tests},
year={2010},
pages={2934-2939},
doi={10.1109/ROBOT.2010.5509674},
ISSN={1050-4729},}

This paper presents a complete artificial vision system development and implementation for a mobile robot recognition and obstacles detection, which integrates a statistical segmentation method with frequency filtering in order to achieve luminosity independence, exploiting the advantages of known image processing techniques by mixing them into a robotic application. The system proposed determines the mobile robot's position and orientation, using a color segmentation approach based on the Mahalanobis Distance, and the position and size of obstacles in the robot's environment using a parallel scheme based on both Sobel edge detector and Otsu's threshold. The Mahalanobis distance calculus was implemented using a real-time FPGA architecture, in order to detect the robot's position. Tests in the real robot's environment are presented obtaining results that are independent from the background characteristics and strongly robust on the luminosity variability.

@inbook{doi:10.1142/9789814291279_0134,
author = {CLAUDIA PÉREZ-D'ARPINO and WILFREDIS MEDINA-MELÉNDEZ and DIMITAR RALEV and JOSÉ GUZMÁN and LEONARDO FERMÍN and JUAN CARLOS GRIECO and GERARDO FERNÁNDEZ-LÓPEZ and MANUEL ARMADA},
title = {FPGA-BASED ARTIFICIAL VISION SYSTEM FOR ROBOT AND OBSTACLES DETECTION UNDER STRONG LUMINOSITY VARIABILITY},
booktitle = {Mobile Robotics},
pages = {1099-1109},
doi = {10.1142/9789814291279_0134},
year = 2012
}

This paper describes the detection of the velocities components of a 6-DOF mobile platform for control applications using an optical flow based approach. The proposed scheme consists in processing the estimated optical flow from one camera attached to the platform, preferably pointing to the floor in order to handle the distance effect. This flow is projected onto the pattern associated with each DOF velocity component, and with the value of these projections, it is mathematically demonstrated that the velocities can be extracted. A synthetic flow was generated for the algorithm testing, and the velocities in each DOF were successful recovered. Different levels of noise are added to the synthetic flow to test the algorithm performance under noisy conditions proving to be a reliable method.

@Inbook{Fermín-León2009, author="Ferm{\'i}n-Le{\'o}n, Leonardo and Medina-Mel{\'e}ndez, Wilfredis and P{\'e}rez-D'Arpino, Claudia and Grieco, Juan C.",
title="Estimation of Velocity Components Using Optical Flow and Inner Product",
bookTitle="Robot Motion and Control 2009",
year="2009",
pages="349--358",
doi="10.1007/978-1-84882-985-5_32",
url="http://dx.doi.org/10.1007/978-1-84882-985-5_32" }

Velocity fields techniques are often used in the path planning and control problem in autonomous navigation. Velocity fields are compounded by vectors that can be expressed in polar form, i.e. Vx = |V |·cos(α) and Vy = |V |·sin(α). In the proposed technique, the problem is divided into two separate problems: the calculation of the orientation angle α, which was treated in a previous work, and the calculation of the linear velocity |V |. This paper addresses the calculation of |V | given a trajectory vector field that encodes α, thus generating the speed reference for a Velocity Field Controller. The linear velocity |V | is generated through a Mamdani-type Fuzzy Inference System, that considers the curl of the trajectory vector field, and an extension of the Voronoi Diagram. Results obtained in both simulations and tests with a real robot show that an adequate intelligent velocity behavior, and a smooth obstacle avoidance are achieved.

@INPROCEEDINGS{4957170,
author={Pérez-D'Arpino, C. and Medina-Meléndez, W. and Guzman, J. and Fermín, L. and Fernández-López, G.},
booktitle={Mechatronics, 2009. ICM 2009. IEEE International Conference on},
title={Fuzzy logic based speed planning for autonomous navigation under Velocity Field Control},
year={2009},
pages={1-6},
doi={10.1109/ICMECH.2009.4957170},}

In this work, a strategy for the generation of the angle reference for a Velocity Field Controller is presented. For an arbitrary trajectory given in its parametric form, it is possible to create a desired velocity field, that could be modified if obstacles were detected by an artificial vision system. Velocity fields are composed by vectors that can be expressed in polar form, i.e. Vx = |V |·cos(α) and Vy = |V |·sin(α). In the velocity field generation technique proposed, the problem is divided into two separate problems: the calculation of the orientation angle α and the calculation of the linear velocity |V |. This paper addresses the calculation of α, using the hydrodynamics theory of an incompressible non-viscous fluid, plus the use of conformal mapping, thus generating a trajectory vector field that can evade obstacles. Results obtained show that a smooth and continuous field for both open and closed trajectories is achieved.

@Inbook{Pérez-D’Arpino2008,
author="P{\'e}rez-D'Arpino, Claudia and Medina-Mel{\'e}ndez, Wilfredis and Ferm{\'i}n, Leonardo and Guzm{\'a}n, Jos{\'e} and Fern{\'a}ndez-L{\'o}pez, Gerardo and Grieco, Juan Carlos",
title="Dynamic Velocity Field Angle Generation for Obstacle Avoidance in Mobile Robots Using Hydrodynamics",
bookTitle="Advances in Artificial Intelligence -- IBERAMIA 2008: 11th Ibero-American Conference on AI, Lisbon, Portugal, October 14-17, 2008. Proceedings",
year="2008",
pages="372--381",
url="http://dx.doi.org/10.1007/978-3-540-88309-8_38" }

Since DC motors are wide used in the industry, many applications control theory has been used. As an alternative the following work presents the design of a fuzzy variable structure control for the speed control of a DC motor. This strategy control combines fuzzy and sliding mode control algorithms. Few fuzzy rules are used for this scheme. The parameters of the fuzzy controller are adjusted by using the VSC for a better response and robustness. The simulations results with the control scheme presented here offer higher robustness and is easier to adjust compared with classical fuzzy controllers.

@INPROCEEDINGS{4605182,
author={Perez, C. and Strefezza, M.},
booktitle={Control Conference, 2008. CCC 2008. 27th Chinese},
title={Speed control of a DC motor by using fuzzy variable structure controller},
year={2008},
pages={311-315},
keywords={DC motors;fuzzy control;variable structure systems;velocity control;DC motor;control theory;fuzzy rules;fuzzy variable structure controller;sliding mode control;speed control;Control systems;Control theory;DC motors;Error correction;Fuzzy control;Robust control;Sliding mode control;Torque;Uncertainty;Velocity control;DC motor;Fuzzy control;Variable structure controller},
doi={10.1109/CHICC.2008.4605182},}

Operating a high degree of freedom mobile manipulator, such as a humanoid, in a field scenario requires constant situational awareness, capable perception modules, and effective mechanisms for interactive motion planning and control. A well-designed operator interface presents the operator with enough context to quickly carry out a mission and the flexibility to handle unforeseen operating scenarios robustly. By contrast, an unintuitive user interface can increase the risk of catastrophic operator error by overwhelming the user with unnecessary information. With these principles in mind, we present the philosophy and design decisions behind Director—the open-source user interface developed by Team MIT to pilot the Atlas robot in the DARPA Robotics Challenge (DRC). At the heart of Director is an integrated task execution system that specifies sequences of actions needed to achieve a substantive task, such as drilling a wall or climbing a staircase. These task sequences, developed a priori, make online queries to automated perception and planning algorithms with outputs that can be reviewed by the operator and executed by our whole-body controller. Our use of Director at the DRC resulted in efficient high-level task operation while being fully competitive with approaches focusing on teleoperation by highly trained operators. We discuss the primary interface elements that comprise Director, and we provide an analysis of its successful use at the DRC.

@article {ROB:ROB21681,
author = {Marion, Pat and Fallon, Maurice and Deits, Robin and Valenzuela, Andrés and Pérez D'Arpino, Claudia and Izatt, Greg and Manuelli, Lucas and Antone, Matt and Dai, Hongkai and Koolen, Twan and Carter, John and Kuindersma, Scott and Tedrake, Russ},
title = {Director: A User Interface Designed for Robot Operation with Shared Autonomy},
journal = {Journal of Field Robotics},
volume = {34},
number = {2},
issn = {1556-4967},
url = {http://dx.doi.org/10.1002/rob.21681},
doi = {10.1002/rob.21681},
pages = {262--280},
year = {2017}, }

The DARPA Robotics Challenge Trials held in December 2013 provided a landmark demonstration of dexterous mobile robots executing a variety of tasks aided by a remote human operator using only data from the robot's sensor suite transmitted over a constrained, field-realistic communications link. We describe the design considerations, architecture, implementation, and performance of the software that Team MIT developed to command and control an Atlas humanoid robot. Our design emphasized human interaction with an efficient motion planner, where operators expressed desired robot actions in terms of affordances fit using perception and manipulated in a custom user interface. We highlight several important lessons we learned while developing our system on a highly compressed schedule.

@article {ROB:ROB21546,
author = {Fallon, Maurice and Kuindersma, Scott and Karumanchi, Sisir and Antone, Matthew and Schneider, Toby and Dai, Hongkai and D'Arpino, Claudia Pérez and Deits, Robin and DiCicco, Matt and Fourie, Dehann and Koolen, Twan and Marion, Pat and Posa, Michael and Valenzuela, Andrés and Yu, Kuan-Ting and Shah, Julie and Iagnemma, Karl and Tedrake, Russ and Teller, Seth},
title = {An Architecture for Online Affordance-based Perception and Whole-body Planning},
journal = {Journal of Field Robotics},
volume = {32},
number = {2},
issn = {1556-4967},
url = {http://dx.doi.org/10.1002/rob.21546},
doi = {10.1002/rob.21546},
pages = {229--254},
year = {2015},
}

Learning from both observations and self-exploration is a key capability of intelligent agents. It is present to various degrees in a number of animals that can combine information distilled from observing a task being performed with further knowledge acquired through self-exploration that contains variations over the demonstrations. We explore this mechanism in robotic arms in the context of learning to execute multi-step manipulation tasks. Specifically, the system accumulates geometric information on how humans typically manipulate objects with simple geometric shapes through learning from demonstrations. This geometric knowledge base is leveraged by a planner to find manipulation strategies given a novel scene. In addition, the new environment presents uncertainty in the values of the physical properties of the objects, such as their mass. We present an agent that has access to a simulator of the environment, instantiated by a physics engine, and uses it to explore multiple manipulation strategies and find successful sequences using Monte Carlo Tree Search. The search runs multiple forward simulations of the novel scene by hypothesizing plausible values for the unknown parameters of the mass of the objects. This abstract presents a preliminary implementation and results of this system, in which a dual-arm 16-DOF manipulator finds novel manipulation strategies by combining a geometric knowledge base learned from demonstrations with self-exploration in hypothesized simulated worlds in a simple task.

@inproceedings{DArpino_WorkshopNeurIPS18_LearnPhysics,
author = {P{\'e}rez-D'Arpino, Julie A. Shah},
title = {Observational and self-learning of multi-step robotic manipulation with unknown physical properties},
booktitle = {Workshop on Modeling the Physical World: Perception, Learning, and Control. 32nd Conference on Neural Information Processing Systems (NeurIPS 2018)},
year = {2018},
}

We present the deployment of a 16-DoF dual-arm mobile manipulator as an on-stage actor in the MIT2016 Pageant, a 60 minute live play performed for the centennial celebration of the Massachusetts Institute of Technology campus move from Boston to Cambridge. The robot performed using expressive motions, navigated a 250ft-long thrust stage through a wireless connection, and was directed remotely by a human operator using a shared autonomy system. We report on the technical framework and human-robot interaction that enabled the performance, including motion planning, coordination of action with human actors, and the challenges in navigation, manipulation, perception and system reliability.

@inproceedings{D'Arpino_HRI2017_Theatrical_Robot,
author = {P{\'e}rez-D'Arpino, Claudia and Agoos, Peter and Zamore, Andrew and Gammons, David R. and Kyritsis, Evie and Shah, Julie A.},
title = {A Theatrical Mobile-Dexterous Robot Directed Through Shared Autonomy},
booktitle = {Proceedings of the Companion of the 2017 ACM/IEEE International Conference on Human-Robot Interaction},
series = {HRI '17},
year = {2017},
location = {Vienna, Austria},
pages = {416--416},
url = {http://doi.acm.org/10.1145/3029798.3036644},
acmid = {3036644},
publisher = {ACM},
address = {New York, NY, USA},
keywords = {field robotics, human-robot interaction, mobile manipulator, robotics, shared autonomy, theatrical robot}, }

The paper describes the system developed by researchers from MIT for the Defense Advanced Research Projects Agency's (DARPA) Virtual Robotics Challenge (VRC), held in June 2013. The VRC was the first competition in the DARPA Robotics Challenge (DRC), a program that aims to "develop ground robotic capabilities to execute complex tasks in dangerous, degraded, human-engineered environments". The VRC required teams to guide a model of Boston Dynamics' humanoid robot, Atlas, through driving, walking, and manipulation tasks in simulation. Team MIT's user interface, the Viewer, provided the operator with a unified representation of all available information. A 3D rendering of the robot depicted its most recently estimated body state with respect to the surrounding environment, represented by point clouds and texture-mapped meshes as sensed by on-board LIDAR and fused over time.

@INPROCEEDINGS{6907140,
author={Tedrake, R. and Fallon, M. and Karumanchi, S. and Kuindersma, S. and Antone, M. and Schneider, T. and Howard, T. and Walter, M. and Dai, H. and Deits, R. and Fleder, M. and Fourie, D. and Hammoud, R. and Hemachandra, S. and Ilardi, P. and Perez-D'Arpino, C. and Pillai, S. and Valenzuela, A. and Cantu, C. and Dolan, C. and Evans, I. and Jorgensen, S. and Kristeller, J. and Shah, J.A. and Iagnemma, K. and Teller, S.},
booktitle={Robotics and Automation (ICRA), 2014 IEEE International Conference on},
title={A summary of team MIT's approach to the virtual robotics challenge},
year={2014},
month={May},
pages={2087-2087},
doi={10.1109/ICRA.2014.6907140},}

The efficient and safe performance of collaborative robots requires advancements in perception, control, design and algorithms, among other factors. With regard to algorithms, representing the structure of collaborative tasks and reasoning about progress toward task completion in an on-line fashion enables a robot to be a fluent and safe collaborator based on its ability to predict the next actions of a human agent. With this goal in mind, we focus on real-time target prediction of human reaching motion and present an algorithm based on time series classification. Results from on-line testing involving a tabletop task with a PR2 robot yielded 70% prediction accuracy with 400 msec of observed trajectory.

@inproceedings{Perez-D'Arpino:2016:FMP:3061053.3061184,
author = {P{\'e}rez-D'Arpino, Claudia and Shah, Julie A.},
title = {Fast Motion Prediction for Collaborative Robotics},
booktitle = {Proceedings of the Twenty-Fifth International Joint Conference on Artificial Intelligence},
series = {IJCAI'16},
year = {2016},
isbn = {978-1-57735-770-4},
location = {New York, New York, USA},
pages = {3988--3989},
numpages = {2},
url = {http://dl.acm.org/citation.cfm?id=3061053.3061184},
acmid = {3061184},
publisher = {AAAI Press}, }