% Our framework ties in several of the aforementioned concepts into
% a unified articulation learning pipeline that allows robots to learn the
% underlying kinematic structure of an articulated object given multiple visual
% lessons, and use its learned structure to predict its motion at future
% encounters.

% Pipeline figure
%
\begin{figure*}[!t]
    \centering
    \includegraphics[width=2\columnwidth]{graphics/pipeline-figure/pipeline.png}
    \caption{Articulation learning architecture.}
    \label{fig:vis-demonstration-pipeline}
\end{figure*}
%
%
\section{Related Work}
\label{sec:related-work}
Providing robots with the ability to learn models of articulated
objects requires a range of perceptual skills such as object tracking,
motion segmentation, pose estimation, and model learning. It is
desirable for robots to learn these models from demonstrations
provided by ordinary users.  This necessitates the ability to deal
with unstructured environments and estimate object motion without
requiring tracking markers. Traditional tracking algorithms such as
KLT~\cite{bouguet2001pyramidal}, or those based on
SIFT~\cite{lowe2004distinctive} depend on sufficient object texture
and may be susceptible to drift when employed over an object's full
range of motion. Alternatives such as large-displacement optical
flow~\cite{brox2011large} or particle video
methods~\cite{sand2008particle} tend to be more accurate but require
substantially more computation.

Articulated motion understanding generally requires a combination of motion
tracking and segmentation. Existing motion segmentation algorithms use feature based
trackers to construct spatio-temporal trajectories from sensor data, and cluster
these trajectories based on rigid-body motion constraints.  Recent work by
~\citet{brox2010object} in segmenting feature trajectories has shown promise in analyzing and
labeling motion profiles of objects in video sequences in an unsupervised
manner. Recent work by \citet{elhamifar2009sparse} has proven
effective at labeling object points based purely on motion visible in
a sequence of standard camera images.

Researchers have studied the problem of learning models from visual
demonstration. \citet{yan2006general} and \citet{huang12} employ
structure from motion techniques to segment the articulated parts of
an object, then estimate the prismatic and rotational degrees of
freedom between these parts. These methods are sensitive to outliers
in the feature matching step, resulting in significant errors in pose
and model estimates. Closely related to our work,
\citet{katz2010interactive} consider the problem of extracting
segmentation and kinematic models from interactive manipulation of an
<<<<<<< .mine
articulated object. They take a deterministic approach, first assuming
that each object linkage is prismatic and proceed to fit a rotational
DOF only if the residual is above a specified threshold. Katz et al.\
learn from observations made in clean, clutter-free environments and
primarily consider objects in close proximity to the RGB-D
sensor. Recently, \citet{katz2013interactive} propose an improved
learning method that has equally good performance with reduced
algorithmic complexity. However, the method does not explicitly reason
over the complexity of the inferred kinematic models, and tends to
over-fit to observed motion. In contrast, our algorithm targets in
situ learning in unstructured environments with probabilistic
techniques that provide robustness to noise. Our method adotps the
work of \citet{sturm2011probabilistic}, which used a probabilistic
approach to reason over the likelihood of the observations while
simultaneously penalizing complexity in the kinematic model. Their
work differs from ours in two main respects:  they required that 
fiducial markers be placed on each object part in order to provide 
nearly noise-free observations; and they assume that the number 
of unique object parts is known a priori.
=======
articulated object. They take a deterministic approach, first assuming that each
object linkage is prismatic and only proceed to fit a rotational
degree-of-freedom if the residual is below a predetermined threshold. Katz et
al.\ learn from observations made in clean, clutter-free environments and
primarily consider objects in close proximity to the RGB-D sensor. Recently,
\citet{katz2013interactive} propose an improved learning method that has equally
good performance with reduced algorithmic complexity. However, the authors'
method do not explicitly reason over the complexity of the inferred kinematic
models and tend to over-fit to the observed motion. In contrast, our algorithm
targets in situ learning in unstructured environments with probabilistic
techniques that provide robustness to uncertainty. In that regard, \citet{sturm2011probabilistic} explicitly model this
uncertainty by learning a probability
distribution over the inferred kinematics, with a penalty on model
complexity that alleviates over-fitting. Their end-to-end articulation
learning pipeline requires
that fiducial markers be placed on each object part in order to provide nearly
noise-free observations. Furthermore, they assume that the number of unique object parts is
known a priori. Our proposed framework similarly learns a probability
distribution over the kinematics, but does not
require the use of fiducials and makes no assumptions
on the articulated object, or on the number of object parts it may contain.
>>>>>>> .r86

% We draw concepts from existing work to introduce a
% robust multi-scale feature tracker that can learn new feature representations
% over time, but also ensures little to no drift by incorporating a
% detection-learning-update cycle.

% Furthermore, no prior information is provided to the robot regarding
% the object being manipulated, or the number of object parts involved in the
% manipulation of the object. In essence, we want to enable robots that can learn
% the underlying kinematic linkages involved in an articulated object purely by
% observing user-provided visual demonstrations. 

% Our framework is largely based on
% this probabilistic approach that enables us to reason over several different
% kinematic model configurations given $SE(3)$ pose observations of the object
% parts, while also considering the overall complexity of the articulated object
% being manipulated.

%
%
% \citet{yan2006general} employ structure from motion techniques to retrieve the
% segmentation of articulated objects by observing the different object parts
% being exercised. They use spectral clustering to identify rigid bodies and
% establish kinematic relationships between these object parts. However, their
% algorithm doesn't particularly distinguish between different articulation
% models, nor do they have the capability to learn from multiple
% demonstrations. Ross et al. \citet{} also rely on structure-from-motion by use
% maximum-likelihood estimation to predict rigid body segmentations. However,
% their implementation has a limition of predicting only rotational degrees of
% freedom. We shall also note that some of the previously mentioned algorithms
% that use structure from motion suffer from high computational costs, and are not
% suitable to real-time robotics applications depending on the amount of
% information that requires processing. Furthermore, we focus on tracking 3-D
% features that are readily available via depth sensors as opposed to having to
% reconstruct 3-D features via structure-from-motion techniques.
%
%

% \subsubsection{Feature tracking} Feature tracking is an essential component
% required in order to trace the trajectory of an object being
% manipulated. Traditional feature tracking algorithms make strong assumptions
% about the texture of the object being tracked. Trackers such as KLT
% \cite{bouguet2001pyramidal}, or feature trajectories based on SIFT
% \cite{lowe2004distinctive} strongly depend on objects being heavily and uniquely
% textured. Variants of these algorithms enable learning the object features in an
% on-line fashion. However, such capabilities give rise to drift in tracking of
% these features, that eventually lead to significant errors in the overall object
% trajectory observed. KLT is also limited in its capabilities as it does not
% handle object occlusions nor does it provide any feature
% reacquisition. SIFT-based trajectories on the other hand are capable of handling
% occlusions, however, they are computationally expensive.  Dense trajectories
% \cite{wang2011action} employ a different strategy involving dense sampling,
% followed by a dense optical flow for their feature extraction and propagation
% steps respectively. Here, the features are sampled via the Shi and Tomasi
% criterion. For the feature propagation step, the authors employ a pyramidal
% implementation of dense optical flow by \citet{farneback2003two}. To avoid drift
% in the feature trajectories extracted, the trajectories are pruned after they
% have achieved a fixed length of 15, after which they are eliminated and new
% trajectories are sampled. The authors specifically compromise on the accuracy
% for speed allowing it to run at approximately 8Hz. On the other hand,
% large-displacement optical flow methods \cite{brox2011large} or particle video
% \cite{sand2008particle} tend to be more accurate for the construction of long
% range trajectories, however, they take a substantially longer time to process.

% We draw concepts from existing work to introduce a robust multi-scale feature
% tracker that can learn new feature representations over time, but also ensures
% little to no drift by incorporating a detection-learning-update cycle.

% \subsubsection{Motion Segmentation} Motion segmentation refers to the partitioning of
% a sequence of observations into multiple spatio-temporal regions with similar
% motion profiles. Existing algorithms in motion segmentation use feature based
% trackers to construct spatio-temporal trajectories from sensor data, and cluster
% these trajectories based on rigid-body motion constraints.  Recent work by
% ~\citet{brox2010object} in segmenting feature trajectories have showed promise in analyzing and
% labeling motion profiles of objects in video sequences in an unsupervised
% manner. The authors in ~\cite{goh2007segmenting}, ~\cite{tron2007benchmark},
% and ~\cite{vidal2008multiframe}, have
% consistently introduced newer algorithms that segment regions in video sequences
% based on rigid body motion. In their work, ~\citet{elhamifar2009sparse} have
% shown effective methods in labeling object points purely based on motion observed
% in a video sequence taken by a standard camera. \citet{katz2013interactive} also
% use similar motion trajectory similarities as suggested in \cite{brox2010object}
% to cluster feature trajectories in 3-D.


% We draw inspiration from these works and transform our motion segmentation
% problem into a kernel-based clustering problem by analyzing the relative motion
% profiles of feature trajectories. 

% \subsubsection{Pose Estimation} Traditional object tracking algorithms work mostly on
% the level of single objects. These methods extract salient features from the
% object and subsequently match them across frames to estimate the relative object
% motion in $SE(3)$. Techniques often apply RANSAC~\citep{fischler1981random} to
% features in successive frames to remove outliers before estimating the relative
% object pose in $SE(3)$. More recent techniques \cite{collet2011moped} simultaneously learn
% new features on the object while tracking and estimating its orientation by
% posing it as a structure-from-motion problem. However, these algorithms focus
% only on pose estimation for a single object in the scene. Since we are
% particularly interested in the motion of articulated objects that consist of
% multiple rigid-object parts, we consider the pose estimation and optimization
% for each object segment independently. 

% \subsubsection{Articulation Learning} Articulation learning refers to understanding of
% the underlying kinematic model of an articulated object, given a sequence of
% observations exhibiting the motion of its object parts. Initial attempts by 
