\documentclass[10pt]{article}
\usepackage[xdvi]{graphicx}
\usepackage{epsfig}
\usepackage{epsf}
\usepackage{wrapfig}
\usepackage{amsmath}
\usepackage{times}
\usepackage{helvet}
\usepackage{courier}
\usepackage{mathrsfs}
\usepackage{calrsfs}
\usepackage{yfonts}
\usepackage{stmaryrd}
\usepackage{mlapa}
\usepackage{subfigure}
\usepackage{icml2005} 



\DeclareMathOperator*{\argmax}{argmax}
\newcommand{\NewCenturyBoldFourteen}{\fontencoding{OT1}%
                                     \fontfamily{pnc}%
                                     \fontsize{13.5pt}{14.0pt}%
                                     \selectfont}



\newcommand{\sketch}{$\mathcal{S}$}

\newcommand{\src}[1]{{\bf\texttt{\bf{\footnotesize{\uppercase{#1}}}}}}
\newcommand{\node}[2]{${\bf\texttt{\bf{\footnotesize{\uppercase{#1}}}}_{#2}}$}
\newcommand{\ogi}{{\boldmath$O_{G_i}$}}
\newcommand{\llgi}{{\boldmath$\lambda_{L(G_i)}$}}
\newcommand{\lobj}[1]{{\boldmath$\lambda_{#1}$}}



\begin{document}
\twocolumn[
\icmltitle{Sketch Interpretation Using Multiscale Models of Temporal Patterns}
\icmlauthor{Tevfik Metin Sezgin$^{\tiny *}$}{mtsezgin@csail.mit.edu}
\icmlauthor{Randall Davis}{davis@csail.mit.edu}
\icmladdress{MIT CSAIL,
    32 Vassar St. 235, Cambridge, MA 02139 USA}
]


\begin{abstract}
With the emergence and widespread use of Tablet PCs, recognizing 
freehand sketches has gained importance as an enabling technology 
for pen-based intelligent human computer interfaces. This paper 
presents a sketch recognition framework based on multiscale
statistical models of temporal patterns. Our previous work in 
sketch recognition has shown that in certain domains the stroke 
orderings used when drawing objects contain temporal patterns that
can aid recognition. We build on this work to show how sketch 
recognition systems can use knowledge of both common stroke 
orderings and common object orderings. We describe a statistical 
framework based on Dynamic Bayesian Networks that can learn temporal 
models of object-level and stroke-level patterns for recognition.
Our recognition framework supports multi-object strokes, multi-stroke objects, 
and it supports real-valued feature representations using a numerically stable 
recognition algorithm. We present recognition results for hand-drawn electronic 
circuit diagrams. The results show that modeling temporal patterns at multiple 
scales provides a significant increase in correct recognition rates, with no 
added computational penalties. 
\end{abstract}

\section{KEYWORDS} 
G.4.h User interfaces, I.2 Artificial Intelligence, I.2.6 Learning, I.4.6 Segmentation


\section{INTRODUCTION}
Sketching is a natural input modality of increasing interest. The emergence of 
hardware such as Tablet PCs and hand-held PDAs provides easy means for capturing 
pen input. These devices combine a display with a computing device, making it 
possible to capture and process sketches online (i.e., as they are drawn).
Online recognition has two main advantages. First, it enables interpreting and 
displaying pen input as it is entered (e.g., an engineer drawing a circuit diagram 
sees ink change color in real-time to indicate which circuit components are 
recognized). Second, in an online sketching system we have access to stroke
ordering information as well as the end product of the drawing process.

%% \begin{wrapfigure}{r}{1.5in}
%% \parbox[b]{2in}{\NewCenturyBoldFourteen
%% {\em  We show how sketch recognition systems can use knowledge 
%% of both common stroke orderings and common object orderings.}}
%% \end{wrapfigure}

Our previous work showed that in certain domains, temporal stroke orderings used when
sketching objects contain predictable patterns that 
can be used for object recognition \cite{sezgin_iui05}.  We call these 
{\bf stroke-level} patterns because they capture the probability of seeing a 
sequence of strokes with certain properties. For example, when people draw
stick-figures, one frequently seen stroke-level pattern is a sequence of a circular 
stroke, followed by a vertical line, followed by two pairs of positively and
negatively sloped lines, corresponding to the head, body, arms
and legs of the stick-figure. 

Another kind of temporal pattern present
in online sketches is {\bf object-level patterns} that capture the probability
of seeing a certain sequence of objects being drawn. Consider the domain of 
UML class diagrams, drawn by software designers using rectangles to indicate classes and a variety of different kinds of arrows to indicate relations among classes, such as 
inheritance, generalization, association, etc. In this domain, 
when a designer draws a new class (indicated by a rectangle), it is natural to expect that the 
new object will soon be connected with an arrow to one or more of the objects drawn earlier.
We describe this as an {\em object-level pattern} indicating that drawing a class is followed by drawing some variety of arrow. Note that 
the kind of arrow to expect can depend on the kind of class drawn: a \verb=final= class, for example, cannot be extended, limiting the kind of arrow that we may expect next.

In domains like UML, where there is a graphical grammar of sorts, it is plausible to 
imagine writing down the grammar and using this to guide a UML diagram recognition 
system. But not every domain is likely to have patterns that are 
as well understood as they are for UML diagrams. And even if experts could 
identify such patterns, incorporating them into a recognition system would be a
laborious task at best, considering all the ways in which various objects may combine
together. We argue that a better way of incorporating object-level temporal patterns
in a recognition framework would be to learn such features from data -- along with 
stroke-level patterns -- and use them in recognition. 

In the rest of this paper, we present a sketch recognition framework where common object 
orderings that naturally appear in sketches can automatically be learned from data 
and used for sketch recognition. The key features that make our framework novel include 
the ability to learn object-level patterns from data, the ability to handle 
multi-stroke objects and multi-object strokes, and support for continuous observable 
features. We show how graphical models can be used to implement our approach  
efficiently and demonstrate that a specialized inference algorithm known as the 
Lauritzen-Jensen stable conditional Gaussian belief propagation should be used to avoid 
numerical instabilities in recognition.

We start with a formal description of the sketch recognition problem. Section 3
presents our approach to modeling common object-level and stroke-level patterns. 
Section 4 presents recognition results obtained for the electrical circuit 
diagrams domain. In section 5, we review related work, and conclude with a summary of
our key contributions.


\begin{figure}
  \centerline{
    \hbox{ 
      \includegraphics[scale=.3]{figures/circuits/circuit_example_2}
    }
  }
  \caption{A hand-drawn circuit diagram showing what we mean by a sketch. }
  \label{sketch_example} \end{figure}


\section{PROBLEM DESCRIPTION}
\subsection{Terminology}
By a sketch, we mean messy, informal hand-done drawings (e.g., Fig.~\ref{sketch_example}).
Specifically we are interested in recognizing sketches with objects of a symbolic nature that 
can be represented using structural descriptions, that have been the focus of the sketch 
recognition community \cite{christine_thesis,shilman,mahoney_ss,kara_sr,sezgin_iui05}. 

We formally define a {\bf \em sketch} $\mathscr{S}=S_1, S_2, ... S_N$ as a sequence 
of strokes captured using a digitizer, preserving the drawing order.\footnote{We will use 
the Matlab notation $begin:end$ as a shorthand for a list of indices that begins with $start$ 
and ends with $end$ (i.e., 1:4 = 1 2 3 4). We will also use this notation in subscripts to 
denote a list of items (i.e., $S_{1:N}=S_1, S_2, ... S_N$).} A {\bf \em stroke} is defined as 
a set of time-stamped points sampled between pen-down and pen-up events during sketching. 

Each stroke is preprocessed using the toolkit described in \cite{sezgin_pui},
converting the sketch $S_{1:N}$ to a time-ordered sequence of geometric {\bf \em primitives} (line and arc segments) $P_{1:T}$. As a single stroke may result in more
than one primitive, the total number of primitives is usually larger than the number of strokes.


We use {\bf \em segmentation} to refer to the task of grouping together primitives constituting 
the same object, and by {\bf \em classification} mean the task of determining 
which object each group of primitives represents (e.g., a stick-figure or a rectangle).
Segmentation produces $K$ groups $G=G_1, G_2, ... G_K$, and classification gives us the
labels for the groups $L = L_1, L_2, ... L_K$. 
%Each $G_i=begin_i:end_i$ specify the range of primitives in that
%group (i.e., group $i$ includes primitives $P_{begin_i:end_i}$.
We define {\bf \em sketch recognition} as the segmentation and classification of a sketch.
\footnote{A simplifying assumption in most sketch recognition systems is that a stroke can be
part of only one object. Our definition of segmentation in terms of primitive groupings is 
more general than a definition based on stroke groupings and allows a stroke to be part 
of multiple objects (e.g., drawing a box and an arrow, or a resistor and a pair of wires in a 
single stroke (Fig.~\ref{dbn-example})).}


By {\em observations} we refer to the sequence of features 
$\mathscr{O}=O_1, O_2, ..., O_T$ obtained from the primitives. We use \ogi
to refer to the sequence of observations corresponding to a group of primitives $G_i$,
and $L(G_i)$ refers to the class label assigned to that group. Finally we will use \llgi to
refer to the stroke-level temporal model corresponding to the label $L(G_i)$, and \lobj{obj}  
to refer to our model for object-level temporal patterns.


\subsection{Problem Formulation}
Given a sequence of time-ordered primitives obtained from a sketch, our goal is to find a
segmentation and classification of the primitives such that the joint likelihood of 
stroke-level patterns and object-level patterns is maximized. 

We formulate the problem starting with the stroke-level model. If all we had were the 
stroke-level patterns, then the recognition task would require finding a segmentation 
and classification of the primitives such that the likelihood of the stroke-level patterns is
maximized. 
In other words, over all possible ways in which the primitives can be grouped, $\textswab{G}$, 
and all possible ways in which these groups can be labeled, $\textswab{L}(G)$, the goal of 
the stroke-level model is to find the grouping $G\in\textswab{G}$ and the labeling 
$L \in \textswab{L}(G)$ that maximizes the joint likelihood of the stroke-level observable 
features \ogi { for} each group $G_i$ given the stroke-level model corresponding to its label 
(\llgi). This amounts to finding the solution maximizing the likelihood expression:


{\small
\begin{equation}
\label{str-lvl-maximization}
\argmax_{G \in \textswab{G}, L \in \textswab{L}(G)} \prod_{i=1}^{|L|} P(\mbox{\boldmath${O_{G_i}}$}|\mbox{\boldmath$\lambda_{L(G_i)}$} ) 
\end{equation}}

In order to obtain the expression corresponding to the stroke-level and object-level
patterns, we multiply  Eq.~\ref{str-lvl-maximization} by a likelihood term corresponding 
to the likelihood of the sequence of labels $L_1, L_2, ..., L_K$ given our model for 
object-level patterns (\lobj{obj}). This gives us the following expression, the solution 
to which is the segmentation and labeling of the input sketch:

{\small
\begin{equation}
\label{maximization}
\argmax_{G \in \textswab{G}, L \in \textswab{L}(G)} P(L_1, ..., L_K| \mbox{\boldmath$\lambda_{obj}$} ) \prod_{i=1}^{K} P(\mbox{\boldmath${O_{G_i}}$}|\mbox{\boldmath$\lambda_{L(G_i)}$} ) 
\end{equation}}

%\noindent
%Note that if we assume that all object-level patterns are equally likely, then the solutions 
%of Eq.~\ref{str-lvl-maximization} and Eq.str-lvl-maximization

\section{APPROACH}
The expression in Eq.~\ref{maximization} is maximized over the set of all groupings and 
their labelings ($\textswab{G}$ and $\textswab{L}(G))$. As it is well known from the vision and
sketch recognition literature, exhaustive search of this space is computationally 
prohibitive \cite{mahoney_ss}.

Noting that we are modeling sequential patterns, we make the assumption that in our target
domains, both stroke-level and object-level patterns can be modeled as products 
of first-order Markov processes. This will allow us to efficiently compute maximum 
likelihood estimates to learn the parameters of our stroke-level and object-level models, and 
will also enable efficient recognition. Our definition of efficiency encompasses the CPU time 
spent when our algorithm is run on realistic problems as well as the theoretical bounds on its
algorithmic complexity. As we report below, it takes our algorithm a few seconds to interpret
moderate size sketches and the time complexity of inference in our DBN based 
representation scales linearly with respect to the problem size.

\subsection{The Model}
We will represent our model of temporal patterns using Dynamic Bayesian Networks 
(DBNs). Bayesian networks encode the joint probability of a set of variables $Z=\{Z_1, ..., Z_n\}$
where the graphical structure of the network encodes the conditional dependencies among 
the variables. DBNs are extensions of Bayesian networks that model joint distribution of 
a set of variables over time by representing the conditional dependencies between the
variables using a pair of Bayesian networks $(B_1,B_{\shortrightarrow})$. $B_1$ defines
the prior for the $Z_i$ values at time $t=1$, and $B_{\shortrightarrow}$ defines how variables at 
time $t+1$ relate to each other and to those from time $t$. Both Bayesian networks and 
DBNs have visual representations in terms of graphs, hence they are called graphical models.
The rest of our discussion assumes that the reader is familiar with DBNs (an excellent 
review can be found in \cite{murphy_dbn_chapter}).

\begin{figure}
  \centerline{
    \hbox{ 
      \includegraphics[width=3in]{figures/stroke_level_model_4}
    }
  }
  \caption{The dynamic Bayesian network representing the model for capturing stroke level patterns. This fragment has a dual representation as a hidden Markov model with continuous observations and $|$\src{M}$|$ mixtures.}
  \label{stroke_level_model} 
\end{figure}


\subsubsection{The input}
The input to our model is the observation sequence $\mathscr{O}=O_{1:T}$ obtained by 
computing features from corresponding primitives $\mathscr{P}=P_{1:T}$. The specific
features we use are described below. 

Working with primitives (rather than strokes) allows us to handle multi-object strokes 
which is an essential feature in certain domains \cite{christine_thesis}, but
is sometimes unsupported for efficiency reasons or by architectural limitations. 
By breaking strokes into primitives and having a model that allows 
primitives obtained from a single stroke to be assigned to different objects, 
we support multiple objects to share a stroke (but not a primitive).

In sketch recognition, we deal with data that is most naturally described using 
geometric features such as the shape of a stroke segment, length and orientation 
of line segments, radii of circles etc. Some of these features are categorical 
(e.g., shape of a stroke segment can be \verb=arc=, \verb=line= etc.) and are 
best represented using discrete variables. Other features such as length and 
orientation are real-valued quantities and should be represented as such. 
Therefore we support both discrete and real-valued (continuous) 
observations. In addition to allowing us to have a richer set of features,
supporting real-valued features make it possible to avoid discretization 
issues such as choosing the optimal discretization parameters (e.g., number of 
bins, bin sizes).


\subsubsection{The stroke-level model}
For the sake of clarity of presentation, we present our model bottom up, 
starting with the stroke-level model. For each object class to be recognized, 
we need a stroke-level model. The purpose of the stroke-level model is to 
compute the degree of agreement between a particular observation sequence 
and those sequences typically observed while drawing a particular class of objects.
The stroke-level models answer questions of the sort ``what is the likelihood of seeing the 
observation sequence \ogi obtained from a group of primitives $G_i$ under the rectangle 
model'', or more formally what is $P($\ogi$|$\lobj{rectangle}$)$. This corresponds to the 
likelihood term $P($\ogi$|$\llgi$)$ in expression \ref{str-lvl-maximization} and the
innermost in expression \ref{maximization}.

Fig.~\ref{stroke_level_model} shows the DBN fragment corresponding to our stroke-level
model. The observable node \src{obs} at time $t$ represents the features extracted from primitive
$P_t$. As noted before, we encode the stroke-level patterns using a first-order Markov process.
The discrete node \src{str} in Fig.~\ref{stroke_level_model} captures the 
dynamics of this Markov process. During training, parameters of the \src{str} 
node are estimated based on the patterns that exist in the training data
provided through the \src{obs} variable. The \src{M} variable is a discrete 
mixture variable and allows us to represent the observations using mixtures of Gaussians. 
Each \src{str} node is connected to the one in the next time slice, reflecting our choice 
of modeling stroke-level patterns as products of a Markov process.


\begin{figure}
  \centerline{
    \hbox{ 
      \includegraphics[width=3.3in]{figures/hierarchical_dbn}
    }
  }
  \caption{The dynamic Bayesian network representing our model.}
  \label{dbn} \end{figure}

\subsubsection{The combined model}
We next augment 
the stroke-level model by adding two nodes \node{obj}{t} and \node{end}{t} (Fig.~\ref{dbn}).\footnote{Our combined DBN model has a dual representation as a hierarchical HMM
with continuous observables modeled using mixtures of Gaussians. We choose to present
the DBN view because DBNs generalize HHMMs and the DBN representation is more efficient.}  
\node{obj}{t} is a multi-valued discrete variable that holds our hypothesis of 
which object the primitive $P_t$ is a part of. It can take as many values as the 
number of classes we can recognize. \node{end}{t} reflects our belief about whether 
the user has just completed drawing object \node{OBJ}{t} by drawing the primitive $P_t$. 

The combined model defines a joint distribution over the variables from the stroke-level model,
the sequence of object hypotheses \node{obj}{1:T}, as well as hypotheses on where each object
begins and ends (\node{end}{1:T}). The joint probability over \node{obj}{1:T} corresponds to
$P(L_1, ..., L_K| $\lobj{obj}$)$, the first term term in expression \ref{maximization}.
Combined with the stroke-level likelihood term $P($\ogi$|$\llgi$)$ from above,
the combined model allows us to compute the most probable values of \node{obj}{1:T} and
\node{end}{1:T}, which gives us the optimal segmentation and labeling of the input sketch.

The value of the \src{obj} node in each time slice is conditioned on the values of 
\src{end} and \src{obj} variables from the previous slice. This encodes the observation
that our belief about the value of \node{obj}{t+1} is based on the values of \node{obj}{t}
and \node{end}{t}. The justification for this dependence is that we would expect the
value of the \src{obj} node to change only if the user has just finished an object 
(i.e., \node{end}{t}$=${\em true}), in which case the next object we would expect to 
see would depend on what object was just completed. 
Also note that the two new inter-slice arcs introduced into our model
\node{obj}{t}$\rightarrow$\node{obj}{t+1} and \node{end}{t}$\rightarrow$\node{obj}{t+1}, cross
only a single slice boundary, thus our model remains first-order Markovian, 
enabling training and recognition to be efficient.

In summary, the \src{obj} 
node keeps track of the class for the objects drawn over time and the joint distribution 
of the \src{obj} nodes over time gives us $P(L_1, ..., L_K|$ \lobj{obj}$)$, the first 
term in expression \ref{maximization}. In the combined model, the distribution of 
the observations is determined based on the value of the \src{obj} node given by 
$P( $\node{obs}{t}$|$\node{obj}{t}$,$\node{str}{t}$,$\node{M}{t}$)$. The choice of which 
stroke-level process is activated is determined by the \src{obj} node because \node{str}{t+1} 
is conditioned on \node{str}{t}, \node{end}{t} and \node{obj}{t+1}.

\subsection{Implementation issues}
Two issues need to be addressed in implementing the model described above, 
both related to the choice of the inference algorithm to be used.

As mentioned earlier, our model has a dual representation as a hierarchical hidden 
Markov model. Unfortunately, the original inference algorithm for HHMMs is not only complicated,
but also has $O(T^3)$ time complexity (where $T$ is the length of the observation sequence)
\cite{murphy_hhmm}. In our case $T$ is the total number of primitives in a sketch, 
and this makes the model impractical for situations where real-time feedback is required.
On the other hand, by using the DBN representation we can apply efficient graphical model
techniques that take linear time \cite{murphy_hhmm}. It is therefore essential to use 
the DBN representation, or convert any HHMM based representation to a DBN prior to inference
and learning.

A major issue that arose during the implementation is the numerical instabilities 
that occur during training, due to the use of continuous observations. Bayesian 
networks that include continuous and discrete variables (mixed networks) usually represent
the continuous variables as Gaussians or mixtures of Gaussians. The conventional
belief propagation algorithm used for inference in these networks is the Lauritzen 
algorithm.% \cite{lauritzen_bp_92}
Unfortunately this algorithm is susceptible to 
numerical underflow. Although this has been documented in the machine learning 
literature, it is not well known because it occurs rarely in reality. The problem 
appears in the form of singular matrices during inference and is hard to localize. 
In order to avoid this numerical instability, we need to use a specialized algorithm 
known as the Lauritzen-Jensen stable conditional Gaussian belief propagation 
\cite{lauritzen_stable_cg}.

\subsubsection{Training}
The goal of training is to estimate the parameters of the conditional probability
distribution functions for each node in our DBN $(B_1,B_{\shortrightarrow})$
given a collection of labeled sketches as training data. 
We use parameter tying to ensure that the probability distribution 
$P($\src{A}$|Parents($\src{A}$))$ relating a node \src{A} and its parents is the same
for each node $A$ in $B_1$ and $B_{\shortrightarrow}$.\footnote{We add dummy parent
nodes to the \src{obj} and \src{str} nodes in $B_1$ to ensure that 
$P($\src{obj}$|Parents($\src{obj}$))$ and $P($\src{str}$|Parents($\src{str}$))$ can be 
represented using conditional probability tables with the same structure for $B_1$ and 
$B_{\shortrightarrow}$.}

During training we know what class each primitive belongs to and we also know
when objects begin and end, so the values of the \src{obj} and \src{end} are 
observable and are supplied during training. In summary, for each labeled sketch,
we use the values \node{obj}{1:T}, \node{end}{1:T} and \node{obs}{1:T} to estimate
the model parameters.

\subsubsection{Recognition}
The input to recognition is \node{obs}{1:T}. During recognition, the goal is to compute 
the values of \node{obj}{1:T} and \node{end}{1:T} that maximize the likelihood of the 
observations \node{obs}{1:T}. This is done using the stable conditional Gaussian
belief propagation algorithm described earlier. Interpretation of complete 
sketches comparable to those in Fig.~\ref{circuit-examples} takes less than 2-3 seconds
on a 2GHZ Pentium 4 machine. 


\begin{figure*}
  \centerline{
  \begin{tabular}{c|c}
    \includegraphics[scale=.33]{figures/circuits/circuit_example_0.eps} &
    \includegraphics[scale=.33]{figures/circuits/circuit_example_9.eps} \\
    \includegraphics[scale=.33]{figures/circuits/circuit_example_7.eps} &
    \includegraphics[scale=.33]{figures/circuits/circuit_example_3.eps} 
  \end{tabular}}
  \caption{Examples of collected circuits.}
  \label{circuit-examples}
\end{figure*}

\section{RESULTS}
In order to see whether modeling object-level patterns improves over modeling 
only stroke-level patterns, we tested our system using sketches from the 
domain of electronic circuit diagrams. This domain is particularly interesting 
because it is illustrative of a group of domains that can be characterized 
as {\em object/connector diagrams} (e.g., organizational charts, flow charts,
UML diagrams). This domain also illustrates our model's ability to deal with
multi-object strokes (e.g., $stroke \#7$ in Fig.~\ref{dbn-example}) and objects 
with varying number of components (e.g., resistors with varying number of humps).

%properties of this domain that is an example of the kinds of things
%that are particularly problematic from a temporal recognition perspective and 
%report how it affects recognition rates.
%We report results on how this affects the recognition 
%rates for our system.
%Finally, as we discovered in our evaluation, our assumption that users finish drawing 
%the current object before starting the next one is sometimes violated in the 
%circuit diagrams domain. 


\subsection{Data collection}
We collected circuit diagrams from undergraduate students who had recently taken
the Microelectronic Circuits and Devices course at MIT and were familiar with the 
textbook used in the class. A total of eight participants contributed circuit diagrams. 
All except participant \#7 were right-handed. We asked the participants to 
draw circuits selected from their course textbook using a sufficiently expressive set 
of electronic circuit components.\footnote{The components were NPN transistors, 
resistors, capacitors, batteries and wires, so $|$\src{obj}$|=5$.} 

During data collection, we first showed participants diagrams of circuits that we wanted
them to draw. We gave each participant some time to study each circuit until they 
understood how the circuit worked, then asked them to explain it verbally, to ensure 
that they really understood it. We next removed the textbook diagram and asked them to 
draw the circuit using a Tablet PC, allowing them to consult the original circuit 
diagram only if needed. Fig.~\ref{circuit-examples} shows examples of circuits drawn by 
the participants.

To train our model we created labeled training data by manually annotating 
10 sketches per participant.
\begin{figure*}
  \centerline{
  \begin{tabular}{c}
    \includegraphics[width=6.5in]{figures/primitive_extraction}\\
    \includegraphics[width=6.5in]{figures/dbn_unrolled}
  \end{tabular}}
  \caption{An example showing how a Bayesian network is created by unfolding 
    $(B_1,B_{\shortrightarrow})$ during recognition. In (a), we see 
    a circuit fragment. During recognition, each stroke is broken into primitives. 
    (b) shows how primitive extraction breaks $stroke \#7$, which 
    also happens to be a multi-object stroke, into primitives $P_{7:13}$. In 
    (c) we show the corresponding fragment of the unfolded network.
    Expected values of \node{obj}{7:13} and \node{end}{7:13} are also shown. For 
    \node{obj}{7:13}, the values $\{1,2\}$ represent ``wire'' and ``resistor'', 
    for \node{end}{7:13} \{\src{T}, \src{F}\} represent {\em true} and {\em false}.    
    Other details are omitted for brevity.}
  \label{dbn-example}
\end{figure*}

\subsection{Generating the Observation Sequence}
Both training and classification require converting a sketch to an observation sequence 
$O_{1:T}$. Using the early sketch processing toolkit described in \cite{sezgin_pui},
we first preprocess each sketch to obtain a sequence of primitives $P_{1:T}$ (in our case simply using 
line segments is sufficient because our domain has no objects with curved strokes). 
For each primitive $P_i$, we obtain an observation vector $O_t$ represented as a five-tuple
{\boldmath$(l_t,\Delta l_t, \theta_t, \Delta\theta_t, sgn_t)$} where: 

\begin{itemize}
{\item {\boldmath$l_t$} is the length of $P_i$,}
{\item {\boldmath$\Delta l_t$} is relative length ($l_t/l_{t-1}$, 1 for $t=1$),}
{\item {\boldmath$\theta_t$} is the angle with respect to the horizontal axis,} 
{\item {\boldmath$\Delta\theta_t$} is the measure of relative angle between $P_i$ and $P_{i-1}$,}
{\item {\boldmath$sgn_t$} is the direction that the stroke turns when moving from $P_{i-1}$ to $P_i$.}
\end{itemize}

The value of {\boldmath$\Delta\theta_t$} is given by the magnitude of the cross product 
$\vec u \times \vec v$ of vectors $\vec u$, $\vec v$, which are length-normalized 
versions of $P_i$ and $P_{i-1}$ pointing in the direction of pen movement along each primitive.
The turn direction ({\boldmath$sgn_t$}) is the only discrete feature and is set to $0$ for 
negative values of $\vec u \times \vec v$ and $1$ for positive values or $t=1$. The use of 
absolute length is based on our observation that for a given device and user, absolute length is 
a reliable feature with high discriminative power. Furthermore, once the corresponding Gaussian 
mixtures are learned, they can easily be scaled to work with data collected using a different
resolution device. The use of absolute angle is based on our observation that components in circuit
diagrams mainly appear in two canonical orientations. We prefer learning a few canonical 
orientations to not being able to use this informative feature for orientation invariance. 
Finally, we preferred using stroke-based edge features. It would be interesting future work 
to see how image-based features such as {\em shape contexts} or \src{SIFT} features would fare.


\subsection{Recognition Results}
For each user, we ran a series of hold-one-out experiments by getting the recognition
rates for each sketch using a model trained on the remaining sketches. For each 
sketch, we also trained a baseline system that modeled only stroke-level patterns 
(i.e., no object-level patterns). The baseline system was obtained by setting 
$P( $\node{obj}{t+1}$|$\node{obj}{t}$,$\node{end}{t}$)$ to be uniform 
($1/|$\src{obj}$|$) if \node{end}{t}$=true$. For \node{end}{t}$=false$, it was set to $1$ for 
\node{obj}{t+1}$=$\node{obj}{t}, and $0$ otherwise. Note that the baseline has the same network
topology and the same junction tree; hence inference for the baseline method has the same 
computational complexity as our method. In both models we used the values $|$\src{str}$|=6$ 
and $|$\src{M}$|=3$.

\subsubsection{Quantitative Results}
Table~\ref{results-table} shows the average correct recognition rates for each participant
obtained using the baseline model and using our model. As seen in the table, the improvement 
in absolute terms ranges from 2.1\% to 6.2\%. The table also shows the average and maximum 
reduction values in the error rates in terms of percentages for each user. As seen here, 
on average, modeling object-level patterns always improves performance, allowing up to 65\% 
of misrecognition errors to be corrected. 

A paired t-test comparing the average correct recognition rates for each user finds the
increase to be statistically significant for $p<0.05$ and 7 degrees of freedom. We believe
the reported decrease in the error rates would substantially improve users' satisfaction
with the recognition system in an actual design setting.

\subsubsection{Qualitative Results}
We believe it is instructive to explore the kinds of errors avoided by 
modeling object-level patterns. Consider the amplifier circuit in 
Fig.~\ref{hierarchy-circuit}. The figure also shows the stroke ordering for 
the fragment of the circuit diagram of particular interest to us. Two 
interpretations of this circuit are shown in Fig.~\ref{hierarchy_example}.

Fig.~\ref{hierarchy_example}-a shows the interpretation obtained by running
the baseline method, which encounters four recognition errors. The 
capacitor \src{c2} is misrecognized as wires, most probably because 
of the hook hanging off one of the strokes. The NPN transistor \src{q2}
is misrecognized because the stroke indicating the 
current direction is noisy. Note that the vertical part of this 
transistor (segment \#11) and the wire connected to the emitter (segment \#12)
are grouped together and labeled as a capacitor. Inspection of the sketch 
reveals that these two strokes have roughly the same length and are sketched 
one after the other (shown by the numbers in Fig.~\ref{hierarchy-circuit}). 
This easily leads to these strokes being mislabeled as a capacitor.\footnote{As we 
describe below, this is an example of the limitations of our order-based recognition.} 
Finally, part of the resistor \src{r1} is misrecognized.

Fig.~\ref{hierarchy_example}-b shows that two of these errors 
are fixed by the our model. Our model has learned that the
chance of a resistor being followed immediately by a transistor, and 
capacitors being immediately followed by resistors with no wires in between
is very small, so these interpretations are penalized. As a result, the two 
misrecognitions are avoided.


\begin{figure}
  \centerline{
    \includegraphics[width=3.25in]{figures/qualitative_examples/hierarchy_circuit.eps}}
    \caption{One of the circuits drawn by our study participants. Stroke 
      ordering for a fragment of the circuit is indicated by numbers.}
  \label{hierarchy-circuit}
\end{figure}

\begin{figure}
  \centerline{
  \begin{tabular}{c}
    \includegraphics[width=3.2in]{figures/qualitative_examples/flat_computed} \\
    (a) Interpretation of the baseline model. \\
    \includegraphics[width=3.2in]{figures/qualitative_examples/hierarchical_computed}\\
    (b) Interpretation of our model.
  \end{tabular}}
  \caption{Examples of errors corrected using context provided by object-level patterns.
    There are four misrecognitions in (a). Note how two of these misrecognitions is 
    fixed using knowledge of object level patterns (resistor in the upper left corner
    and the wire connected to the emitter of the misrecognized transistor). 
    %See Fig.~\ref{hierarchy_example-bw} for B\&W printing. 
  }
  \label{hierarchy_example}
\end{figure}



\begin{table}
 \begin{center}
   \begin{tabular}{l||c|c|c|c|c|c|c|c||}
     \cline{2-9}
     & \multicolumn{8}{|c|}{Participant ID}\\
     \cline{2-9}
     
                                                        &\scriptsize 1     &\scriptsize 2     &\scriptsize 3     &\scriptsize 4     &\scriptsize 5     &\scriptsize 6     &\scriptsize 7     &\scriptsize 8\\
     \hline
     \multicolumn{1}{|l||}{\small $\mu_b$}              &\scriptsize 83.2  &\scriptsize  86.5 &\scriptsize  85.2 &\scriptsize 73.8  &\scriptsize  87.1 &\scriptsize 90.7  &\scriptsize 79.1 &\scriptsize 85.0\\
     \hline
     \multicolumn{1}{|l||}{\small $\mu_c$}              &\scriptsize 89.4  &\scriptsize 89.4  &\scriptsize 87.3  &\scriptsize 77.4  &\scriptsize 89.8  &\scriptsize 93.0  &\scriptsize 84.6 &\scriptsize 88.2\\
     \hline
     \multicolumn{1}{|l||}{\small $\Delta{err}$}        &\scriptsize 36.9  &\scriptsize 21.4  &\scriptsize 14.1  &\scriptsize 13.7  &\scriptsize 20.9  &\scriptsize 24.7  &\scriptsize 18.6  &\scriptsize 21.3\\
     \hline
     \multicolumn{1}{|l||}{\small $max \Delta$}         &\scriptsize 65.0  &\scriptsize 45    &\scriptsize 31.2  &\scriptsize 15.0  &\scriptsize 35.5  &\scriptsize 40.0  &\scriptsize 30.7  &\scriptsize 54.5\\
     \hline
     \hline
   \end{tabular}
 \end{center}
 \caption{Mean correct recognition rates for the baseline system  ($\mu_b$) 
   and our model ($\mu_c$) on sketches collected from 8 
   participants. The table also shows the percentage reductions in the error 
   rates and maximum error reductions achieved for each user as percentages 
   ($\Delta_{err}$ and $max \Delta$). On average, modeling object-level 
   patterns always improves performance. }
 \label{results-table}
\end{table}



Inspecting the conditional probability table 
$P( $\node{obj}{t+1}$|$\node{obj}{t}$,$\node{end}{t}$)$ reveals the kinds of 
object-level patterns learned by our models. For the 8
participants in our study, all the models learned (the unsurprising fact) 
that most of the time circuit components are preceded and followed by wires. 

A more interesting observation comes from looking at what happens when an object is not 
followed by a wire. In these cases, we see that components that frequently appear together 
in conceptually meaningful circuit fragments have a higher probability of being drawn 
consecutively. For example, resistors appear in series when drawing voltage dividers,
hence when a resistor is not immediately followed by a wire temporally, it is 
more likely to be followed by a resistor than by anything else. It is conceivable that a 
circuit design expert may predict the existence of such patterns and try to incorporate them
in the design of a circuit recognition system, but this would be a laborious task
and require precision in balancing biases for different kinds of patterns. 
Our model learns and uses such patterns automatically in a mathematically sound framework.

\subsubsection{Discussion}
Our experiments also helped us identify a limitation of relying on temporal patterns
alone. Our approach does not incorporate any spatial or geometric constraints beyond 
those used to encode stroke sequences, and as a result recognition is based strictly 
on temporal patterns and not shape. Therefore a sequence of strokes that has the 
right temporal character 
but the wrong shape can be classified incorrectly. For example, inspection
of the circuit in  Fig.~\ref{hierarchy_example} reveals that strokes \#11 and 
\#12 were drawn one after the other, they are parallel, they have roughly the 
same length and were misclassified as a capacitor even though the 
strokes are offset from one another. It can be argued that incorporating 
relative positions of consecutive primitives to the existing feature representation
may fix the error in this specific case, but incorporating shape information 
requires modeling complex long-term dependencies (constraints) such as those described in 
\cite{tracy-language} between primitives arbitrarily apart in time. 
This is beyond the reach of our first-order Markovian model. 
% We consider below how this limitation can 
% be addressed by using shape based classifiers as external sources of information.

Our evaluation also revealed a surprising drawing convention employed while 
drawing transistors. Although the participants completed each circuit component
before starting a new one, four of them sometimes interspersed wires during 
the course of drawing transistors. By interspersing we refer to the
situation where the user starts drawing one object but draws one or more other 
objects before it is completed (e.g., wires \#3 and \#6 in Fig.~\ref{interspersing_diagram}).
Because our model assumes 
the users complete each object before moving on to the next, interspersed drawing
causes misrecognitions. 

In our case, interspersings only happened while drawing
transistors.\footnote{Although there is psychology research linking
drawing orders to the way people think about the function, significance and appearance of 
objects \cite{tversky_drawing}, the circuit diagrams domain and the interspersed drawing 
behavior has not been studied. Based on our discussions with expert circuit designers,
we hypothesize that some users intersperse wires while drawing transistors primarily 
because transistors have three points where wires can connect and circuit designers 
tend to follow conventions such as ``drawing from left-to-right'' and ``following the 
current''.} In order to measure the effects of interspersing on correct recognition rates
we selected a control group of ten circuits containing 2-4 transistors with no interspersings
and a test group that contained versions of these sketches with interspersings. The 
interspersed version of each sketch was obtained by moving one of the wires 
preceding/following a transistor back/forward in time. The way we set-up our control
and test sets ensures that we measure the misrecognition effects due only to interspersing.
Table~\ref{per-interspersing-results} summarizes the recognition results obtained for
the control and test sets carried out in a leave-one-out cross-validation setup. In either
case the training was done using circuits with no interspersings, consistent with the 
assumptions of our model. As the results in Table~\ref{per-interspersing-results} indicate,
violation of our ``one object at a time'' assumption always introduces new recognition errors.
Even when the number of misclassifications is normalized by the number of interspersings,
a transistor-wire interspersing causes 2-3 incorrect classifications per interspersing.
In the future work section, we point out how this issue can be addressed by modeling 
interspersed drawing behavior explicitly. 

\begin{figure}
  \centerline{
    \hbox{ 
      \includegraphics[width=3.1in]{figures/circuits/interspersing_example}
    }
  }
  \caption{An example of interspersing: The user draws two other objects (wires \#3 and \#6) over the course of drawing the transistor. The drawing order is indicated by numbers.}
  \label{interspersing_diagram} 
\end{figure}


\begin{table*}
 \begin{center}
   \begin{tabular}{l||c|c|c|c|c|c|c|c|c|c|c||}
     \cline{2-11}
              & \multicolumn{10}{|c|}{Sketch ID}\\
     \cline{2-12}
                                                                &\scriptsize 1     &\scriptsize 2     &\scriptsize 3     &\scriptsize 4     &\scriptsize 5     &\scriptsize 6     &\scriptsize 7     &\scriptsize 8 &\scriptsize 9 &\scriptsize 10 &\scriptsize $\mu$   \\
     \hline
     \multicolumn{1}{|r||}{\scriptsize Total primitives}             &\scriptsize 90    &\scriptsize 85   &\scriptsize 104   &\scriptsize 101    &\scriptsize 92    &\scriptsize 100   &\scriptsize 87    &\scriptsize 105    &\scriptsize 80    &\scriptsize 98  &\scriptsize  94.2 \\
     \hline
     \multicolumn{1}{|r||}{\scriptsize Added interspersings}         &\scriptsize 2     &\scriptsize 2    &\scriptsize 4     &\scriptsize 4      &\scriptsize 3     &\scriptsize 3     &\scriptsize 2     &\scriptsize 2      &\scriptsize 2     &\scriptsize 4   &\scriptsize  2.8  \\
     \hline
     \multicolumn{1}{|r||}{\scriptsize Missed in the control set}    &\scriptsize 5     &\scriptsize 5     &\scriptsize 2     &\scriptsize 1     &\scriptsize 4     &\scriptsize 2     &\scriptsize 5     &\scriptsize 8      &\scriptsize 9     &\scriptsize 16  &\scriptsize  5.7  \\
     \hline
     \multicolumn{1}{|r||}{\scriptsize Missed in the test set}       &\scriptsize 7     &\scriptsize 9    &\scriptsize 15    &\scriptsize 12     &\scriptsize 12    &\scriptsize 12    &\scriptsize 13    &\scriptsize 13     &\scriptsize 11    &\scriptsize 19  &\scriptsize  12.3 \\
     \hline
     \multicolumn{1}{|r||}{\scriptsize Absolute difference}          &\scriptsize 2     &\scriptsize 4    &\scriptsize 13    &\scriptsize 11     &\scriptsize 8     &\scriptsize 10    &\scriptsize 8     &\scriptsize 5      &\scriptsize 2     &\scriptsize 3   &\scriptsize  6.6 \\
     \hline
     \multicolumn{1}{|r||}{\scriptsize Missed prim./intersp.}        &\scriptsize 1     &\scriptsize 2    &\scriptsize 3.25  &\scriptsize 2.75   &\scriptsize 2.67  &\scriptsize 3.33  &\scriptsize 4     &\scriptsize 2.5    &\scriptsize 1     &\scriptsize 0.75&\scriptsize  2.33 \\
     \hline
     \hline
   \end{tabular}
 \end{center}
 \caption{This table shows the number of misrecognized primitives per interspersing. The first two rows show the number of total primitives in each sketch and
 the number of added interspersings. Next two rows show the number of primitives incorrectly recognized when our system is run on the control and test sets. The next row shows the number of primitives that our system misses because it cannot handle interspersings. The last row shows the number of primitives missed per interspersing.}
 \label{per-interspersing-results}
\end{table*}



\section{RELATED WORK}
%There is a large body of work in sketch recognition that does not use temporal
%features. As these approaches are complementary to ours, we keep their discussion 
% brief and focus on  systems that do use temporal features.
There is a large body of work in sketch recognition that does not use temporal
features. These approaches are complementary to ours. Most of these algorithms 
are based on structural and syntactic methods. Structural pattern 
recognition methods (also known as model-based or template-based methods) 
formulate the recognition problem based on implicit or explicit representations 
of objects in terms of their structure (i.e., components, sub-components and 
their relationships). Examples include work in 
\cite{christine_thesis}, \cite{shilman} and \cite{mahoney_ss}. 
The main drawback of such methods is their high computational requirements that 
make them unsuitable for real-time recognition of realistic sketches. These models 
also require experts to specify domain knowledge or manually input structural 
descriptions of the objects in the domain. This is further complicated by the fact 
that not everyone draws objects the same. For example, Fig.~\ref{transistors} shows 
how transistors drawn by four different participants were structurally different. Thus 
in a structural recognition setting, the expert may have to create different 
object descriptions for different users. Our system learns each variation
from examples.

Other non-temporal recognition algorithms include work in \cite{kara_sr} which 
requires domain information to be entered by an expert. Again our system does
not require expert knowledge. Viola et al. describe a spatial recognition and 
grouping method for text and graphics \cite{viola_shilman}. 
They make some assumptions about the objects in the domain in order to keep the search
tractable (e.g., objects in their domain have no more than 8 strokes, or that the 
strokes constituting an object are within a certain distance). These assumptions seem 
to work for flowchart-like drawings where the connectors and objects are sufficiently 
separated from one another, but the practicality of these algorithms is yet to be 
demonstrated in general for domains where objects vary in size and shape where 
assumptions on the object size and scale may not hold.

\begin{figure}
  \centerline{
    \hbox{ 
      \includegraphics[width=3in]{figures/circuits/transistors}
    }
  }
  \caption{Examples showing the structural variation across transistors drawn by 
    different users. Each column shows two transistors drawn by a different user.
    Note how the route traced by the pen when drawing the emitter and the current
    direction.}
  \label{transistors} \end{figure}







%% There is a large body of work in sketch recognition that does not use temporal
%% features (\cite{christine_thesis,shilman,mahoney_ss,kara_sr} just 
%% to count a few). Two of these systems report results on circuit recognition 
%% \cite{christine_thesis,kara_sr}. Although a direct comparison of correct recognition 
%% rates with these systems is complicated by several reasons,\footnote{The data-sets 
%% used with these systems are either not readily available or are too small
%% for our approach which uses machine learning to estimate model parameters.
%% Furthermore we learn everything from training data and do not require hand-crafted 
%% domain specific knowledge or segmentation techniques.} on average our system has 
%% comparable or better recognition rates. Also our system uses temporal features 
%% and is complementary to these systems, so we limit our discussion here to systems 
%% that do use temporal features.

The handwriting recognition community has come up with a large number of methods 
for recognizing single stroke or segmented pen input using HMMs. These systems 
take advantage of a vast number of robust preprocessing segmentation methods to 
simplify the problem to an isolated recognition problem.

 
Several systems have used HMM or HHMM based frameworks to recognize pen input. In the sketch 
recognition community, work in \cite{anderson_hmm_ss} describes an HMM-based
symbol recognizer that uses chain-code-like features to recognize isolated symbols.
Our work does not assume segmented input and builds a joint model for complete
sketches. 

Our previous work describes an HMM based sketch recognition system that 
does segmentation and recognition by combining outputs of individual HMMs using
dynamic programming (DP) \cite{sezgin_iui05}. This method is a weaker version of our baseline method in that it
does not support continuous features. Its two layer HMM/DP recognition method suffers from 
the $O(T^3)$ complexity, just like HHMMs. 

In \cite{simhon_dudek}, the authors present a sketch interpretation and curve refinement
system that uses an HHMM. This work shares the same motivation as ours. In their
work, they assume that the parameters of the object-level process (which they refer
to as the scene level) is supplied by the user using a semantic graph 
representation. We learn the parameters of the stroke-level and object-level
patterns from data, and use the more efficient DBN representation to avoid the 
$O(T^3)$ complexity of the HHMMs. We also support multi-stroke objects
and our use of real-valued continuous observations allow using a rich set of 
features.


\section{CONTRIBUTIONS}
Until very recently, temporal models were considered to be unsuitable for sketch
recognition \cite{christine_thesis}. We have presented a sketch recognition framework 
based on dynamic Bayesian networks that demonstrates we can recognize sketches by 
exploiting both stroke-level and object-level temporal orderings. Our framework allows 
learning stroke-level and object-level patterns from data, handles multi-stroke objects 
and multi-object strokes efficiently, and supports continuous observable features. We 
showed how graphical models can be used to implement our model efficiently, and showed 
that a specialized inference algorithm known as the Lauritzen-Jensen stable conditional 
Gaussian belief propagation should be used to avoid numerical instabilities in recognition. 
We have demonstrated that modeling object-level patterns improves classification performance 
for the electronic circuit diagrams domain. Finally we presented a framework that allows joint
parameter estimation for the stroke-level and the object-level 
processes. This avoids the task of training the models separately and then stitching them 
together, as it is usually done in HHMM-based frameworks.

\section{FUTURE DIRECTIONS}
An interesting avenue to explore would be to develop and evaluate a 
complete pen-based interface to a computer-aided circuit design tool 
using our system as the recognition engine. This would allow us to 
quantify the subjective utility of recognition and evaluate
user reactions in a real design setting. Issues that need to be explored
include the editing method (modeless vs. modal), display of the 
recognition results (delayed vs. immediate feedback) and their 
effect on the drawing behavior (e.g., do users still intersperse
objects). Similar experiments can also be carried out for other domains
such as UML diagrams, web page design, mechanical engineering. 


One limitation of our model is its assumption that the user completes drawing 
one object before moving on to the next. The evaluation results presented above
indicate that violation of this assumption will almost always lead to a significant
number of misrecognitions. We are developing ways of modifying our 
model such that it handles interspersed drawing. The general idea is to make
the fact that users can intersperse certain types of objects an explicit part of 
the DBN model. This can be done by extending the state space of each time slice 
with a new variable that tracks if the user is interspersing any objects, and 
if so, what objects are being interspersed (e.g., the user is interspersing 
wires and transistors). Early results that we have obtained show that it is 
possible to extend the state space of our model using the parsimonious 
representation property of DBNs and obtain a model where both training and inference 
remains tractable \cite{sezgin_phd_thesis}. 

The motivation for our work was to identify the kinds of temporal 
patterns present in online sketches and to present an approach 
that can perform recognition based on such patterns. We have illustrated 
that sketches contain a fair amount of rich temporal patterns that can be 
used for recognition. However, we don't expect temporal patterns to be used
alone. Ultimately, we envision a unified recognition architecture that can 
combine the efficient segmentation and recognition features of temporal 
approaches with the rich constraints and representational power of shape-based 
models. Our choice of modeling the sketching process as a first-order Markov 
process does not allow us to exploit spatial or geometric constraints 
between arbitrary components of a shape. A na\"{\i}ve attempt to incorporate 
such constraints using higher-order Markov processes is not practical because
the state-space of our model increases exponentially with the order of the process. 
We believe a better approach would be to explore ways of 
incorporating shape information by connecting external recognizers that use 
spatial or structural information into our model using a mechanism known as 
virtual evidence nodes in the graphical models machine learning community
\cite{bilmes_ve}. Another approach would be to learn a composite function of
spatial and temporal information using a dynamic programming framework as 
described in \cite{sezgin_phd_thesis}. This would be a first step toward combining 
temporal and spatial/structural information for sketch recognition. 





%% As mentioned above, our approach is based on drawing orders and not shape. 
%% Our choice of modeling the sketching process as a first-order Markov 
%% process does not allow us to exploit spatial or geometric constraints 
%% between arbitrary components of a shape. Although it is possible 
%% to use a higher order Markov process to model such constraints, this is 
%% not practical because the state-space of our model increases exponentially with 
%% the order of the process. We believe a better approach would be to explore ways of 
%% incorporating shape information by connecting external recognizers that use 
%% spatial or structural information into our model using a mechanism known as 
%% virtual evidence nodes in the graphical models machine learning community
%% \cite{bilmes_ve}. This would be a first step toward combining 
%% temporal and spatial/structural information for sketch recognition. 



\section{ACKNOWLEDGEMENTS}
This work was supported in part by the MIT/Microsoft iCampus Project. We would like 
to thank the anonynomous referees for valuable feedback that improved the 
presentation and the content of this manuscript.

\bibliography{cga} 
\bibliographystyle{mlapa} 


\end{document}

