Some possibilities for asynchrony modeling

The figures show just the sub-part of the structure consisting of the state indices and synchrony constraints, for two frames. These figures assume three feature streams (e.g. they could be the L, T, G features from Feature Set 2).

Separate synchrony constraints for separate feature subsets

This is the model used in Livescu & Glass. Here the 3 features are divided into two subsets of two features each, but the constraints could also be hierarchical instead (as in the paper above). The value of each async variable is the degree of asynchrony (state index difference) between the associated features in the current frame. The chkSync variables' distribution is P(chkSync=1 | async, i1, i2) = 1 if |i1 - i2| = async; 0 otherwise.

Single asynchrony constraint for all features

Just one async/chkSync variable for all features, whose distributions represent the overall "goodness" of different state index configurations. There are many ways in which this could be done. E.g. we could have a distribution over the maximum deviation of each index from the mean, or over the vector of differences among indices (in which case async would be a vector).

Coupled HMM-style model

Each feature's state transition probabilities depend on some of the other features' values in the previous frame. Here each feature's transitions depend on one other feature, but there could also be more dependencies.

-- KarenLivescu - 05 May 2006