Overall summary: ---------------- At our first meeting on Mon. over dinner, we had some great ceviche and nachos. This did not produce much technical discussion :) However, we did get some great suggestions from our advisors Katrin and Eric (all suggestions are summarized below). At our second meeting (just Mark, Simon, and Karen), we skipped the food. This was much more productive in terms of technical details. We determined the set of acoustic models we'd like to compare, which differ along several dimensions: -Type of classifier: general Gaussian mixtures, neural networks, SVMs -For GM case, factored vs. unfactored model (i.e. separate P(obs|F_i) for each feature F_i vs. one big distribution P(obs|F_1, ..., F_N)) -For NNs/SVMs, soft evidence vs. using discriminant values as observations of a GM These all assume a pronunciation model that maps words to articulatory feature values. One last option is to ditch the explicit pronunciation model, and to instead use whole-word models consisting of N hidden state streams per word, with each stream having as observations the discriminant values of one articulatory feature classifier (NN/SVM). This is the architecture used in Saenko et al., CVPR 2005. We also discussed whether or not we need to implement state tying for this project. One option would be to use a single pool of Gaussians, with states differing only in their weight distributions. Finally, we discussed different ways of modeling asynchrony between hidden feature streams. One way is as previously done by Karen, grouping features into subsets and imposing a distribution on differences between mean indices in the subsets. Another suggestion was to have only one "asynchrony variable" each of whose values represents a different combination of feature indices; this would allow for more freedom and simplicity, but the variable would have a very large CPT (which would hopefully be sparse, however). Finally, we could have direct links from the index variable of one feature to the index var of another feature in the next frame. There seemed to be the most support for the second option. At our last meeting (Mark, Simon, Jeff, Katrin, Karen), we mainly discussed the need (or lack thereof) for state tying, and how this could be done as a GMTK add-on. More detailed outcomes described below. Suggestions: ------------ -Devise a way of determining, for a given task, what phenomena need to be modeled for the best performance (i.e. which are discriminatory); we need not necessarily model all of human speech articulation to improve ASR on a given task. -Do a manual feature-level transcription of part of the Buckeye corpus as a source of "ground truth"? We could bootstrap Simon et al.'s articulatory DBN recognizers on this data, then measure how well they automatically transcribe the data. If it is well enough, we can use these DBNs to automatically generate a larger set of "ground truth" for further experiments. Questions that came up: ----------------------- -Is the reduction of error rate (in some domain) a goal of this project? Or rather the exploration of the space of possible feature-based models? Or both? Depending on the answers, different experimental approaches may be appropriate. -Is a baseline that gets ~70% WER (the current performance of Karen's baseline, and of a 3-state HMM-based model, on the 100-word SVitchboard task) useful? Do we need to make sure to have stronger baselines before the start of the workshop? Things to do in the near future: -------------------------------- -Karen, Simon, Mark: Determine two feature sets (one for pron model, one for acoustic model) that we will use -Karen, Eric: Discuss the feasibility & cost of transcribing part of the Buckeye corpus using feature labels of the kind we will use -Simon, Jeff, Karen: Work on clustering tool for GMTK -Karen: Put together baseline numbers on SVitchboard, for both a basic feature-based model and a simple HMM-based one (the same comparison done in Karen's thesis experiments, which yielded a feat-based model that performs about as well as a 3-state context-independent HMM-based model; this needs to be extended to other SVitchboard sets)