Spoken Conversational Interface for Language Learning
(SCILL)
SCILL is a collaborative research project between the MIT CSAIL Spoken
Language Systems Group and the Cambridge Machine Intelligence
Laboratory, whose goal is to provide a framework for practicing
conversation in a language being learned through interactive dialogue
with a computer.
The project is funded within the framework of the Pervasive Computing
Knowledge Integration Community (KIC), over the time interval from
November 2003 to September 2006.
Web-based
Demo System for Practicing Mandarin Chinese in the Flight Domain
Video clip illustrating spoken interaction with weather domain spoken dialogue system
Video clip illustrating language practice in flight domain
Audio clip illustrating spoken interaction in weather domain
Audio clip illustrating spoken language translation game in flight domain
Project Aims
The aim of this project is to develop a language learning system with
four major components:
-
an intelligent agent capable of maintaining an interactive dialogue
with the student in the target language. This agent would act as the
"native speaker" versed on some topic selected from a pre-defined set.
- a second intelligent agent capable of translating within the
topic
domain between the student's native language and the target language.
This agent will act as an on-line "tutor" giving advice on how to say
required words and phrases.
- an assessment component which provides a post-mortem analysis of
the
conversation and gives feedback on errors and areas to improve.
- a set of tools which allow second-language teachers to author
specific dialogues and scenarios
In practice, the target language will be Mandarin Chinese and the
native
language will be US or UK English. However, the system design will as
far as possible be language independent.
The system will enable a student to participate in a dialogue with the
system in Mandarin whilst simultaneously having access to a "tutor"
that could tell them how to say certain phrases. For example, the
topic might be about the weather in a particular city, and the
bi-lingual "tutor" would provide the student with helpful hints on how
to communicate in Mandarin with the system. This allows the student
to engage in practice conversation in a non-threatening environment.
The student would be able to gauge their success by the degree to
which the system understood their Mandarin queries, and the number of
times they needed to consult the "tutor" for translation advice.
A later off-line interaction using the assessment component would
allow the student to re-examine their speech. The system could provide
feedback on their overall pronunciation quality, as well as
identifying words that were poorly enunciated, allowing them to
compare their pronunciation with a standard. The system could also
conceivably repair the tone production, while preserving the overall
quality of the student's voice.
Key Project Personnel
MIT
- Stephanie Seneff (co-PI at MIT) seneff@csail.mit.edu
- James Glass (co-PI at MIT) jrg@mit.edu
- Chao Wang wangc@sls.csail.mit.edu
Cambridge
- Steve Young (PI at CAMBRIDGE) sjy@eng.cam.ac.uk
- Yulan He yh213@eng.cam.ac.uk
- Hui (KK) Ye hy216@eng.cam.ac.uk
Presentation and Publications
Presentations:
Papers:
-
Stephanie Seneff, Chao Wang, and John Lee,
"Combining Linguistic and Statistical Methods for
Bi-directional English Chinese Translation in the Flight
Domain",
submitted to AMTA (Association for Machine Translation in the
Americas), 2006.
-
Stephanie Seneff, Chao Wang, Mitchell Peabody, and Victor Zue
"Second Language Acquisition through Human Computer
Dialogue",
in Proc. of ISCSLP, 2004.
-
Stephanie Seneff, Chao Wang, and Julia Zhang
"Spoken Conversational Interaction for Language Learning"
,
in Proc. of InSTIL/ICALL Symposium, 2004.
-
Chao Wang and Stephanie Seneff
"High-Quality Speech Translation for Language Learning",
in Proc. of InSTIL/ICALL Symposium, 2004.
-
Mitch Peabody, Stephanie Seneff, and Chao Wang
"Mandarin Tone Acquisition through Typed",
in Proc. of InSTIL/ICALL Symposium, 2004.
Thesis: