CALL-SLT: A Spoken CALL System based on grammar and speech recognition

Manny Rayner, Nikos Tsourakis, Claudia Baur, Pierrette Bouillon, Johanna Gerlach


We describe CALL-SLT, a speech-enabled Computer-Assisted Language
Learning application where the central idea is to prompt the student
with an abstract representation of what they are supposed to say, and
then use a combination of grammar-based speech recognition and
rule-based translation to rate their response. The system has been
developed to the level of a mature prototype, freely deployed on the
web, with versions for several languages. We present an overview of
the core system architecture and the various types of content we have
developed. Finally, we describe several evaluations, the last of which
is a study carried out over about a week using 130 subjects recruited
through the Amazon Mechanical Turk, in which CALL-SLT was contrasted
against a control version where the speech recognition component was
disabled. The improvement in student learning performance between the
two groups was significant at p < 0.02.


CALL, speech, grammar, evaluation

Full Text:



Baur, C., M. Rayner, and N. Tsourakis. 2014. Using a serious game to collect

a child learner speech corpus. In Proceedings of LREC 2014 . Reykjavik,


Bouillon, P., C. Cervini, A. Mandich, M. Rayner, and N. Tsourakis. 2011a.

Speech recognition for online language learning: Connecting CALL-SLT

and DALIA. In Proceedings of the International Conference on ICT for

Language Learning. Florence, Italy.

Bouillon, P., J. Gerlach, C. Baur, C. Cervini, and R.B. Gasser. 2012.

Intégration d’un jeu de traduction orale sur le web pour l’apprentissage

d’une langue seconde. In Proceedings of TICE 2012 . Lyon, France.

Bouillon, P., M. Rayner, B. Novellas, M. Starlander, M. Santaholma,

Y. Nakao, and N. Chatzichrisafis. 2007. Une grammaire partagée multitâche pour le traitement de la parole: application aux langues romanes.


Bouillon, P., M. Rayner, N. Tsourakis, and Q. Zhang. 2011b. A studentcentered

evaluation of a web-based spoken translation game. In Proceedings

of the SLaTE Workshop. Venice, Italy.

Coyne, B., C. Schudel, M. Bitz, and J. Hirschberg. 2011. Evaluating a textto-

scene generation system as an aid to literacy. In Speech and Language

Technology in Education.

Eskenazi, M., G.-A. Levow, H. Meng, G. Parent, and D. Suendermann, eds.

Crowdsourcing for Speech Processing: Applications to Data Collec-

tion, Transcription and Assessment . John Wiley & Sons.

Franco, H., H. Bratt, R. Rossier, V. Rao Gadde, E. Shriberg, V. Abrash,

and K. Precoda. 2010. Eduspeak: A speech recognition and pronunciation

scoring toolkit for computer-aided language learning applications.

Language Testing 27(3):401.

Fuchs, M., N. Tsourakis, and M. Rayner. 2012. A scalable architecture for

web deployment of spoken dialogue systems. In Proceedings of LREC 2012 .

Istanbul, Turkey.

Gruenstein, A., I. McGraw, and I. Badr. 2008. The WAMI toolkit for developing,

deploying, and evaluating web-accessible multimodal interfaces. In

Proceedings of the 10th international conference on Multimodal interfaces,

pages 141–148. ACM.

Johnson,W.L. and A. Valente. 2009. Tactical Language and Culture Training

Systems: using AI to teach foreign languages and cultures. AI Magazine


Jurcicek, F., S. Keizer, M. Gasic, F. Mairesse, B. Thomson, K. Yu, and

S. Young. 2011. Real user evaluation of spoken dialogue systems using

Amazon Mechanical Turk. In Proceedings of Interspeech 2011 . Florence,


Kennedy, C. H. 2005. Single-Case Designs for Educational Research. Allyn

and Bacon.

Korsah, G.A., J. Mostow, M.B. Dias, T.M. Sweet, S.M. Belousov, M.F. Dias,

and H. Gong. 2010. Improving child literacy in Africa: Experiments with

an automated reading tutor. Information Technologies and International

Development 6(2):1–19.

Kulik, J., C.Kulik, and P. Cohen. 1980. Effectiveness of computer-based college

teaching : A meta-analysis of findings. Review of Educational Research


Poulsen, R. 2004. Tutoring Bilingual Students With an Automated Reading

Tutor That Listens: Results of a Two-Month Pilot Study. DePaul University,

Chicago, IL: Masters Thesis.

Rayner, M., P. Bouillon, and J. Gerlach. 2012a. Evaluating appropriateness

of system responses in a spoken CALL game. In Proceedings of LREC

. Istanbul, Turkey.

Rayner, M., P. Bouillon, B.A. Hockey, and Y. Nakao. 2008. Almost flat

functional semantics for speech translation. In Proceedings of COLING-

. Manchester, England.

Rayner, M., P. Bouillon, N. Tsourakis, J. Gerlach, M. Georgescul, Y. Nakao,

and C. Baur. 2010. A multilingual CALL game based on speech translation.

In Proceedings of LREC 2010 . Valetta, Malta.

Rayner, M., I. Frank, C. Chua, N. Tsourakis, and P. Bouillon. 2011. For

a fistful of dollars: Using crowd-sourcing to evaluate a spoken language

CALL application. In Proceedings of the SLaTE Workshop. Venice, Italy.

Rayner, M., J. Gerlach, M. Starlander, N. Tsourakis, A. Kruckenberg, R. Eklund,

A. J¨onsson, and A. McAllister. 2012b. A web-deployed Swedish spoken

CALL system based on a large shared English/Swedish feature grammar.

In Proceedings of the SLTC Workshop on NLP for CALL. Lund,


Rayner, M., B.A. Hockey, and P. Bouillon. 2006. Putting Linguistics into

Speech Recognition: The Regulus Grammar Compiler. Chicago: CSLI


Reeder, K., J. Shapiro, and J. Wakefield. 2007. The effectiveness of speech

recognition technology in promoting reading proficiency and attitudes for

Canadian immigrant children. In 15th European Conference on Reading.

Stent, A., J. Dowding, J. Gawron, E. Bratt, and R. Moore. 1999. The CommandTalk

spoken dialogue system. In Proceedings of the Thirty-Seventh

Annual Meeting of the Association for Computational Linguistics, pages


Tsourakis, N., M. Rayner, and C. Baur. 2014. Formative feedback in an

interactive spoken CALL system. In Proceedings of The Second Workshop

on Child-Computer Interaction. Singapore.

Wang, C. and S. Seneff. 2007. Automatic assessment of student translations

for foreign language tutoring. In Proceedings of NAACL/HLT 2007 .

Rochester, NY.


  • There are currently no refbacks.