Laura Bosdriesz – “Adding Speech to Dialogues with a Council of Coaches”


StudentLaura Bosdriesz
StudyInteraction Technology, University of Twente (the Netherlands)
Host OrganisationRoessingh Research and Development
Finalized DateIn progress…


Currently in COUCH the user interacts with the coaches by selecting one of several predefined multiple-choice responses on a tablet or computer interface. Although this is a reliable way to capture input from the user, it may not be ideal for COUCH its target user group of older adults. Spoken dialogues might improve the user experience of COUCH for older adults.

In this project the COUCH system is adapted to support spoken interactions. Based on the research topics, the choice is made to use the NLSpraak speech recognizer for understanding the older adult’s speech. In addition to incorporating automatic speech recognition, smart ways are investigated to organise the dialogue to facilitate adequate recognition in noisy and uncertain settings while keeping the conversation going. Finally, the user experience and the quality of dialogue progress in various settings is evaluated and compared to the original COUCH system and thereby a conclusion will be drawn about the robustness of state-of-the-art speech recognition in home settings.

Results & Outcomes

The assignment is still going on, so no final conclusions can be drawn yet. So far, the work has focused on the implementation of the NLSpraak speech recognizer and the Google text-to-speech synthesizer. First there has been looked into ways to deal with the spoken speech, starting with the recognizer listening to the first word of a sentence. This has been changed to the last word of a sentence and in some cases, manually adding keywords to the WOOL dialogue files. Furthermore, strategies to organise the interaction in cases when the speech is not appropriately recognized are implemented.