A WoZ Study for an Incremental Proficiency Scoring Interview Agent Eliciting Ratable Samples

Mao Saeki*, Weronika Demkow, Tetsunori Kobayashi, Yoichi Matsuyama

*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

To assess the conversational proficiency of language learners, it is essential to samples that are representative of the learner’s full linguistic ability. This is realized through the adjustment of oral interview questions to the learner’s perceived proficiency level. An automatic system eliciting ratable samples must incrementally predict the approximate proficiency from a few turns of dialog and employ an adaptable question generation strategy according to this prediction. This study investigates the feasibility of such incremental adjustment of oral interview question difficulty during the interaction between a virtual agent and learner. First, we create an interview scenario with questions designed for different levels of proficiency and collect interview data using a Wizard-of-Oz virtual agent. Next, we build an incremental scoring model and analyze the accuracy. Finally, we discuss the future direction of automated adaptive interview system design.

Original languageEnglish
Title of host publicationConversational AI for Natural Human-Centric Interaction - 12th International Workshop on Spoken Dialogue System Technology, IWSDS 2021
EditorsSvetlana Stoyanchev, Stefan Ultes, Haizhou Li
PublisherSpringer Science and Business Media Deutschland GmbH
Pages193-201
Number of pages9
ISBN (Print)9789811955372
DOIs
Publication statusPublished - 2022
Event12th International Workshop on Spoken Dialogue System Technology, IWSDS 2021 - Virtual, Online
Duration: 2021 Nov 152021 Nov 17

Publication series

NameLecture Notes in Electrical Engineering
Volume943
ISSN (Print)1876-1100
ISSN (Electronic)1876-1119

Conference

Conference12th International Workshop on Spoken Dialogue System Technology, IWSDS 2021
CityVirtual, Online
Period21/11/1521/11/17

ASJC Scopus subject areas

  • Industrial and Manufacturing Engineering

Fingerprint

Dive into the research topics of 'A WoZ Study for an Incremental Proficiency Scoring Interview Agent Eliciting Ratable Samples'. Together they form a unique fingerprint.

Cite this