Investigating the impact of automated transcripts on non-native speakers' listening comprehension

Xun Cao, Naomi Yamashita, Toru Ishida

研究成果: Conference contribution

3 被引用数 (Scopus)

抄録

Real-Time transcripts generated by automatic speech recognition (ASR) technologies hold potential to facilitate non-native speakers' (NNSs) listening comprehension. While introducing another modality (i.e., ASR transcripts) to NNSs provides supplemental information to understand speech, it also runs the risk of overwhelming them with excessive information. The aim of this paper is to understand the advantages and disadvantages of presenting ASR transcripts to NNSs and to study how such transcripts affect listening experiences. To explore these issues, we conducted a laboratory experiment with 20 NNSs who engaged in two listening tasks in different conditions: Audio only and audio+ASR transcripts. In each condition, the participants described the comprehension problems they encountered while listening. From the analysis, we found that ASR transcripts helped NNSs solve certain problems (e.g., "do not recognize words they know"), but imperfect ASR transcripts (e.g., errors and no punctuation) sometimes confused them and even generated new problems. Furthermore, post-Task interviews and gaze analysis of the participants revealed that NNSs did not have enough time to fully exploit the transcripts. For example, NNSs had difficulty shifting between multimodal contents. Based on our findings, we discuss the implications for designing better multimodal interfaces for NNSs.

本文言語English
ホスト出版物のタイトルICMI 2016 - Proceedings of the 18th ACM International Conference on Multimodal Interaction
編集者Catherine Pelachaud, Yukiko I. Nakano, Toyoaki Nishida, Carlos Busso, Louis-Philippe Morency, Elisabeth Andre
出版社Association for Computing Machinery, Inc
ページ121-128
ページ数8
ISBN(電子版)9781450345569
DOI
出版ステータスPublished - 2016 10月 31
外部発表はい
イベント18th ACM International Conference on Multimodal Interaction, ICMI 2016 - Tokyo, Japan
継続期間: 2016 11月 122016 11月 16

出版物シリーズ

名前ICMI 2016 - Proceedings of the 18th ACM International Conference on Multimodal Interaction

Conference

Conference18th ACM International Conference on Multimodal Interaction, ICMI 2016
国/地域Japan
CityTokyo
Period16/11/1216/11/16

ASJC Scopus subject areas

  • コンピュータ サイエンスの応用
  • 人間とコンピュータの相互作用
  • ハードウェアとアーキテクチャ
  • コンピュータ ビジョンおよびパターン認識

フィンガープリント

「Investigating the impact of automated transcripts on non-native speakers' listening comprehension」の研究トピックを掘り下げます。これらがまとまってユニークなフィンガープリントを構成します。

引用スタイル