Perceiving foreign-accented auditory-visual speech in noise

The influence of visual form and timing information

Saya Kawase, Jeesun Kim, Vincent Aubanel, Chris Davis

Research output: Contribution to journalArticle

Abstract

The present study examined the extent to which visual form and timing information assisted in the perception of native English and Japanese-accented English speech in noise. We also examined whether the degree of visual facilitation would be mediated by the talkers’ English experience. Thirty native Australian English listeners performed a speech perception in noise task with English sentences produced by inexperienced and experienced Japanese talkers as well as a native English talker. The Japanese speakers were selected from a previous study where acoustic analyses showed that the speech rhythm of the inexperienced talker was more influenced by their native language than to the experienced one. The stimulus sentences were presented under the three conditions: Audioonly, Audio-visual (visual form and timing) and Audio-visual with mouth covered (visual timing only). The results showed a visual timing facilitation effect for the stimuli produced by the experienced but not in the inexperienced Japanese talker. A facilitative form effect was found for all the talker groups but the size of this effect decreased as the degree of the non-native experience decreased. Our findings illustrate the influence of L2 talker’s experience on the effectiveness of their visual form and timing cues.

Original languageEnglish
Pages (from-to)487-488
Number of pages2
JournalProceedings of the International Conference on Speech Prosody
Volume2016-January
Publication statusPublished - 2016
Externally publishedYes

Fingerprint

stimulus
experience
listener
acoustics
language
Talkers
Hearing
Group
Facilitation
Stimulus

Keywords

  • Auditory and visual speech processing
  • Foreign-accented speech perception
  • Form and timing
  • Visual speech

ASJC Scopus subject areas

  • Language and Linguistics
  • Linguistics and Language

Cite this

Perceiving foreign-accented auditory-visual speech in noise : The influence of visual form and timing information. / Kawase, Saya; Kim, Jeesun; Aubanel, Vincent; Davis, Chris.

In: Proceedings of the International Conference on Speech Prosody, Vol. 2016-January, 2016, p. 487-488.

Research output: Contribution to journalArticle

@article{b10a81f453b0429dbb8b94e78d365324,
title = "Perceiving foreign-accented auditory-visual speech in noise: The influence of visual form and timing information",
abstract = "The present study examined the extent to which visual form and timing information assisted in the perception of native English and Japanese-accented English speech in noise. We also examined whether the degree of visual facilitation would be mediated by the talkers’ English experience. Thirty native Australian English listeners performed a speech perception in noise task with English sentences produced by inexperienced and experienced Japanese talkers as well as a native English talker. The Japanese speakers were selected from a previous study where acoustic analyses showed that the speech rhythm of the inexperienced talker was more influenced by their native language than to the experienced one. The stimulus sentences were presented under the three conditions: Audioonly, Audio-visual (visual form and timing) and Audio-visual with mouth covered (visual timing only). The results showed a visual timing facilitation effect for the stimuli produced by the experienced but not in the inexperienced Japanese talker. A facilitative form effect was found for all the talker groups but the size of this effect decreased as the degree of the non-native experience decreased. Our findings illustrate the influence of L2 talker’s experience on the effectiveness of their visual form and timing cues.",
keywords = "Auditory and visual speech processing, Foreign-accented speech perception, Form and timing, Visual speech",
author = "Saya Kawase and Jeesun Kim and Vincent Aubanel and Chris Davis",
year = "2016",
language = "English",
volume = "2016-January",
pages = "487--488",
journal = "Proceedings of the International Conference on Speech Prosody",
issn = "2333-2042",

}

TY - JOUR

T1 - Perceiving foreign-accented auditory-visual speech in noise

T2 - The influence of visual form and timing information

AU - Kawase, Saya

AU - Kim, Jeesun

AU - Aubanel, Vincent

AU - Davis, Chris

PY - 2016

Y1 - 2016

N2 - The present study examined the extent to which visual form and timing information assisted in the perception of native English and Japanese-accented English speech in noise. We also examined whether the degree of visual facilitation would be mediated by the talkers’ English experience. Thirty native Australian English listeners performed a speech perception in noise task with English sentences produced by inexperienced and experienced Japanese talkers as well as a native English talker. The Japanese speakers were selected from a previous study where acoustic analyses showed that the speech rhythm of the inexperienced talker was more influenced by their native language than to the experienced one. The stimulus sentences were presented under the three conditions: Audioonly, Audio-visual (visual form and timing) and Audio-visual with mouth covered (visual timing only). The results showed a visual timing facilitation effect for the stimuli produced by the experienced but not in the inexperienced Japanese talker. A facilitative form effect was found for all the talker groups but the size of this effect decreased as the degree of the non-native experience decreased. Our findings illustrate the influence of L2 talker’s experience on the effectiveness of their visual form and timing cues.

AB - The present study examined the extent to which visual form and timing information assisted in the perception of native English and Japanese-accented English speech in noise. We also examined whether the degree of visual facilitation would be mediated by the talkers’ English experience. Thirty native Australian English listeners performed a speech perception in noise task with English sentences produced by inexperienced and experienced Japanese talkers as well as a native English talker. The Japanese speakers were selected from a previous study where acoustic analyses showed that the speech rhythm of the inexperienced talker was more influenced by their native language than to the experienced one. The stimulus sentences were presented under the three conditions: Audioonly, Audio-visual (visual form and timing) and Audio-visual with mouth covered (visual timing only). The results showed a visual timing facilitation effect for the stimuli produced by the experienced but not in the inexperienced Japanese talker. A facilitative form effect was found for all the talker groups but the size of this effect decreased as the degree of the non-native experience decreased. Our findings illustrate the influence of L2 talker’s experience on the effectiveness of their visual form and timing cues.

KW - Auditory and visual speech processing

KW - Foreign-accented speech perception

KW - Form and timing

KW - Visual speech

UR - http://www.scopus.com/inward/record.url?scp=84982881441&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84982881441&partnerID=8YFLogxK

M3 - Article

VL - 2016-January

SP - 487

EP - 488

JO - Proceedings of the International Conference on Speech Prosody

JF - Proceedings of the International Conference on Speech Prosody

SN - 2333-2042

ER -