Development of multimodal discourse comprehension: cohesive use of space by gestures

Kazuki Sekine, Sotaro Kita

Research output: Contribution to journalArticle

4 Citations (Scopus)

Abstract

This study examined how well 5-, 6-, 10-year-olds and adults integrated information from spoken discourse with cohesive use of space in gesture, in comprehension. In Experiment 1, participants were presented with a combination of spoken discourse and a sequence of cohesive gestures, which consistently located each of the two protagonists in two distinct locations in gesture space. Participants were asked to select an interpretation of the final sentence that best matched the preceding spoken and gestural contexts. Adults and 10-year-olds performed better than 5-year-olds, who were at chance level. In Experiment 2, another group of 5-year-olds was presented with the same stimuli as in Experiment 1, except that the actor showed hand-held pictures, instead of producing cohesive gestures. Unlike cohesive gestures, one set of pictures was self-explanatory and did not require integration with the concurrent speech to derive the referent. With these pictures, 5-year-olds performed nearly perfectly and their performance in the identifiable pictures was significantly better than those in the unidentifiable pictures. These results suggest that young children failed to integrate spoken discourse and cohesive use of space in gestures, because they cannot derive a referent of cohesive gestures from the local speech context.

Original languageEnglish
Pages (from-to)1245-1258
Number of pages14
JournalLanguage, Cognition and Neuroscience
Volume30
Issue number10
DOIs
Publication statusPublished - 2015 Nov 26
Externally publishedYes

Keywords

  • discourse
  • gesture
  • multimodal communication
  • preschool children
  • speech

ASJC Scopus subject areas

  • Language and Linguistics
  • Experimental and Cognitive Psychology
  • Linguistics and Language
  • Cognitive Neuroscience

Fingerprint Dive into the research topics of 'Development of multimodal discourse comprehension: cohesive use of space by gestures'. Together they form a unique fingerprint.

  • Cite this