Gestural cue analysis in automated semantic miscommunication annotation

Masashi Inoue, Mitsunori Ogihara, Ryoko Hanada, Nobuhiro Furuyama

Research output: Contribution to journalArticle

2 Citations (Scopus)

Abstract

The automated annotation of conversational video by semantic miscommunication labels is a challenging topic. Although miscommunications are often obvious to the speakers as well as the observers, it is difficult for machines to detect them from the low-level features. We investigate the utility of gestural cues in this paper among various non-verbal features. Compared with gesture recognition tasks in human-computer interaction, this process is difficult due to the lack of understanding on which cues contribute to miscommunications and the implicitness of gestures. Nine simple gestural features are taken from gesture data, and both simple and complex classifiers are constructed using machine learning. The experimental results suggest that there is no single gestural feature that can predict or explain the occurrence of semantic miscommunication in our setting.

Original languageEnglish
Pages (from-to)7-20
Number of pages14
JournalMultimedia Tools and Applications
Volume61
Issue number1
DOIs
Publication statusPublished - 2012 Nov
Externally publishedYes

Fingerprint

Semantics
Gesture recognition
Human computer interaction
Learning systems
Labels
Classifiers

Keywords

  • Face-to-face
  • Gesture
  • Psychotherapy
  • Semantic indexing

ASJC Scopus subject areas

  • Media Technology
  • Hardware and Architecture
  • Computer Networks and Communications
  • Software

Cite this

Gestural cue analysis in automated semantic miscommunication annotation. / Inoue, Masashi; Ogihara, Mitsunori; Hanada, Ryoko; Furuyama, Nobuhiro.

In: Multimedia Tools and Applications, Vol. 61, No. 1, 11.2012, p. 7-20.

Research output: Contribution to journalArticle

Inoue, Masashi ; Ogihara, Mitsunori ; Hanada, Ryoko ; Furuyama, Nobuhiro. / Gestural cue analysis in automated semantic miscommunication annotation. In: Multimedia Tools and Applications. 2012 ; Vol. 61, No. 1. pp. 7-20.
@article{2788a89b30484545af9f1938f5e98bac,
title = "Gestural cue analysis in automated semantic miscommunication annotation",
abstract = "The automated annotation of conversational video by semantic miscommunication labels is a challenging topic. Although miscommunications are often obvious to the speakers as well as the observers, it is difficult for machines to detect them from the low-level features. We investigate the utility of gestural cues in this paper among various non-verbal features. Compared with gesture recognition tasks in human-computer interaction, this process is difficult due to the lack of understanding on which cues contribute to miscommunications and the implicitness of gestures. Nine simple gestural features are taken from gesture data, and both simple and complex classifiers are constructed using machine learning. The experimental results suggest that there is no single gestural feature that can predict or explain the occurrence of semantic miscommunication in our setting.",
keywords = "Face-to-face, Gesture, Psychotherapy, Semantic indexing",
author = "Masashi Inoue and Mitsunori Ogihara and Ryoko Hanada and Nobuhiro Furuyama",
year = "2012",
month = "11",
doi = "10.1007/s11042-010-0701-1",
language = "English",
volume = "61",
pages = "7--20",
journal = "Multimedia Tools and Applications",
issn = "1380-7501",
publisher = "Springer Netherlands",
number = "1",

}

TY - JOUR

T1 - Gestural cue analysis in automated semantic miscommunication annotation

AU - Inoue, Masashi

AU - Ogihara, Mitsunori

AU - Hanada, Ryoko

AU - Furuyama, Nobuhiro

PY - 2012/11

Y1 - 2012/11

N2 - The automated annotation of conversational video by semantic miscommunication labels is a challenging topic. Although miscommunications are often obvious to the speakers as well as the observers, it is difficult for machines to detect them from the low-level features. We investigate the utility of gestural cues in this paper among various non-verbal features. Compared with gesture recognition tasks in human-computer interaction, this process is difficult due to the lack of understanding on which cues contribute to miscommunications and the implicitness of gestures. Nine simple gestural features are taken from gesture data, and both simple and complex classifiers are constructed using machine learning. The experimental results suggest that there is no single gestural feature that can predict or explain the occurrence of semantic miscommunication in our setting.

AB - The automated annotation of conversational video by semantic miscommunication labels is a challenging topic. Although miscommunications are often obvious to the speakers as well as the observers, it is difficult for machines to detect them from the low-level features. We investigate the utility of gestural cues in this paper among various non-verbal features. Compared with gesture recognition tasks in human-computer interaction, this process is difficult due to the lack of understanding on which cues contribute to miscommunications and the implicitness of gestures. Nine simple gestural features are taken from gesture data, and both simple and complex classifiers are constructed using machine learning. The experimental results suggest that there is no single gestural feature that can predict or explain the occurrence of semantic miscommunication in our setting.

KW - Face-to-face

KW - Gesture

KW - Psychotherapy

KW - Semantic indexing

UR - http://www.scopus.com/inward/record.url?scp=84864036877&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84864036877&partnerID=8YFLogxK

U2 - 10.1007/s11042-010-0701-1

DO - 10.1007/s11042-010-0701-1

M3 - Article

VL - 61

SP - 7

EP - 20

JO - Multimedia Tools and Applications

JF - Multimedia Tools and Applications

SN - 1380-7501

IS - 1

ER -