Abstract
In multimedia data analysis, automated indexing of conversational video is an emerging topic. One challenging problem in this topic is the recognition of higher-level concepts, such as miscommunications in conversations. While detecting miscommunications is generally easy for speakers as well as observers, it is not currently understood which cues contribute to their detection and to what extent. To make use of the knowledge on gestural cues in multimedia systems, the applicability of machine learning is investigated as a means of detecting miscommunication from gestural patterns observed in psychotherapeutic face-to-face conversations. Various features are taken from gesture data, and both simple and complex classifiers are constructed using these features. Both short-term and long-term effects are tested using different time window sizes. Also, two types of gestures, communicative and non-communicative, are considered. The experimental results suggest that there is no single gestural feature that can explain the occurrence of semantic miscommunication. Another interesting finding is that gestural cues correlate more with long-term gestural patterns than with short-term ones.
| Original language | English |
|---|---|
| Title of host publication | 2010 5th International Conference on Future Information Technology, FutureTech 2010 - Proceedings |
| DOIs | |
| Publication status | Published - 2010 |
| Externally published | Yes |
| Event | 5th International Conference on Future Information Technology, FutureTech 2010 - Busan Duration: 2010 May 20 → 2010 May 24 |
Other
| Other | 5th International Conference on Future Information Technology, FutureTech 2010 |
|---|---|
| City | Busan |
| Period | 10/5/20 → 10/5/24 |
Fingerprint
Keywords
- Face-toface
- Gesture
- Psychotherapy
- Semantic indexing
ASJC Scopus subject areas
- Computer Networks and Communications
- Information Systems
Cite this
Utility of gestural cues in indexing semantic miscommunication. / Inoue, Masashi; Ogihara, Mitsunori; Hanada, Ryoko; Furuyama, Nobuhiro.
2010 5th International Conference on Future Information Technology, FutureTech 2010 - Proceedings. 2010. 5482653.Research output: Chapter in Book/Report/Conference proceeding › Conference contribution
}
TY - GEN
T1 - Utility of gestural cues in indexing semantic miscommunication
AU - Inoue, Masashi
AU - Ogihara, Mitsunori
AU - Hanada, Ryoko
AU - Furuyama, Nobuhiro
PY - 2010
Y1 - 2010
N2 - In multimedia data analysis, automated indexing of conversational video is an emerging topic. One challenging problem in this topic is the recognition of higher-level concepts, such as miscommunications in conversations. While detecting miscommunications is generally easy for speakers as well as observers, it is not currently understood which cues contribute to their detection and to what extent. To make use of the knowledge on gestural cues in multimedia systems, the applicability of machine learning is investigated as a means of detecting miscommunication from gestural patterns observed in psychotherapeutic face-to-face conversations. Various features are taken from gesture data, and both simple and complex classifiers are constructed using these features. Both short-term and long-term effects are tested using different time window sizes. Also, two types of gestures, communicative and non-communicative, are considered. The experimental results suggest that there is no single gestural feature that can explain the occurrence of semantic miscommunication. Another interesting finding is that gestural cues correlate more with long-term gestural patterns than with short-term ones.
AB - In multimedia data analysis, automated indexing of conversational video is an emerging topic. One challenging problem in this topic is the recognition of higher-level concepts, such as miscommunications in conversations. While detecting miscommunications is generally easy for speakers as well as observers, it is not currently understood which cues contribute to their detection and to what extent. To make use of the knowledge on gestural cues in multimedia systems, the applicability of machine learning is investigated as a means of detecting miscommunication from gestural patterns observed in psychotherapeutic face-to-face conversations. Various features are taken from gesture data, and both simple and complex classifiers are constructed using these features. Both short-term and long-term effects are tested using different time window sizes. Also, two types of gestures, communicative and non-communicative, are considered. The experimental results suggest that there is no single gestural feature that can explain the occurrence of semantic miscommunication. Another interesting finding is that gestural cues correlate more with long-term gestural patterns than with short-term ones.
KW - Face-toface
KW - Gesture
KW - Psychotherapy
KW - Semantic indexing
UR - http://www.scopus.com/inward/record.url?scp=77954400664&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=77954400664&partnerID=8YFLogxK
U2 - 10.1109/FUTURETECH.2010.5482653
DO - 10.1109/FUTURETECH.2010.5482653
M3 - Conference contribution
SN - 9781424469505
BT - 2010 5th International Conference on Future Information Technology, FutureTech 2010 - Proceedings
ER -