THE HINGE between input and output: Understanding the multimodal input fusion results in an agent-based multimodal presentation system

Yong Sun, Helmut Prendinger, Yu Shi, Fang Chen, Vera Chung, Mitsuru Ishizuka

Research output: Chapter in Book/Report/Conference proceedingConference contribution

3 Citations (Scopus)

Abstract

A multimodal interface provides multiple modalities for input and output, such as speech, eye gaze and facial expression. With the recent progresses in multimodal interfaces, various approaches about multimodal input fusion and output generation have been proposed. However, less attention has been paid to how to integrate them together in a multimodal input and output system. This paper proposes an approach, termed as THE HINGE, in providing agent-based multimodal presentations in accordance with multimodal input fusion results. The analysis of experiment result shows the proposed approach enhances the flexibility of the system while maintains its stability.

Original languageEnglish
Title of host publicationConference on Human Factors in Computing Systems - Proceedings
Pages3483-3488
Number of pages6
DOIs
Publication statusPublished - 2008
Externally publishedYes
Event28th Annual CHI Conference on Human Factors in Computing Systems - Florence
Duration: 2008 Apr 52008 Apr 10

Other

Other28th Annual CHI Conference on Human Factors in Computing Systems
CityFlorence
Period08/4/508/4/10

Keywords

  • Discourse representation
  • Input understanding
  • Multimodal input fusion
  • Multimodal interfaces

ASJC Scopus subject areas

  • Human-Computer Interaction
  • Computer Graphics and Computer-Aided Design
  • Software

Fingerprint Dive into the research topics of 'THE HINGE between input and output: Understanding the multimodal input fusion results in an agent-based multimodal presentation system'. Together they form a unique fingerprint.

  • Cite this

    Sun, Y., Prendinger, H., Shi, Y., Chen, F., Chung, V., & Ishizuka, M. (2008). THE HINGE between input and output: Understanding the multimodal input fusion results in an agent-based multimodal presentation system. In Conference on Human Factors in Computing Systems - Proceedings (pp. 3483-3488) https://doi.org/10.1145/1358628.1358878