THE HINGE between input and output: Understanding the multimodal input fusion results in an agent-based multimodal presentation system

Yong Sun, Helmut Prendinger, Yu Shi, Fang Chen, Vera Chung, Mitsuru Ishizuka

Research output: Chapter in Book/Report/Conference proceedingConference contribution

3 Citations (Scopus)

Abstract

A multimodal interface provides multiple modalities for input and output, such as speech, eye gaze and facial expression. With the recent progresses in multimodal interfaces, various approaches about multimodal input fusion and output generation have been proposed. However, less attention has been paid to how to integrate them together in a multimodal input and output system. This paper proposes an approach, termed as THE HINGE, in providing agent-based multimodal presentations in accordance with multimodal input fusion results. The analysis of experiment result shows the proposed approach enhances the flexibility of the system while maintains its stability.

Original languageEnglish
Title of host publicationConference on Human Factors in Computing Systems - Proceedings
Pages3483-3488
Number of pages6
DOIs
Publication statusPublished - 2008
Externally publishedYes
Event28th Annual CHI Conference on Human Factors in Computing Systems - Florence
Duration: 2008 Apr 52008 Apr 10

Other

Other28th Annual CHI Conference on Human Factors in Computing Systems
CityFlorence
Period08/4/508/4/10

Fingerprint

Fusion reactions
Experiments

Keywords

  • Discourse representation
  • Input understanding
  • Multimodal input fusion
  • Multimodal interfaces

ASJC Scopus subject areas

  • Human-Computer Interaction
  • Computer Graphics and Computer-Aided Design
  • Software

Cite this

Sun, Y., Prendinger, H., Shi, Y., Chen, F., Chung, V., & Ishizuka, M. (2008). THE HINGE between input and output: Understanding the multimodal input fusion results in an agent-based multimodal presentation system. In Conference on Human Factors in Computing Systems - Proceedings (pp. 3483-3488) https://doi.org/10.1145/1358628.1358878

THE HINGE between input and output : Understanding the multimodal input fusion results in an agent-based multimodal presentation system. / Sun, Yong; Prendinger, Helmut; Shi, Yu; Chen, Fang; Chung, Vera; Ishizuka, Mitsuru.

Conference on Human Factors in Computing Systems - Proceedings. 2008. p. 3483-3488.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Sun, Y, Prendinger, H, Shi, Y, Chen, F, Chung, V & Ishizuka, M 2008, THE HINGE between input and output: Understanding the multimodal input fusion results in an agent-based multimodal presentation system. in Conference on Human Factors in Computing Systems - Proceedings. pp. 3483-3488, 28th Annual CHI Conference on Human Factors in Computing Systems, Florence, 08/4/5. https://doi.org/10.1145/1358628.1358878
Sun Y, Prendinger H, Shi Y, Chen F, Chung V, Ishizuka M. THE HINGE between input and output: Understanding the multimodal input fusion results in an agent-based multimodal presentation system. In Conference on Human Factors in Computing Systems - Proceedings. 2008. p. 3483-3488 https://doi.org/10.1145/1358628.1358878
Sun, Yong ; Prendinger, Helmut ; Shi, Yu ; Chen, Fang ; Chung, Vera ; Ishizuka, Mitsuru. / THE HINGE between input and output : Understanding the multimodal input fusion results in an agent-based multimodal presentation system. Conference on Human Factors in Computing Systems - Proceedings. 2008. pp. 3483-3488
@inproceedings{e4f0632c52e440bb8c1a45a87404c75d,
title = "THE HINGE between input and output: Understanding the multimodal input fusion results in an agent-based multimodal presentation system",
abstract = "A multimodal interface provides multiple modalities for input and output, such as speech, eye gaze and facial expression. With the recent progresses in multimodal interfaces, various approaches about multimodal input fusion and output generation have been proposed. However, less attention has been paid to how to integrate them together in a multimodal input and output system. This paper proposes an approach, termed as THE HINGE, in providing agent-based multimodal presentations in accordance with multimodal input fusion results. The analysis of experiment result shows the proposed approach enhances the flexibility of the system while maintains its stability.",
keywords = "Discourse representation, Input understanding, Multimodal input fusion, Multimodal interfaces",
author = "Yong Sun and Helmut Prendinger and Yu Shi and Fang Chen and Vera Chung and Mitsuru Ishizuka",
year = "2008",
doi = "10.1145/1358628.1358878",
language = "English",
pages = "3483--3488",
booktitle = "Conference on Human Factors in Computing Systems - Proceedings",

}

TY - GEN

T1 - THE HINGE between input and output

T2 - Understanding the multimodal input fusion results in an agent-based multimodal presentation system

AU - Sun, Yong

AU - Prendinger, Helmut

AU - Shi, Yu

AU - Chen, Fang

AU - Chung, Vera

AU - Ishizuka, Mitsuru

PY - 2008

Y1 - 2008

N2 - A multimodal interface provides multiple modalities for input and output, such as speech, eye gaze and facial expression. With the recent progresses in multimodal interfaces, various approaches about multimodal input fusion and output generation have been proposed. However, less attention has been paid to how to integrate them together in a multimodal input and output system. This paper proposes an approach, termed as THE HINGE, in providing agent-based multimodal presentations in accordance with multimodal input fusion results. The analysis of experiment result shows the proposed approach enhances the flexibility of the system while maintains its stability.

AB - A multimodal interface provides multiple modalities for input and output, such as speech, eye gaze and facial expression. With the recent progresses in multimodal interfaces, various approaches about multimodal input fusion and output generation have been proposed. However, less attention has been paid to how to integrate them together in a multimodal input and output system. This paper proposes an approach, termed as THE HINGE, in providing agent-based multimodal presentations in accordance with multimodal input fusion results. The analysis of experiment result shows the proposed approach enhances the flexibility of the system while maintains its stability.

KW - Discourse representation

KW - Input understanding

KW - Multimodal input fusion

KW - Multimodal interfaces

UR - http://www.scopus.com/inward/record.url?scp=57049088566&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=57049088566&partnerID=8YFLogxK

U2 - 10.1145/1358628.1358878

DO - 10.1145/1358628.1358878

M3 - Conference contribution

AN - SCOPUS:57049088566

SP - 3483

EP - 3488

BT - Conference on Human Factors in Computing Systems - Proceedings

ER -