Abstract
A multimodal interface provides multiple modalities for input and output, such as speech, eye gaze and facial expression. With the recent progresses in multimodal interfaces, various approaches about multimodal input fusion and output generation have been proposed. However, less attention has been paid to how to integrate them together in a multimodal input and output system. This paper proposes an approach, termed as THE HINGE, in providing agent-based multimodal presentations in accordance with multimodal input fusion results. The analysis of experiment result shows the proposed approach enhances the flexibility of the system while maintains its stability.
| Original language | English |
|---|---|
| Title of host publication | Conference on Human Factors in Computing Systems - Proceedings |
| Pages | 3483-3488 |
| Number of pages | 6 |
| DOIs | |
| Publication status | Published - 2008 |
| Externally published | Yes |
| Event | 28th Annual CHI Conference on Human Factors in Computing Systems - Florence Duration: 2008 Apr 5 → 2008 Apr 10 |
Other
| Other | 28th Annual CHI Conference on Human Factors in Computing Systems |
|---|---|
| City | Florence |
| Period | 08/4/5 → 08/4/10 |
Fingerprint
Keywords
- Discourse representation
- Input understanding
- Multimodal input fusion
- Multimodal interfaces
ASJC Scopus subject areas
- Human-Computer Interaction
- Computer Graphics and Computer-Aided Design
- Software
Cite this
THE HINGE between input and output : Understanding the multimodal input fusion results in an agent-based multimodal presentation system. / Sun, Yong; Prendinger, Helmut; Shi, Yu; Chen, Fang; Chung, Vera; Ishizuka, Mitsuru.
Conference on Human Factors in Computing Systems - Proceedings. 2008. p. 3483-3488.Research output: Chapter in Book/Report/Conference proceeding › Conference contribution
}
TY - GEN
T1 - THE HINGE between input and output
T2 - Understanding the multimodal input fusion results in an agent-based multimodal presentation system
AU - Sun, Yong
AU - Prendinger, Helmut
AU - Shi, Yu
AU - Chen, Fang
AU - Chung, Vera
AU - Ishizuka, Mitsuru
PY - 2008
Y1 - 2008
N2 - A multimodal interface provides multiple modalities for input and output, such as speech, eye gaze and facial expression. With the recent progresses in multimodal interfaces, various approaches about multimodal input fusion and output generation have been proposed. However, less attention has been paid to how to integrate them together in a multimodal input and output system. This paper proposes an approach, termed as THE HINGE, in providing agent-based multimodal presentations in accordance with multimodal input fusion results. The analysis of experiment result shows the proposed approach enhances the flexibility of the system while maintains its stability.
AB - A multimodal interface provides multiple modalities for input and output, such as speech, eye gaze and facial expression. With the recent progresses in multimodal interfaces, various approaches about multimodal input fusion and output generation have been proposed. However, less attention has been paid to how to integrate them together in a multimodal input and output system. This paper proposes an approach, termed as THE HINGE, in providing agent-based multimodal presentations in accordance with multimodal input fusion results. The analysis of experiment result shows the proposed approach enhances the flexibility of the system while maintains its stability.
KW - Discourse representation
KW - Input understanding
KW - Multimodal input fusion
KW - Multimodal interfaces
UR - http://www.scopus.com/inward/record.url?scp=57049088566&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=57049088566&partnerID=8YFLogxK
U2 - 10.1145/1358628.1358878
DO - 10.1145/1358628.1358878
M3 - Conference contribution
AN - SCOPUS:57049088566
SP - 3483
EP - 3488
BT - Conference on Human Factors in Computing Systems - Proceedings
ER -