Multi-modal integration for personalized conversation

Towards a humanoid in daily life

Shinya Fujie, Daichi Watanabe, Yuhi Ichikawa, Hikaru Taniyama, Kosuke Hosoya, Yoichi Matsuyama, Tetsunori Kobayashi

Research output: Chapter in Book/Report/Conference proceedingConference contribution

3 Citations (Scopus)

Abstract

Humanoid with spoken language communication ability is proposed and developed. To make humanoid live with people, spoken language communication is fundamental because we use this kind of communication everyday. However, due to difficulties of speech recognition itself and implementation on the robot, a robot with such an ability has not been developed. In this study, we propose a robot with the technique implemented to overcome these problems. This proposed system includes three key features, image processing, sound source separation, and turn-taking timing control. Processing image captured with camera mounted on the robot's eyes enables to find and identify whom the robot should talked to. Sound source separation enables distant speech recognition, so that people need no special device, such as head-set microphones. Turn-taking timing control is often lacked in many conventional spoken dialogue system, but this is fundamental because the conversation proceeds in realtime. The effectiveness of these elements as well as the example of conversation are shown in experiments.

Original languageEnglish
Title of host publication2008 8th IEEE-RAS International Conference on Humanoid Robots, Humanoids 2008
Pages617-622
Number of pages6
DOIs
Publication statusPublished - 2008
Event2008 8th IEEE-RAS International Conference on Humanoid Robots, Humanoids 2008 - Daejeon
Duration: 2008 Dec 12008 Dec 3

Other

Other2008 8th IEEE-RAS International Conference on Humanoid Robots, Humanoids 2008
CityDaejeon
Period08/12/108/12/3

Fingerprint

Robots
Source separation
Speech recognition
Communication
Image processing
Acoustic waves
Microphones
Cameras
Experiments

ASJC Scopus subject areas

  • Artificial Intelligence
  • Computer Vision and Pattern Recognition
  • Human-Computer Interaction

Cite this

Fujie, S., Watanabe, D., Ichikawa, Y., Taniyama, H., Hosoya, K., Matsuyama, Y., & Kobayashi, T. (2008). Multi-modal integration for personalized conversation: Towards a humanoid in daily life. In 2008 8th IEEE-RAS International Conference on Humanoid Robots, Humanoids 2008 (pp. 617-622). [4756014] https://doi.org/10.1109/ICHR.2008.4756014

Multi-modal integration for personalized conversation : Towards a humanoid in daily life. / Fujie, Shinya; Watanabe, Daichi; Ichikawa, Yuhi; Taniyama, Hikaru; Hosoya, Kosuke; Matsuyama, Yoichi; Kobayashi, Tetsunori.

2008 8th IEEE-RAS International Conference on Humanoid Robots, Humanoids 2008. 2008. p. 617-622 4756014.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Fujie, S, Watanabe, D, Ichikawa, Y, Taniyama, H, Hosoya, K, Matsuyama, Y & Kobayashi, T 2008, Multi-modal integration for personalized conversation: Towards a humanoid in daily life. in 2008 8th IEEE-RAS International Conference on Humanoid Robots, Humanoids 2008., 4756014, pp. 617-622, 2008 8th IEEE-RAS International Conference on Humanoid Robots, Humanoids 2008, Daejeon, 08/12/1. https://doi.org/10.1109/ICHR.2008.4756014
Fujie S, Watanabe D, Ichikawa Y, Taniyama H, Hosoya K, Matsuyama Y et al. Multi-modal integration for personalized conversation: Towards a humanoid in daily life. In 2008 8th IEEE-RAS International Conference on Humanoid Robots, Humanoids 2008. 2008. p. 617-622. 4756014 https://doi.org/10.1109/ICHR.2008.4756014
Fujie, Shinya ; Watanabe, Daichi ; Ichikawa, Yuhi ; Taniyama, Hikaru ; Hosoya, Kosuke ; Matsuyama, Yoichi ; Kobayashi, Tetsunori. / Multi-modal integration for personalized conversation : Towards a humanoid in daily life. 2008 8th IEEE-RAS International Conference on Humanoid Robots, Humanoids 2008. 2008. pp. 617-622
@inproceedings{a8ff80de142b4d939e4e3904154f903c,
title = "Multi-modal integration for personalized conversation: Towards a humanoid in daily life",
abstract = "Humanoid with spoken language communication ability is proposed and developed. To make humanoid live with people, spoken language communication is fundamental because we use this kind of communication everyday. However, due to difficulties of speech recognition itself and implementation on the robot, a robot with such an ability has not been developed. In this study, we propose a robot with the technique implemented to overcome these problems. This proposed system includes three key features, image processing, sound source separation, and turn-taking timing control. Processing image captured with camera mounted on the robot's eyes enables to find and identify whom the robot should talked to. Sound source separation enables distant speech recognition, so that people need no special device, such as head-set microphones. Turn-taking timing control is often lacked in many conventional spoken dialogue system, but this is fundamental because the conversation proceeds in realtime. The effectiveness of these elements as well as the example of conversation are shown in experiments.",
author = "Shinya Fujie and Daichi Watanabe and Yuhi Ichikawa and Hikaru Taniyama and Kosuke Hosoya and Yoichi Matsuyama and Tetsunori Kobayashi",
year = "2008",
doi = "10.1109/ICHR.2008.4756014",
language = "English",
isbn = "9781424428229",
pages = "617--622",
booktitle = "2008 8th IEEE-RAS International Conference on Humanoid Robots, Humanoids 2008",

}

TY - GEN

T1 - Multi-modal integration for personalized conversation

T2 - Towards a humanoid in daily life

AU - Fujie, Shinya

AU - Watanabe, Daichi

AU - Ichikawa, Yuhi

AU - Taniyama, Hikaru

AU - Hosoya, Kosuke

AU - Matsuyama, Yoichi

AU - Kobayashi, Tetsunori

PY - 2008

Y1 - 2008

N2 - Humanoid with spoken language communication ability is proposed and developed. To make humanoid live with people, spoken language communication is fundamental because we use this kind of communication everyday. However, due to difficulties of speech recognition itself and implementation on the robot, a robot with such an ability has not been developed. In this study, we propose a robot with the technique implemented to overcome these problems. This proposed system includes three key features, image processing, sound source separation, and turn-taking timing control. Processing image captured with camera mounted on the robot's eyes enables to find and identify whom the robot should talked to. Sound source separation enables distant speech recognition, so that people need no special device, such as head-set microphones. Turn-taking timing control is often lacked in many conventional spoken dialogue system, but this is fundamental because the conversation proceeds in realtime. The effectiveness of these elements as well as the example of conversation are shown in experiments.

AB - Humanoid with spoken language communication ability is proposed and developed. To make humanoid live with people, spoken language communication is fundamental because we use this kind of communication everyday. However, due to difficulties of speech recognition itself and implementation on the robot, a robot with such an ability has not been developed. In this study, we propose a robot with the technique implemented to overcome these problems. This proposed system includes three key features, image processing, sound source separation, and turn-taking timing control. Processing image captured with camera mounted on the robot's eyes enables to find and identify whom the robot should talked to. Sound source separation enables distant speech recognition, so that people need no special device, such as head-set microphones. Turn-taking timing control is often lacked in many conventional spoken dialogue system, but this is fundamental because the conversation proceeds in realtime. The effectiveness of these elements as well as the example of conversation are shown in experiments.

UR - http://www.scopus.com/inward/record.url?scp=63549150003&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=63549150003&partnerID=8YFLogxK

U2 - 10.1109/ICHR.2008.4756014

DO - 10.1109/ICHR.2008.4756014

M3 - Conference contribution

SN - 9781424428229

SP - 617

EP - 622

BT - 2008 8th IEEE-RAS International Conference on Humanoid Robots, Humanoids 2008

ER -