Real-time robot audition system that recognizes simultaneous speech in the real world

Shun'ichi Yamamoto, Kazuhiro Nakadai, Mikio Nakano, Hiroshi Tsujino, Jean Marc Valin, Kazunori Komatani, Tetsuya Ogata, Hiroshi G. Okuno

Research output: Chapter in Book/Report/Conference proceedingConference contribution

51 Citations (Scopus)

Abstract

This paper presents a robot audition system that recognizes simultaneous speech in the real world by using robot-embedded microphones. We have previously reported Missing Feature Theory (MFT) based integration of Sound Source Separation (SSS) and Automatic Speech Recognition (ASR) for building robust robot audition. We demonstrated that a MFT-based prototype system drastically improved the performance of speech recognition even when three speakers talked to a robot simultaneously. However, the prototype system had three problems; being offline, hand-tuning of system parameters, and failure in Voice Activity Detection (VAD). To attain online processing, we introduced FlowDesigner-based architecture to integrate sound source localization (SSL), SSS and ASR. This architecture brings fast processing and easy implementation because it provides a simple framework of shared-object-based integration. To optimize the parameters, we developed Genetic Algorithm (GA) based parameter optimization, because it is difficult to build an analytical optimization model for mutually dependent system parameters. To improve VAD, we integrated new VAD based on a power spectrum and location of a sound source into the system, since conventional VAD relying only on power often fails due to low signal-to-noise ratio of simultaneous speech. We, then, constructed a robot audition system for Honda ASIMO. As a result, we showed that the system worked online and fast, and had a better performance in robustness and accuracy through experiments on recognition of simultaneous speech in a noisy and echoic environment.

Original languageEnglish
Title of host publication2006 IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS 2006
Pages5333-5338
Number of pages6
DOIs
Publication statusPublished - 2006
Event2006 IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS 2006 - Beijing, China
Duration: 2006 Oct 92006 Oct 15

Publication series

NameIEEE International Conference on Intelligent Robots and Systems

Conference

Conference2006 IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS 2006
CountryChina
CityBeijing
Period06/10/906/10/15

Keywords

  • Genetic algorithm
  • Missing feature theory
  • Parameter optimization
  • Real-time processing
  • Robot audition
  • Voice activity detection

ASJC Scopus subject areas

  • Control and Systems Engineering
  • Software
  • Computer Vision and Pattern Recognition
  • Computer Science Applications

Fingerprint Dive into the research topics of 'Real-time robot audition system that recognizes simultaneous speech in the real world'. Together they form a unique fingerprint.

  • Cite this

    Yamamoto, S., Nakadai, K., Nakano, M., Tsujino, H., Valin, J. M., Komatani, K., Ogata, T., & Okuno, H. G. (2006). Real-time robot audition system that recognizes simultaneous speech in the real world. In 2006 IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS 2006 (pp. 5333-5338). [4059274] (IEEE International Conference on Intelligent Robots and Systems). https://doi.org/10.1109/IROS.2006.282037