Auditory and visual integration based localization and tracking of humans in daily-life environments

Hyun Don Kim, Kazunori Komatani, Tetsuya Ogata, Hiroshi G. Okuno

Research output: Chapter in Book/Report/Conference proceedingConference contribution

16 Citations (Scopus)

Abstract

The purpose of this research is to develop techniques that enable robots to choose and track a desired person for interaction in daily-life environments. Therefore, localizing multiple moving sounds and human faces is necessary so that robots can locate a desired person. For sound source localization, we used a cross-power spectrum phase analysis (CSP) method and showed that CSP can localize sound sources only using two microphones and does not need impulse response data. An expectation-maximization (EM) algorithm was shown to enable a robot to cope with multiple moving sound sources. For face localization, we developed a method that can reliably detect several faces using the skin color classification obtained by using the EM algorithm. To deal with a change in color state according to illumination condition and various skin colors, the robot can obtain new skin color features of faces detected by OpenCV, an open vision library, for detecting human faces. Finally, we developed a probability based method to integrate auditory and visual information and to produce a reliable tracking path in real time. Furthermore, the developed system chose and tracked people while dealing with various background noises that are considered loud, even in the daily-life environments.

Original languageEnglish
Title of host publicationProceedings of the 2007 IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS 2007
Pages2021-2027
Number of pages7
DOIs
Publication statusPublished - 2007 Dec 1
Event2007 IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS 2007 - San Diego, CA, United States
Duration: 2007 Oct 292007 Nov 2

Publication series

NameIEEE International Conference on Intelligent Robots and Systems

Conference

Conference2007 IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS 2007
CountryUnited States
CitySan Diego, CA
Period07/10/2907/11/2

ASJC Scopus subject areas

  • Control and Systems Engineering
  • Software
  • Computer Vision and Pattern Recognition
  • Computer Science Applications

Fingerprint Dive into the research topics of 'Auditory and visual integration based localization and tracking of humans in daily-life environments'. Together they form a unique fingerprint.

  • Cite this

    Kim, H. D., Komatani, K., Ogata, T., & Okuno, H. G. (2007). Auditory and visual integration based localization and tracking of humans in daily-life environments. In Proceedings of the 2007 IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS 2007 (pp. 2021-2027). [4399331] (IEEE International Conference on Intelligent Robots and Systems). https://doi.org/10.1109/IROS.2007.4399331