Sound and visual tracking for humanoid robot

Hiroshi G. Okuno, Kazuhiro Nakadai, Tino Lourens, Hiroaki Kitano

Research output: Contribution to journalArticle

14 Citations (Scopus)

Abstract

Mobile robots capable of auditory perception usually adopt the "stop-perceive-act" principle to avoid sounds made during moving due to motor noise. Although this principle reduces the complexity of the problems involved in auditory processing for mobile robots, it restricts their capabilities of auditory processing. In this paper, sound and visual tracking are investigated to compensate each other's drawbacks in tracking objects and to attain robust object tracking. Visual tracking may be difficult in case of occlusion, while sound tracking may be ambiguous in localization due to the nature of auditory processing. For this purpose, we present an active audition system for humanoid robot. The audition system of the highly intelligent humanoid requires localization of sound sources and identification of meanings of the sound in the auditory scene. The active audition reported in this paper focuses on improved sound source tracking by integrating audition, vision, and motor control. Given the multiple sound sources in the auditory scene, SIG the humanoid actively moves its head to improve localization by aligning microphones orthogonal to the sound source and by capturing the possible sound sources by vision. The system adaptively cancels motor noises using motor control signals. The experimental result demonstrates the effectiveness of sound and visual tracking.

Original languageEnglish
Pages (from-to)253-266
Number of pages14
JournalApplied Intelligence
Volume20
Issue number3
DOIs
Publication statusPublished - 2004 May
Externally publishedYes

Fingerprint

Acoustic waves
Robots
Audition
Mobile robots
Processing
Microphones
Acoustic noise

Keywords

  • Audio-visual integration
  • Audio-visual tracking
  • Computational auditory scene analysis
  • Robot audition

ASJC Scopus subject areas

  • Control and Systems Engineering
  • Artificial Intelligence

Cite this

Sound and visual tracking for humanoid robot. / Okuno, Hiroshi G.; Nakadai, Kazuhiro; Lourens, Tino; Kitano, Hiroaki.

In: Applied Intelligence, Vol. 20, No. 3, 05.2004, p. 253-266.

Research output: Contribution to journalArticle

Okuno, Hiroshi G. ; Nakadai, Kazuhiro ; Lourens, Tino ; Kitano, Hiroaki. / Sound and visual tracking for humanoid robot. In: Applied Intelligence. 2004 ; Vol. 20, No. 3. pp. 253-266.
@article{ba1378be75f241b58fde669c9cc9fd57,
title = "Sound and visual tracking for humanoid robot",
abstract = "Mobile robots capable of auditory perception usually adopt the {"}stop-perceive-act{"} principle to avoid sounds made during moving due to motor noise. Although this principle reduces the complexity of the problems involved in auditory processing for mobile robots, it restricts their capabilities of auditory processing. In this paper, sound and visual tracking are investigated to compensate each other's drawbacks in tracking objects and to attain robust object tracking. Visual tracking may be difficult in case of occlusion, while sound tracking may be ambiguous in localization due to the nature of auditory processing. For this purpose, we present an active audition system for humanoid robot. The audition system of the highly intelligent humanoid requires localization of sound sources and identification of meanings of the sound in the auditory scene. The active audition reported in this paper focuses on improved sound source tracking by integrating audition, vision, and motor control. Given the multiple sound sources in the auditory scene, SIG the humanoid actively moves its head to improve localization by aligning microphones orthogonal to the sound source and by capturing the possible sound sources by vision. The system adaptively cancels motor noises using motor control signals. The experimental result demonstrates the effectiveness of sound and visual tracking.",
keywords = "Audio-visual integration, Audio-visual tracking, Computational auditory scene analysis, Robot audition",
author = "Okuno, {Hiroshi G.} and Kazuhiro Nakadai and Tino Lourens and Hiroaki Kitano",
year = "2004",
month = "5",
doi = "10.1023/B:APIN.0000021417.62541.e0",
language = "English",
volume = "20",
pages = "253--266",
journal = "Applied Intelligence",
issn = "0924-669X",
publisher = "Springer Netherlands",
number = "3",

}

TY - JOUR

T1 - Sound and visual tracking for humanoid robot

AU - Okuno, Hiroshi G.

AU - Nakadai, Kazuhiro

AU - Lourens, Tino

AU - Kitano, Hiroaki

PY - 2004/5

Y1 - 2004/5

N2 - Mobile robots capable of auditory perception usually adopt the "stop-perceive-act" principle to avoid sounds made during moving due to motor noise. Although this principle reduces the complexity of the problems involved in auditory processing for mobile robots, it restricts their capabilities of auditory processing. In this paper, sound and visual tracking are investigated to compensate each other's drawbacks in tracking objects and to attain robust object tracking. Visual tracking may be difficult in case of occlusion, while sound tracking may be ambiguous in localization due to the nature of auditory processing. For this purpose, we present an active audition system for humanoid robot. The audition system of the highly intelligent humanoid requires localization of sound sources and identification of meanings of the sound in the auditory scene. The active audition reported in this paper focuses on improved sound source tracking by integrating audition, vision, and motor control. Given the multiple sound sources in the auditory scene, SIG the humanoid actively moves its head to improve localization by aligning microphones orthogonal to the sound source and by capturing the possible sound sources by vision. The system adaptively cancels motor noises using motor control signals. The experimental result demonstrates the effectiveness of sound and visual tracking.

AB - Mobile robots capable of auditory perception usually adopt the "stop-perceive-act" principle to avoid sounds made during moving due to motor noise. Although this principle reduces the complexity of the problems involved in auditory processing for mobile robots, it restricts their capabilities of auditory processing. In this paper, sound and visual tracking are investigated to compensate each other's drawbacks in tracking objects and to attain robust object tracking. Visual tracking may be difficult in case of occlusion, while sound tracking may be ambiguous in localization due to the nature of auditory processing. For this purpose, we present an active audition system for humanoid robot. The audition system of the highly intelligent humanoid requires localization of sound sources and identification of meanings of the sound in the auditory scene. The active audition reported in this paper focuses on improved sound source tracking by integrating audition, vision, and motor control. Given the multiple sound sources in the auditory scene, SIG the humanoid actively moves its head to improve localization by aligning microphones orthogonal to the sound source and by capturing the possible sound sources by vision. The system adaptively cancels motor noises using motor control signals. The experimental result demonstrates the effectiveness of sound and visual tracking.

KW - Audio-visual integration

KW - Audio-visual tracking

KW - Computational auditory scene analysis

KW - Robot audition

UR - http://www.scopus.com/inward/record.url?scp=3543111972&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=3543111972&partnerID=8YFLogxK

U2 - 10.1023/B:APIN.0000021417.62541.e0

DO - 10.1023/B:APIN.0000021417.62541.e0

M3 - Article

VL - 20

SP - 253

EP - 266

JO - Applied Intelligence

JF - Applied Intelligence

SN - 0924-669X

IS - 3

ER -