Real-time speaker localization and speech separation by audio-visual integration

Kazuhiro Nakadai; Ken Ichi Hidai; Hiroshi G. Okuno; Hiroaki Kitano

Real-time speaker localization and speech separation by audio-visual integration

Kazuhiro Nakadai^*, Ken Ichi Hidai, Hiroshi G. Okuno, Hiroaki Kitano

^*この研究の対応する著者

研究成果: Conference contribution

37 被引用数 (Scopus)

抄録

Robot audition in real-world should cope with motor and other noises caused by the robot's own movements in addition to environmental noises and reverberation. This paper reports how auditory processing is improved by audio-visual integration with active movements. The key idea resides in hierarchical integration of auditory and visual streams to disambiguate auditory or visual processing. The system runs in real-time by using distributed processing on 4 PCs connected by Gigabit Ethernet. The system implemented in a upper-torso humanoid tracks multiple talkers and extracts speech from a mixture of sounds. The performance of epipolar geometry based sound source localization and sound source separation by active and adaptive direction-pass filtering is also reported.

本文言語	English
ホスト出版物のタイトル	Proceedings - IEEE International Conference on Robotics and Automation
ページ	1043-1049
ページ数	7
巻	1
出版ステータス	Published - 2002
外部発表	はい
イベント	2002 IEEE International Conference on Robotics and Automation - Washington, DC, United States 継続期間: 2002 5月 11 → 2002 5月 15

Other

Other	2002 IEEE International Conference on Robotics and Automation
国/地域	United States
City	Washington, DC
Period	02/5/11 → 02/5/15

ASJC Scopus subject areas

ソフトウェア
制御およびシステム工学

他のファイルとリンク

引用スタイル

@inproceedings{c5c3d206bc574b5d9fe3fc0ad7fa1c37,

title = "Real-time speaker localization and speech separation by audio-visual integration",

abstract = "Robot audition in real-world should cope with motor and other noises caused by the robot's own movements in addition to environmental noises and reverberation. This paper reports how auditory processing is improved by audio-visual integration with active movements. The key idea resides in hierarchical integration of auditory and visual streams to disambiguate auditory or visual processing. The system runs in real-time by using distributed processing on 4 PCs connected by Gigabit Ethernet. The system implemented in a upper-torso humanoid tracks multiple talkers and extracts speech from a mixture of sounds. The performance of epipolar geometry based sound source localization and sound source separation by active and adaptive direction-pass filtering is also reported.",

keywords = "Audio-visual integration, Multiple speaker tracking, Robot audition, Sound source localization, Sound source separation",

author = "Kazuhiro Nakadai and Hidai, {Ken Ichi} and Okuno, {Hiroshi G.} and Hiroaki Kitano",

year = "2002",

language = "English",

volume = "1",

pages = "1043--1049",

booktitle = "Proceedings - IEEE International Conference on Robotics and Automation",

note = "2002 IEEE International Conference on Robotics and Automation ; Conference date: 11-05-2002 Through 15-05-2002",

}

TY - GEN

T1 - Real-time speaker localization and speech separation by audio-visual integration

AU - Nakadai, Kazuhiro

AU - Hidai, Ken Ichi

AU - Okuno, Hiroshi G.

AU - Kitano, Hiroaki

PY - 2002

Y1 - 2002

N2 - Robot audition in real-world should cope with motor and other noises caused by the robot's own movements in addition to environmental noises and reverberation. This paper reports how auditory processing is improved by audio-visual integration with active movements. The key idea resides in hierarchical integration of auditory and visual streams to disambiguate auditory or visual processing. The system runs in real-time by using distributed processing on 4 PCs connected by Gigabit Ethernet. The system implemented in a upper-torso humanoid tracks multiple talkers and extracts speech from a mixture of sounds. The performance of epipolar geometry based sound source localization and sound source separation by active and adaptive direction-pass filtering is also reported.

AB - Robot audition in real-world should cope with motor and other noises caused by the robot's own movements in addition to environmental noises and reverberation. This paper reports how auditory processing is improved by audio-visual integration with active movements. The key idea resides in hierarchical integration of auditory and visual streams to disambiguate auditory or visual processing. The system runs in real-time by using distributed processing on 4 PCs connected by Gigabit Ethernet. The system implemented in a upper-torso humanoid tracks multiple talkers and extracts speech from a mixture of sounds. The performance of epipolar geometry based sound source localization and sound source separation by active and adaptive direction-pass filtering is also reported.

KW - Audio-visual integration

KW - Multiple speaker tracking

KW - Robot audition

KW - Sound source localization

KW - Sound source separation

UR - http://www.scopus.com/inward/record.url?scp=0036058193&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=0036058193&partnerID=8YFLogxK

M3 - Conference contribution

AN - SCOPUS:0036058193

VL - 1

SP - 1043

EP - 1049

BT - Proceedings - IEEE International Conference on Robotics and Automation

T2 - 2002 IEEE International Conference on Robotics and Automation

Y2 - 11 May 2002 through 15 May 2002

ER -

Real-time speaker localization and speech separation by audio-visual integration

抄録

Other

ASJC Scopus subject areas

他のファイルとリンク

フィンガープリント

引用スタイル