Using vision to improve sound source separation

Yukiko Nakagawa, Hiroshi G. Okuno, Hiroaki Kitano

研究成果: Chapter

15 被引用数 (Scopus)

抄録

We present a method of improving sound source separation using vision. The sound source separation is an essential function to accomplish auditory scene understanding by separating stream of sounds generated from multiple sound sources. By separating a stream of sounds, recognition process, such as speech recognition, can simply work on a single stream, not mixed sound of several speakers. The performance is known to be improved by using stereo/binaural microphone and microphone array which provides spatial information for separation. However, these methods still have more than 20 degree of positional ambiguities. In this paper, we further added visual information to provide more specific and accurate position information. As a result, separation capability was drastically improved. In addition, we found that the use of approximate direction information drastically improve object tracking accuracy of a simple vision system, which in turn improves performance of the auditory system. We claim that the integration of vision and auditory inputs improves performance of tasks in each perception, such as sound source separation and object tracking, by bootstrapping.

本文言語English
ホスト出版物のタイトルProceedings of the National Conference on Artificial Intelligence
Place of PublicationMenlo Park, CA, United States
出版社AAAI
ページ768-775
ページ数8
ISBN(印刷版)0262511061
出版ステータスPublished - 1999
外部発表はい
イベントProceedings of the 1999 16th National Conference on Artificial Intelligence (AAAI-99), 11th Innovative Applications of Artificial Intelligence Conference (IAAI-99) - Orlando, FL, USA
継続期間: 1999 7 181999 7 22

Other

OtherProceedings of the 1999 16th National Conference on Artificial Intelligence (AAAI-99), 11th Innovative Applications of Artificial Intelligence Conference (IAAI-99)
CityOrlando, FL, USA
Period99/7/1899/7/22

ASJC Scopus subject areas

  • Software

フィンガープリント 「Using vision to improve sound source separation」の研究トピックを掘り下げます。これらがまとまってユニークなフィンガープリントを構成します。

引用スタイル