Particle-filter based audio-visual beat-tracking for music robot ensemble with human guitarist

Tatsuhiko Itohara, Takuma Otsuka, Takeshi Mizumoto, Tetsuya Ogata, Hiroshi G. Okuno

研究成果: Conference contribution

10 被引用数 (Scopus)

抄録

This paper presents an audio-visual beat-tracking method for ensemble robots with a human guitarist. Beat-tracking, or estimation of tempo and beat times of music, is critical to the high quality of musical ensemble performance. Since a human plays the guitar in out-beat in back beat and syncopation, the main problems of beat-tracking of a human's guitar playing are twofold: tempo changes and varying note lengths. Most conventional methods have not addressed human's guitar playing. Therefore, they lack the adaptation of either of the problems. To solve the problems simultaneously, our method uses not only audio but visual features. We extract audio features with Spectro-Temporal Pattern Matching (STPM) and visual features with optical flow, mean shift and Hough transform. Our beat-tracking estimates tempo and beat time using a particle filter; both acoustic feature of guitar sounds and visual features of arm motions are represented as particles. The particle is determined based on prior distribution of audio and visual features, respectively Experimental results confirm that our integrated audio-visual approach is robust against tempo changes and varying note lengths. In addition, they also show that estimation convergence rate depends only a little on the number of particles. The real-time factor is 0.88 when the number of particles is 200, and this shows out method works in real-time.

本文言語English
ホスト出版物のタイトルIROS'11 - 2011 IEEE/RSJ International Conference on Intelligent Robots and Systems
ホスト出版物のサブタイトルCelebrating 50 Years of Robotics
ページ118-124
ページ数7
DOI
出版ステータスPublished - 2011 12 29
外部発表はい
イベント2011 IEEE/RSJ International Conference on Intelligent Robots and Systems: Celebrating 50 Years of Robotics, IROS'11 - San Francisco, CA, United States
継続期間: 2011 9 252011 9 30

出版物シリーズ

名前IEEE International Conference on Intelligent Robots and Systems

Conference

Conference2011 IEEE/RSJ International Conference on Intelligent Robots and Systems: Celebrating 50 Years of Robotics, IROS'11
CountryUnited States
CitySan Francisco, CA
Period11/9/2511/9/30

ASJC Scopus subject areas

  • Control and Systems Engineering
  • Software
  • Computer Vision and Pattern Recognition
  • Computer Science Applications

フィンガープリント 「Particle-filter based audio-visual beat-tracking for music robot ensemble with human guitarist」の研究トピックを掘り下げます。これらがまとまってユニークなフィンガープリントを構成します。

引用スタイル