A robot listens to music and counts its beats aloud by separating music from counting voice

Takeshi Mizumoto, Ryu Takeda, Kazuyoshi Yoshii, Kazunori Komatani, Tetsuya Ogata, Hiroshi G. Okuno

Research output: Chapter in Book/Report/Conference proceedingConference contribution

10 Citations (Scopus)

Abstract

This paper presents a beat-counting robot that can count musical beats aloud, i.e., speak "one, two, three, four, one, two, ..." along music, while listening to music by using its own ears. Music-understanding robots that interact with humans should be able not only to recognize music internally, but also to express their own internal states. To develop our beat-counting robot, we have tackled three issues: (1) recognition of hierarchical beat structures, (2) expression of these structures by counting beats, and (3) suppression of counting voice (self-generated sound) in sound mixtures recorded by ears. The main issue is (3) because the interference of counting voice in music causes the decrease of the beat recognition accuracy. So we designed the architecture for music-understanding robot that is capable of dealing with the issue of self-generated sounds. To solve these issues, we took the following approaches: (1) beat structure prediction based on musical knowledge on chords and drums, (2) speed control of counting voice according to music tempo via a vocoder called STRAIGHT, and (3) semi-blind separation of sound mixtures into music and counting voice via an adaptive filter based on ICA (Independent Component Analysis) that uses the waveform of the counting voice as a prior knowledge. Experimental result showed that suppressing robot's own voice improved music recognition capability.

Original languageEnglish
Title of host publication2008 IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS
Pages1538-1543
Number of pages6
DOIs
Publication statusPublished - 2008
Externally publishedYes
Event2008 IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS - Nice
Duration: 2008 Sep 222008 Sep 26

Other

Other2008 IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS
CityNice
Period08/9/2208/9/26

Fingerprint

Robots
Acoustic waves
Independent component analysis
Adaptive filters
Speed control

ASJC Scopus subject areas

  • Artificial Intelligence
  • Computer Vision and Pattern Recognition
  • Control and Systems Engineering
  • Electrical and Electronic Engineering

Cite this

Mizumoto, T., Takeda, R., Yoshii, K., Komatani, K., Ogata, T., & Okuno, H. G. (2008). A robot listens to music and counts its beats aloud by separating music from counting voice. In 2008 IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS (pp. 1538-1543). [4650821] https://doi.org/10.1109/IROS.2008.4650821

A robot listens to music and counts its beats aloud by separating music from counting voice. / Mizumoto, Takeshi; Takeda, Ryu; Yoshii, Kazuyoshi; Komatani, Kazunori; Ogata, Tetsuya; Okuno, Hiroshi G.

2008 IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS. 2008. p. 1538-1543 4650821.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Mizumoto, T, Takeda, R, Yoshii, K, Komatani, K, Ogata, T & Okuno, HG 2008, A robot listens to music and counts its beats aloud by separating music from counting voice. in 2008 IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS., 4650821, pp. 1538-1543, 2008 IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS, Nice, 08/9/22. https://doi.org/10.1109/IROS.2008.4650821
Mizumoto T, Takeda R, Yoshii K, Komatani K, Ogata T, Okuno HG. A robot listens to music and counts its beats aloud by separating music from counting voice. In 2008 IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS. 2008. p. 1538-1543. 4650821 https://doi.org/10.1109/IROS.2008.4650821
Mizumoto, Takeshi ; Takeda, Ryu ; Yoshii, Kazuyoshi ; Komatani, Kazunori ; Ogata, Tetsuya ; Okuno, Hiroshi G. / A robot listens to music and counts its beats aloud by separating music from counting voice. 2008 IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS. 2008. pp. 1538-1543
@inproceedings{34f0ef439a6240fa8a8ad5ac8f3bb394,
title = "A robot listens to music and counts its beats aloud by separating music from counting voice",
abstract = "This paper presents a beat-counting robot that can count musical beats aloud, i.e., speak {"}one, two, three, four, one, two, ...{"} along music, while listening to music by using its own ears. Music-understanding robots that interact with humans should be able not only to recognize music internally, but also to express their own internal states. To develop our beat-counting robot, we have tackled three issues: (1) recognition of hierarchical beat structures, (2) expression of these structures by counting beats, and (3) suppression of counting voice (self-generated sound) in sound mixtures recorded by ears. The main issue is (3) because the interference of counting voice in music causes the decrease of the beat recognition accuracy. So we designed the architecture for music-understanding robot that is capable of dealing with the issue of self-generated sounds. To solve these issues, we took the following approaches: (1) beat structure prediction based on musical knowledge on chords and drums, (2) speed control of counting voice according to music tempo via a vocoder called STRAIGHT, and (3) semi-blind separation of sound mixtures into music and counting voice via an adaptive filter based on ICA (Independent Component Analysis) that uses the waveform of the counting voice as a prior knowledge. Experimental result showed that suppressing robot's own voice improved music recognition capability.",
author = "Takeshi Mizumoto and Ryu Takeda and Kazuyoshi Yoshii and Kazunori Komatani and Tetsuya Ogata and Okuno, {Hiroshi G.}",
year = "2008",
doi = "10.1109/IROS.2008.4650821",
language = "English",
isbn = "9781424420582",
pages = "1538--1543",
booktitle = "2008 IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS",

}

TY - GEN

T1 - A robot listens to music and counts its beats aloud by separating music from counting voice

AU - Mizumoto, Takeshi

AU - Takeda, Ryu

AU - Yoshii, Kazuyoshi

AU - Komatani, Kazunori

AU - Ogata, Tetsuya

AU - Okuno, Hiroshi G.

PY - 2008

Y1 - 2008

N2 - This paper presents a beat-counting robot that can count musical beats aloud, i.e., speak "one, two, three, four, one, two, ..." along music, while listening to music by using its own ears. Music-understanding robots that interact with humans should be able not only to recognize music internally, but also to express their own internal states. To develop our beat-counting robot, we have tackled three issues: (1) recognition of hierarchical beat structures, (2) expression of these structures by counting beats, and (3) suppression of counting voice (self-generated sound) in sound mixtures recorded by ears. The main issue is (3) because the interference of counting voice in music causes the decrease of the beat recognition accuracy. So we designed the architecture for music-understanding robot that is capable of dealing with the issue of self-generated sounds. To solve these issues, we took the following approaches: (1) beat structure prediction based on musical knowledge on chords and drums, (2) speed control of counting voice according to music tempo via a vocoder called STRAIGHT, and (3) semi-blind separation of sound mixtures into music and counting voice via an adaptive filter based on ICA (Independent Component Analysis) that uses the waveform of the counting voice as a prior knowledge. Experimental result showed that suppressing robot's own voice improved music recognition capability.

AB - This paper presents a beat-counting robot that can count musical beats aloud, i.e., speak "one, two, three, four, one, two, ..." along music, while listening to music by using its own ears. Music-understanding robots that interact with humans should be able not only to recognize music internally, but also to express their own internal states. To develop our beat-counting robot, we have tackled three issues: (1) recognition of hierarchical beat structures, (2) expression of these structures by counting beats, and (3) suppression of counting voice (self-generated sound) in sound mixtures recorded by ears. The main issue is (3) because the interference of counting voice in music causes the decrease of the beat recognition accuracy. So we designed the architecture for music-understanding robot that is capable of dealing with the issue of self-generated sounds. To solve these issues, we took the following approaches: (1) beat structure prediction based on musical knowledge on chords and drums, (2) speed control of counting voice according to music tempo via a vocoder called STRAIGHT, and (3) semi-blind separation of sound mixtures into music and counting voice via an adaptive filter based on ICA (Independent Component Analysis) that uses the waveform of the counting voice as a prior knowledge. Experimental result showed that suppressing robot's own voice improved music recognition capability.

UR - http://www.scopus.com/inward/record.url?scp=69549128353&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=69549128353&partnerID=8YFLogxK

U2 - 10.1109/IROS.2008.4650821

DO - 10.1109/IROS.2008.4650821

M3 - Conference contribution

AN - SCOPUS:69549128353

SN - 9781424420582

SP - 1538

EP - 1543

BT - 2008 IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS

ER -