A robot singer with music recognition based on real-time beat tracking

Kazumasa Murata, Kazuhiro Nakadai, Kazuyoshi Yoshii, Ryu Takeda, Toyotaka Torii, Hiroshi G. Okuno, Yuji Hasegawa, Hiroshi Tsujino

Research output: Chapter in Book/Report/Conference proceedingConference contribution

8 Citations (Scopus)

Abstract

A robot that can provide an active and enjoyable user interface is one of the most challenging applications for music information processing, because the robot should cope with high-power noises including self voices and motor noises. This paper proposes noise-robust musical beat tracking by using a robot-embedded microphone, and describes its application to a robot singer with music recognition. The proposed beat tracking introduces two key techniques, that is, spectro-temporal pattern matching and echo cancellation. The former realizes robust tempo estimation with a shorter window length, thus, it can quickly adapt to tempo changes. The latter is effective to cancel self periodic noises such as stepping, scatting, and singing. We constructed a robot singer based on the proposed beat tracking for Honda ASIMO. The robot detects a musical beat with its own microphone in a noisy environment. It tries to recognize music based on the detected musical beat. When it successfully recognizes music, it sings while stepping according to the beat. Otherwise, it performs scatting instead of singing because the lyrics are unavailable. Experimental results showed fast adaptation to tempo changes and high robustness in beat tracking even when stepping, scatting and singing.

Original languageEnglish
Title of host publicationISMIR 2008 - 9th International Conference on Music Information Retrieval
Pages199-204
Number of pages6
Publication statusPublished - 2008
Externally publishedYes
Event9th International Conference on Music Information Retrieval, ISMIR 2008 - Philadelphia, PA
Duration: 2008 Sep 142008 Sep 18

Other

Other9th International Conference on Music Information Retrieval, ISMIR 2008
CityPhiladelphia, PA
Period08/9/1408/9/18

Fingerprint

Robots
Microphones
Echo suppression
Pattern matching
User interfaces
Singers
Robot
Music

ASJC Scopus subject areas

  • Music
  • Information Systems

Cite this

Murata, K., Nakadai, K., Yoshii, K., Takeda, R., Torii, T., Okuno, H. G., ... Tsujino, H. (2008). A robot singer with music recognition based on real-time beat tracking. In ISMIR 2008 - 9th International Conference on Music Information Retrieval (pp. 199-204)

A robot singer with music recognition based on real-time beat tracking. / Murata, Kazumasa; Nakadai, Kazuhiro; Yoshii, Kazuyoshi; Takeda, Ryu; Torii, Toyotaka; Okuno, Hiroshi G.; Hasegawa, Yuji; Tsujino, Hiroshi.

ISMIR 2008 - 9th International Conference on Music Information Retrieval. 2008. p. 199-204.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Murata, K, Nakadai, K, Yoshii, K, Takeda, R, Torii, T, Okuno, HG, Hasegawa, Y & Tsujino, H 2008, A robot singer with music recognition based on real-time beat tracking. in ISMIR 2008 - 9th International Conference on Music Information Retrieval. pp. 199-204, 9th International Conference on Music Information Retrieval, ISMIR 2008, Philadelphia, PA, 08/9/14.
Murata K, Nakadai K, Yoshii K, Takeda R, Torii T, Okuno HG et al. A robot singer with music recognition based on real-time beat tracking. In ISMIR 2008 - 9th International Conference on Music Information Retrieval. 2008. p. 199-204
Murata, Kazumasa ; Nakadai, Kazuhiro ; Yoshii, Kazuyoshi ; Takeda, Ryu ; Torii, Toyotaka ; Okuno, Hiroshi G. ; Hasegawa, Yuji ; Tsujino, Hiroshi. / A robot singer with music recognition based on real-time beat tracking. ISMIR 2008 - 9th International Conference on Music Information Retrieval. 2008. pp. 199-204
@inproceedings{6b85fd0ddba04d40bb839c85f1debf38,
title = "A robot singer with music recognition based on real-time beat tracking",
abstract = "A robot that can provide an active and enjoyable user interface is one of the most challenging applications for music information processing, because the robot should cope with high-power noises including self voices and motor noises. This paper proposes noise-robust musical beat tracking by using a robot-embedded microphone, and describes its application to a robot singer with music recognition. The proposed beat tracking introduces two key techniques, that is, spectro-temporal pattern matching and echo cancellation. The former realizes robust tempo estimation with a shorter window length, thus, it can quickly adapt to tempo changes. The latter is effective to cancel self periodic noises such as stepping, scatting, and singing. We constructed a robot singer based on the proposed beat tracking for Honda ASIMO. The robot detects a musical beat with its own microphone in a noisy environment. It tries to recognize music based on the detected musical beat. When it successfully recognizes music, it sings while stepping according to the beat. Otherwise, it performs scatting instead of singing because the lyrics are unavailable. Experimental results showed fast adaptation to tempo changes and high robustness in beat tracking even when stepping, scatting and singing.",
author = "Kazumasa Murata and Kazuhiro Nakadai and Kazuyoshi Yoshii and Ryu Takeda and Toyotaka Torii and Okuno, {Hiroshi G.} and Yuji Hasegawa and Hiroshi Tsujino",
year = "2008",
language = "English",
isbn = "9780615248493",
pages = "199--204",
booktitle = "ISMIR 2008 - 9th International Conference on Music Information Retrieval",

}

TY - GEN

T1 - A robot singer with music recognition based on real-time beat tracking

AU - Murata, Kazumasa

AU - Nakadai, Kazuhiro

AU - Yoshii, Kazuyoshi

AU - Takeda, Ryu

AU - Torii, Toyotaka

AU - Okuno, Hiroshi G.

AU - Hasegawa, Yuji

AU - Tsujino, Hiroshi

PY - 2008

Y1 - 2008

N2 - A robot that can provide an active and enjoyable user interface is one of the most challenging applications for music information processing, because the robot should cope with high-power noises including self voices and motor noises. This paper proposes noise-robust musical beat tracking by using a robot-embedded microphone, and describes its application to a robot singer with music recognition. The proposed beat tracking introduces two key techniques, that is, spectro-temporal pattern matching and echo cancellation. The former realizes robust tempo estimation with a shorter window length, thus, it can quickly adapt to tempo changes. The latter is effective to cancel self periodic noises such as stepping, scatting, and singing. We constructed a robot singer based on the proposed beat tracking for Honda ASIMO. The robot detects a musical beat with its own microphone in a noisy environment. It tries to recognize music based on the detected musical beat. When it successfully recognizes music, it sings while stepping according to the beat. Otherwise, it performs scatting instead of singing because the lyrics are unavailable. Experimental results showed fast adaptation to tempo changes and high robustness in beat tracking even when stepping, scatting and singing.

AB - A robot that can provide an active and enjoyable user interface is one of the most challenging applications for music information processing, because the robot should cope with high-power noises including self voices and motor noises. This paper proposes noise-robust musical beat tracking by using a robot-embedded microphone, and describes its application to a robot singer with music recognition. The proposed beat tracking introduces two key techniques, that is, spectro-temporal pattern matching and echo cancellation. The former realizes robust tempo estimation with a shorter window length, thus, it can quickly adapt to tempo changes. The latter is effective to cancel self periodic noises such as stepping, scatting, and singing. We constructed a robot singer based on the proposed beat tracking for Honda ASIMO. The robot detects a musical beat with its own microphone in a noisy environment. It tries to recognize music based on the detected musical beat. When it successfully recognizes music, it sings while stepping according to the beat. Otherwise, it performs scatting instead of singing because the lyrics are unavailable. Experimental results showed fast adaptation to tempo changes and high robustness in beat tracking even when stepping, scatting and singing.

UR - http://www.scopus.com/inward/record.url?scp=84455176629&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84455176629&partnerID=8YFLogxK

M3 - Conference contribution

AN - SCOPUS:84455176629

SN - 9780615248493

SP - 199

EP - 204

BT - ISMIR 2008 - 9th International Conference on Music Information Retrieval

ER -