Natural human-robot musical interaction: Understanding the music conductor gestures by using the WB-4 inertial measurement system

Sarah Cosentino; Klaus Petersen; Zhuohua Lin; Luca Bartolomeo; Salvatore Sessa; Massimiliano Zecca; Atsuo Takanishi

doi:10.1080/01691864.2014.889577

Natural human-robot musical interaction: Understanding the music conductor gestures by using the WB-4 inertial measurement system

Sarah Cosentino^*, Klaus Petersen, Zhuohua Lin, Luca Bartolomeo, Salvatore Sessa, Massimiliano Zecca, Atsuo Takanishi

^*Corresponding author for this work

Research output: Contribution to journal › Article › peer-review

13 Citations (Scopus)

Abstract

This paper presents an inertial measurement unit-based human gesture recognition system for a robot instrument player to understand the instructions dictated by an orchestra conductor and accordingly adapt its musical performance. It is an extension of our previous publications on natural human-robot musical interaction. With this system, the robot can understand the real-time variations in musical parameters dictated by the conductor’s movements, adding expression to its performance while being synchronized with all the other human partner musicians. The enhanced interaction ability would obviously lead to an improvement of the overall live performance, but also allow the partner musicians, as well as the conductor, to better appreciate a joint musical performance, thanks to the complete naturalness of the interaction.

Original language	English
Pages (from-to)	781-792
Number of pages	12
Journal	Advanced Robotics
Volume	28
Issue number	11
DOIs	https://doi.org/10.1080/01691864.2014.889577
Publication status	Published - 2014 Feb 25

Keywords

Human-robot interaction
Humanoid robot
Inertial measurement unit
Musical robot

ASJC Scopus subject areas

Control and Systems Engineering
Software
Human-Computer Interaction
Hardware and Architecture
Computer Science Applications

Access to Document

10.1080/01691864.2014.889577

Cite this

@article{7ba7bc1c4268439da6299250eadd113d,

title = "Natural human-robot musical interaction: Understanding the music conductor gestures by using the WB-4 inertial measurement system",

abstract = "This paper presents an inertial measurement unit-based human gesture recognition system for a robot instrument player to understand the instructions dictated by an orchestra conductor and accordingly adapt its musical performance. It is an extension of our previous publications on natural human-robot musical interaction. With this system, the robot can understand the real-time variations in musical parameters dictated by the conductor{\textquoteright}s movements, adding expression to its performance while being synchronized with all the other human partner musicians. The enhanced interaction ability would obviously lead to an improvement of the overall live performance, but also allow the partner musicians, as well as the conductor, to better appreciate a joint musical performance, thanks to the complete naturalness of the interaction.",

keywords = "Human-robot interaction, Humanoid robot, Inertial measurement unit, Musical robot",

author = "Sarah Cosentino and Klaus Petersen and Zhuohua Lin and Luca Bartolomeo and Salvatore Sessa and Massimiliano Zecca and Atsuo Takanishi",

note = "Funding Information: This research was partially supported by a grant by STMicroelec-tronics, which also provided the core sensors and the microcontroller. This work was also supported in part by: the Global COE Program {\textquoteleft}Global Robot Academia,{\textquoteright} MEXT, Japan; and the Consolidated Research Institute for Advanced Science and Medical Care, Waseda University (ASMeW). Publisher Copyright: {\textcopyright} 2014 Taylor & Francis and The Robotics Society of Japan.",

year = "2014",

month = feb,

day = "25",

doi = "10.1080/01691864.2014.889577",

language = "English",

volume = "28",

pages = "781--792",

journal = "Advanced Robotics",

issn = "0169-1864",

publisher = "Taylor and Francis Ltd.",

number = "11",

}

TY - JOUR

T1 - Natural human-robot musical interaction

T2 - Understanding the music conductor gestures by using the WB-4 inertial measurement system

AU - Cosentino, Sarah

AU - Petersen, Klaus

AU - Lin, Zhuohua

AU - Bartolomeo, Luca

AU - Sessa, Salvatore

AU - Zecca, Massimiliano

AU - Takanishi, Atsuo

N1 - Funding Information: This research was partially supported by a grant by STMicroelec-tronics, which also provided the core sensors and the microcontroller. This work was also supported in part by: the Global COE Program ‘Global Robot Academia,’ MEXT, Japan; and the Consolidated Research Institute for Advanced Science and Medical Care, Waseda University (ASMeW). Publisher Copyright: © 2014 Taylor & Francis and The Robotics Society of Japan.

PY - 2014/2/25

Y1 - 2014/2/25

N2 - This paper presents an inertial measurement unit-based human gesture recognition system for a robot instrument player to understand the instructions dictated by an orchestra conductor and accordingly adapt its musical performance. It is an extension of our previous publications on natural human-robot musical interaction. With this system, the robot can understand the real-time variations in musical parameters dictated by the conductor’s movements, adding expression to its performance while being synchronized with all the other human partner musicians. The enhanced interaction ability would obviously lead to an improvement of the overall live performance, but also allow the partner musicians, as well as the conductor, to better appreciate a joint musical performance, thanks to the complete naturalness of the interaction.

AB - This paper presents an inertial measurement unit-based human gesture recognition system for a robot instrument player to understand the instructions dictated by an orchestra conductor and accordingly adapt its musical performance. It is an extension of our previous publications on natural human-robot musical interaction. With this system, the robot can understand the real-time variations in musical parameters dictated by the conductor’s movements, adding expression to its performance while being synchronized with all the other human partner musicians. The enhanced interaction ability would obviously lead to an improvement of the overall live performance, but also allow the partner musicians, as well as the conductor, to better appreciate a joint musical performance, thanks to the complete naturalness of the interaction.

KW - Human-robot interaction

KW - Humanoid robot

KW - Inertial measurement unit

KW - Musical robot

UR - http://www.scopus.com/inward/record.url?scp=84894283675&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84894283675&partnerID=8YFLogxK

U2 - 10.1080/01691864.2014.889577

DO - 10.1080/01691864.2014.889577

M3 - Article

AN - SCOPUS:84894283675

VL - 28

SP - 781

EP - 792

JO - Advanced Robotics

JF - Advanced Robotics

SN - 0169-1864

IS - 11

ER -

Natural human-robot musical interaction: Understanding the music conductor gestures by using the WB-4 inertial measurement system

Abstract

Keywords

ASJC Scopus subject areas

Access to Document

Other files and links

Fingerprint

Cite this