Audio-based automatic generation of a piano reduction score by considering the musical structure

Hirofumi Takamori, Takayuki Nakatsuka, Satoru Fukayama, Masataka Goto, Shigeo Morishima

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

This study describes a method that automatically generates a piano reduction score from the audio recordings of popular music while considering the musical structure. The generated score comprises both right- and left-hand piano parts, which reflect the melodies, chords, and rhythms extracted from the original audio signals. Generating such a reduction score from an audio recording is challenging because automatic music transcription is still considered to be inefficient when the input contains sounds from various instruments. Reflecting the long-term correlation structure behind similar repetitive bars is also challenging; further, previous methods have independently generated each bar. Our approach addresses the aforementioned issues by integrating musical analysis, especially structural analysis, with music generation. Our method extracts rhythmic features as well as melodies and chords from the input audio recording and reflects them in the score. To consider the long-term correlation between bars, we use similarity matrices, created for several acoustical features, as constraints. We further conduct a multivariate regression analysis to determine the acoustical features that represent the most valuable constraints for generating a musical structure. We have generated piano scores using our method and have observed that we can produce scores that differently balance between the ability to achieve rhythmic characteristics and the ability to obtain musical structures.

Original languageEnglish
Title of host publicationMultiMedia Modeling - 25th International Conference, MMM 2019, Proceedings
EditorsBenoit Huet, Ioannis Kompatsiaris, Stefanos Vrochidis, Vasileios Mezaris, Wen-Huang Cheng, Cathal Gurrin
PublisherSpringer-Verlag
Pages169-181
Number of pages13
ISBN (Print)9783030057152
DOIs
Publication statusPublished - 2019 Jan 1
Event25th International Conference on MultiMedia Modeling, MMM 2019 - Thessaloniki, Greece
Duration: 2019 Jan 82019 Jan 11

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume11296 LNCS
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Other

Other25th International Conference on MultiMedia Modeling, MMM 2019
CountryGreece
CityThessaloniki
Period19/1/819/1/11

Fingerprint

Audio recordings
Music
Transcription
Chord or secant line
Structural analysis
Regression analysis
Acoustic waves
Term Structure
Multivariate Regression
Multivariate Analysis
Correlation Structure
Structural Analysis
Regression Analysis

Keywords

  • Acoustic feature
  • Multivariate regression analysis
  • Musical structure
  • Piano reduction
  • Self-similarity matrix

ASJC Scopus subject areas

  • Theoretical Computer Science
  • Computer Science(all)

Cite this

Takamori, H., Nakatsuka, T., Fukayama, S., Goto, M., & Morishima, S. (2019). Audio-based automatic generation of a piano reduction score by considering the musical structure. In B. Huet, I. Kompatsiaris, S. Vrochidis, V. Mezaris, W-H. Cheng, & C. Gurrin (Eds.), MultiMedia Modeling - 25th International Conference, MMM 2019, Proceedings (pp. 169-181). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 11296 LNCS). Springer-Verlag. https://doi.org/10.1007/978-3-030-05716-9_14

Audio-based automatic generation of a piano reduction score by considering the musical structure. / Takamori, Hirofumi; Nakatsuka, Takayuki; Fukayama, Satoru; Goto, Masataka; Morishima, Shigeo.

MultiMedia Modeling - 25th International Conference, MMM 2019, Proceedings. ed. / Benoit Huet; Ioannis Kompatsiaris; Stefanos Vrochidis; Vasileios Mezaris; Wen-Huang Cheng; Cathal Gurrin. Springer-Verlag, 2019. p. 169-181 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 11296 LNCS).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Takamori, H, Nakatsuka, T, Fukayama, S, Goto, M & Morishima, S 2019, Audio-based automatic generation of a piano reduction score by considering the musical structure. in B Huet, I Kompatsiaris, S Vrochidis, V Mezaris, W-H Cheng & C Gurrin (eds), MultiMedia Modeling - 25th International Conference, MMM 2019, Proceedings. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 11296 LNCS, Springer-Verlag, pp. 169-181, 25th International Conference on MultiMedia Modeling, MMM 2019, Thessaloniki, Greece, 19/1/8. https://doi.org/10.1007/978-3-030-05716-9_14
Takamori H, Nakatsuka T, Fukayama S, Goto M, Morishima S. Audio-based automatic generation of a piano reduction score by considering the musical structure. In Huet B, Kompatsiaris I, Vrochidis S, Mezaris V, Cheng W-H, Gurrin C, editors, MultiMedia Modeling - 25th International Conference, MMM 2019, Proceedings. Springer-Verlag. 2019. p. 169-181. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). https://doi.org/10.1007/978-3-030-05716-9_14
Takamori, Hirofumi ; Nakatsuka, Takayuki ; Fukayama, Satoru ; Goto, Masataka ; Morishima, Shigeo. / Audio-based automatic generation of a piano reduction score by considering the musical structure. MultiMedia Modeling - 25th International Conference, MMM 2019, Proceedings. editor / Benoit Huet ; Ioannis Kompatsiaris ; Stefanos Vrochidis ; Vasileios Mezaris ; Wen-Huang Cheng ; Cathal Gurrin. Springer-Verlag, 2019. pp. 169-181 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).
@inproceedings{25faaf1a6b8d4f609007c4b6e20f73e9,
title = "Audio-based automatic generation of a piano reduction score by considering the musical structure",
abstract = "This study describes a method that automatically generates a piano reduction score from the audio recordings of popular music while considering the musical structure. The generated score comprises both right- and left-hand piano parts, which reflect the melodies, chords, and rhythms extracted from the original audio signals. Generating such a reduction score from an audio recording is challenging because automatic music transcription is still considered to be inefficient when the input contains sounds from various instruments. Reflecting the long-term correlation structure behind similar repetitive bars is also challenging; further, previous methods have independently generated each bar. Our approach addresses the aforementioned issues by integrating musical analysis, especially structural analysis, with music generation. Our method extracts rhythmic features as well as melodies and chords from the input audio recording and reflects them in the score. To consider the long-term correlation between bars, we use similarity matrices, created for several acoustical features, as constraints. We further conduct a multivariate regression analysis to determine the acoustical features that represent the most valuable constraints for generating a musical structure. We have generated piano scores using our method and have observed that we can produce scores that differently balance between the ability to achieve rhythmic characteristics and the ability to obtain musical structures.",
keywords = "Acoustic feature, Multivariate regression analysis, Musical structure, Piano reduction, Self-similarity matrix",
author = "Hirofumi Takamori and Takayuki Nakatsuka and Satoru Fukayama and Masataka Goto and Shigeo Morishima",
year = "2019",
month = "1",
day = "1",
doi = "10.1007/978-3-030-05716-9_14",
language = "English",
isbn = "9783030057152",
series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",
publisher = "Springer-Verlag",
pages = "169--181",
editor = "Benoit Huet and Ioannis Kompatsiaris and Stefanos Vrochidis and Vasileios Mezaris and Wen-Huang Cheng and Cathal Gurrin",
booktitle = "MultiMedia Modeling - 25th International Conference, MMM 2019, Proceedings",

}

TY - GEN

T1 - Audio-based automatic generation of a piano reduction score by considering the musical structure

AU - Takamori, Hirofumi

AU - Nakatsuka, Takayuki

AU - Fukayama, Satoru

AU - Goto, Masataka

AU - Morishima, Shigeo

PY - 2019/1/1

Y1 - 2019/1/1

N2 - This study describes a method that automatically generates a piano reduction score from the audio recordings of popular music while considering the musical structure. The generated score comprises both right- and left-hand piano parts, which reflect the melodies, chords, and rhythms extracted from the original audio signals. Generating such a reduction score from an audio recording is challenging because automatic music transcription is still considered to be inefficient when the input contains sounds from various instruments. Reflecting the long-term correlation structure behind similar repetitive bars is also challenging; further, previous methods have independently generated each bar. Our approach addresses the aforementioned issues by integrating musical analysis, especially structural analysis, with music generation. Our method extracts rhythmic features as well as melodies and chords from the input audio recording and reflects them in the score. To consider the long-term correlation between bars, we use similarity matrices, created for several acoustical features, as constraints. We further conduct a multivariate regression analysis to determine the acoustical features that represent the most valuable constraints for generating a musical structure. We have generated piano scores using our method and have observed that we can produce scores that differently balance between the ability to achieve rhythmic characteristics and the ability to obtain musical structures.

AB - This study describes a method that automatically generates a piano reduction score from the audio recordings of popular music while considering the musical structure. The generated score comprises both right- and left-hand piano parts, which reflect the melodies, chords, and rhythms extracted from the original audio signals. Generating such a reduction score from an audio recording is challenging because automatic music transcription is still considered to be inefficient when the input contains sounds from various instruments. Reflecting the long-term correlation structure behind similar repetitive bars is also challenging; further, previous methods have independently generated each bar. Our approach addresses the aforementioned issues by integrating musical analysis, especially structural analysis, with music generation. Our method extracts rhythmic features as well as melodies and chords from the input audio recording and reflects them in the score. To consider the long-term correlation between bars, we use similarity matrices, created for several acoustical features, as constraints. We further conduct a multivariate regression analysis to determine the acoustical features that represent the most valuable constraints for generating a musical structure. We have generated piano scores using our method and have observed that we can produce scores that differently balance between the ability to achieve rhythmic characteristics and the ability to obtain musical structures.

KW - Acoustic feature

KW - Multivariate regression analysis

KW - Musical structure

KW - Piano reduction

KW - Self-similarity matrix

UR - http://www.scopus.com/inward/record.url?scp=85059836205&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85059836205&partnerID=8YFLogxK

U2 - 10.1007/978-3-030-05716-9_14

DO - 10.1007/978-3-030-05716-9_14

M3 - Conference contribution

AN - SCOPUS:85059836205

SN - 9783030057152

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 169

EP - 181

BT - MultiMedia Modeling - 25th International Conference, MMM 2019, Proceedings

A2 - Huet, Benoit

A2 - Kompatsiaris, Ioannis

A2 - Vrochidis, Stefanos

A2 - Mezaris, Vasileios

A2 - Cheng, Wen-Huang

A2 - Gurrin, Cathal

PB - Springer-Verlag

ER -