Audio-based automatic generation of a piano reduction score by considering the musical structure

Hirofumi Takamori, Takayuki Nakatsuka, Satoru Fukayama, Masataka Goto, Shigeo Morishima

研究成果: Conference contribution

抄録

This study describes a method that automatically generates a piano reduction score from the audio recordings of popular music while considering the musical structure. The generated score comprises both right- and left-hand piano parts, which reflect the melodies, chords, and rhythms extracted from the original audio signals. Generating such a reduction score from an audio recording is challenging because automatic music transcription is still considered to be inefficient when the input contains sounds from various instruments. Reflecting the long-term correlation structure behind similar repetitive bars is also challenging; further, previous methods have independently generated each bar. Our approach addresses the aforementioned issues by integrating musical analysis, especially structural analysis, with music generation. Our method extracts rhythmic features as well as melodies and chords from the input audio recording and reflects them in the score. To consider the long-term correlation between bars, we use similarity matrices, created for several acoustical features, as constraints. We further conduct a multivariate regression analysis to determine the acoustical features that represent the most valuable constraints for generating a musical structure. We have generated piano scores using our method and have observed that we can produce scores that differently balance between the ability to achieve rhythmic characteristics and the ability to obtain musical structures.

元の言語English
ホスト出版物のタイトルMultiMedia Modeling - 25th International Conference, MMM 2019, Proceedings
編集者Benoit Huet, Ioannis Kompatsiaris, Stefanos Vrochidis, Vasileios Mezaris, Wen-Huang Cheng, Cathal Gurrin
出版者Springer-Verlag
ページ169-181
ページ数13
ISBN(印刷物)9783030057152
DOI
出版物ステータスPublished - 2019 1 1
外部発表Yes
イベント25th International Conference on MultiMedia Modeling, MMM 2019 - Thessaloniki, Greece
継続期間: 2019 1 82019 1 11

出版物シリーズ

名前Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
11296 LNCS
ISSN(印刷物)0302-9743
ISSN(電子版)1611-3349

Other

Other25th International Conference on MultiMedia Modeling, MMM 2019
Greece
Thessaloniki
期間19/1/819/1/11

Fingerprint

Audio recordings
Music
Transcription
Chord or secant line
Structural analysis
Regression analysis
Acoustic waves
Term Structure
Multivariate Regression
Multivariate Analysis
Correlation Structure
Structural Analysis
Regression Analysis

Keywords

    ASJC Scopus subject areas

    • Theoretical Computer Science
    • Computer Science(all)

    これを引用

    Takamori, H., Nakatsuka, T., Fukayama, S., Goto, M., & Morishima, S. (2019). Audio-based automatic generation of a piano reduction score by considering the musical structure. : B. Huet, I. Kompatsiaris, S. Vrochidis, V. Mezaris, W-H. Cheng, & C. Gurrin (版), MultiMedia Modeling - 25th International Conference, MMM 2019, Proceedings (pp. 169-181). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); 巻数 11296 LNCS). Springer-Verlag. https://doi.org/10.1007/978-3-030-05716-9_14

    Audio-based automatic generation of a piano reduction score by considering the musical structure. / Takamori, Hirofumi; Nakatsuka, Takayuki; Fukayama, Satoru; Goto, Masataka; Morishima, Shigeo.

    MultiMedia Modeling - 25th International Conference, MMM 2019, Proceedings. 版 / Benoit Huet; Ioannis Kompatsiaris; Stefanos Vrochidis; Vasileios Mezaris; Wen-Huang Cheng; Cathal Gurrin. Springer-Verlag, 2019. p. 169-181 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); 巻 11296 LNCS).

    研究成果: Conference contribution

    Takamori, H, Nakatsuka, T, Fukayama, S, Goto, M & Morishima, S 2019, Audio-based automatic generation of a piano reduction score by considering the musical structure. : B Huet, I Kompatsiaris, S Vrochidis, V Mezaris, W-H Cheng & C Gurrin (版), MultiMedia Modeling - 25th International Conference, MMM 2019, Proceedings. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), 巻. 11296 LNCS, Springer-Verlag, pp. 169-181, 25th International Conference on MultiMedia Modeling, MMM 2019, Thessaloniki, Greece, 19/1/8. https://doi.org/10.1007/978-3-030-05716-9_14
    Takamori H, Nakatsuka T, Fukayama S, Goto M, Morishima S. Audio-based automatic generation of a piano reduction score by considering the musical structure. : Huet B, Kompatsiaris I, Vrochidis S, Mezaris V, Cheng W-H, Gurrin C, 編集者, MultiMedia Modeling - 25th International Conference, MMM 2019, Proceedings. Springer-Verlag. 2019. p. 169-181. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). https://doi.org/10.1007/978-3-030-05716-9_14
    Takamori, Hirofumi ; Nakatsuka, Takayuki ; Fukayama, Satoru ; Goto, Masataka ; Morishima, Shigeo. / Audio-based automatic generation of a piano reduction score by considering the musical structure. MultiMedia Modeling - 25th International Conference, MMM 2019, Proceedings. 編集者 / Benoit Huet ; Ioannis Kompatsiaris ; Stefanos Vrochidis ; Vasileios Mezaris ; Wen-Huang Cheng ; Cathal Gurrin. Springer-Verlag, 2019. pp. 169-181 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).
    @inproceedings{25faaf1a6b8d4f609007c4b6e20f73e9,
    title = "Audio-based automatic generation of a piano reduction score by considering the musical structure",
    abstract = "This study describes a method that automatically generates a piano reduction score from the audio recordings of popular music while considering the musical structure. The generated score comprises both right- and left-hand piano parts, which reflect the melodies, chords, and rhythms extracted from the original audio signals. Generating such a reduction score from an audio recording is challenging because automatic music transcription is still considered to be inefficient when the input contains sounds from various instruments. Reflecting the long-term correlation structure behind similar repetitive bars is also challenging; further, previous methods have independently generated each bar. Our approach addresses the aforementioned issues by integrating musical analysis, especially structural analysis, with music generation. Our method extracts rhythmic features as well as melodies and chords from the input audio recording and reflects them in the score. To consider the long-term correlation between bars, we use similarity matrices, created for several acoustical features, as constraints. We further conduct a multivariate regression analysis to determine the acoustical features that represent the most valuable constraints for generating a musical structure. We have generated piano scores using our method and have observed that we can produce scores that differently balance between the ability to achieve rhythmic characteristics and the ability to obtain musical structures.",
    keywords = "Acoustic feature, Multivariate regression analysis, Musical structure, Piano reduction, Self-similarity matrix",
    author = "Hirofumi Takamori and Takayuki Nakatsuka and Satoru Fukayama and Masataka Goto and Shigeo Morishima",
    year = "2019",
    month = "1",
    day = "1",
    doi = "10.1007/978-3-030-05716-9_14",
    language = "English",
    isbn = "9783030057152",
    series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",
    publisher = "Springer-Verlag",
    pages = "169--181",
    editor = "Benoit Huet and Ioannis Kompatsiaris and Stefanos Vrochidis and Vasileios Mezaris and Wen-Huang Cheng and Cathal Gurrin",
    booktitle = "MultiMedia Modeling - 25th International Conference, MMM 2019, Proceedings",

    }

    TY - GEN

    T1 - Audio-based automatic generation of a piano reduction score by considering the musical structure

    AU - Takamori, Hirofumi

    AU - Nakatsuka, Takayuki

    AU - Fukayama, Satoru

    AU - Goto, Masataka

    AU - Morishima, Shigeo

    PY - 2019/1/1

    Y1 - 2019/1/1

    N2 - This study describes a method that automatically generates a piano reduction score from the audio recordings of popular music while considering the musical structure. The generated score comprises both right- and left-hand piano parts, which reflect the melodies, chords, and rhythms extracted from the original audio signals. Generating such a reduction score from an audio recording is challenging because automatic music transcription is still considered to be inefficient when the input contains sounds from various instruments. Reflecting the long-term correlation structure behind similar repetitive bars is also challenging; further, previous methods have independently generated each bar. Our approach addresses the aforementioned issues by integrating musical analysis, especially structural analysis, with music generation. Our method extracts rhythmic features as well as melodies and chords from the input audio recording and reflects them in the score. To consider the long-term correlation between bars, we use similarity matrices, created for several acoustical features, as constraints. We further conduct a multivariate regression analysis to determine the acoustical features that represent the most valuable constraints for generating a musical structure. We have generated piano scores using our method and have observed that we can produce scores that differently balance between the ability to achieve rhythmic characteristics and the ability to obtain musical structures.

    AB - This study describes a method that automatically generates a piano reduction score from the audio recordings of popular music while considering the musical structure. The generated score comprises both right- and left-hand piano parts, which reflect the melodies, chords, and rhythms extracted from the original audio signals. Generating such a reduction score from an audio recording is challenging because automatic music transcription is still considered to be inefficient when the input contains sounds from various instruments. Reflecting the long-term correlation structure behind similar repetitive bars is also challenging; further, previous methods have independently generated each bar. Our approach addresses the aforementioned issues by integrating musical analysis, especially structural analysis, with music generation. Our method extracts rhythmic features as well as melodies and chords from the input audio recording and reflects them in the score. To consider the long-term correlation between bars, we use similarity matrices, created for several acoustical features, as constraints. We further conduct a multivariate regression analysis to determine the acoustical features that represent the most valuable constraints for generating a musical structure. We have generated piano scores using our method and have observed that we can produce scores that differently balance between the ability to achieve rhythmic characteristics and the ability to obtain musical structures.

    KW - Acoustic feature

    KW - Multivariate regression analysis

    KW - Musical structure

    KW - Piano reduction

    KW - Self-similarity matrix

    UR - http://www.scopus.com/inward/record.url?scp=85059836205&partnerID=8YFLogxK

    UR - http://www.scopus.com/inward/citedby.url?scp=85059836205&partnerID=8YFLogxK

    U2 - 10.1007/978-3-030-05716-9_14

    DO - 10.1007/978-3-030-05716-9_14

    M3 - Conference contribution

    SN - 9783030057152

    T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

    SP - 169

    EP - 181

    BT - MultiMedia Modeling - 25th International Conference, MMM 2019, Proceedings

    A2 - Huet, Benoit

    A2 - Kompatsiaris, Ioannis

    A2 - Vrochidis, Stefanos

    A2 - Mezaris, Vasileios

    A2 - Cheng, Wen-Huang

    A2 - Gurrin, Cathal

    PB - Springer-Verlag

    ER -