Changing timbre and phrase in existing musical performances as you like - Manipulations of single part using harmonic and inharmonic models

Naoki Yasuraoka, Takehiro Abe, Katsutoshi Itoyama, Toru Takahashi, Tetsuya Ogata, Hiroshi G. Okuno

Research output: Chapter in Book/Report/Conference proceedingConference contribution

9 Citations (Scopus)

Abstract

This paper presents a new music manipulation method that can change the timbre and phrases of an existing instrumental performance in a polyphonic sound mixture. This method consists of three primitive functions: 1) extracting and analyzing of a single instrumental part from polyphonic music signals, 2) mixing the instrument timbre with another, and 3) rendering a new phrase expression for another given score. The resulting customized part is re-mixed with the remaining parts of the original performance to generate new polyphonic music signals. A single instrumental part is extracted by using an integrated tone model that consists of harmonic and inharmonic tone models with the aid of the score of the single instrumental part. The extraction incorporates a residual model for the single instrumental part in order to avoid crosstalk between instrumental parts. The extracted model parameters are classified into their averages and deviations. The former is treated as instrument timbre and is customized by mixing, while the latter is treated as phrase expression and is customized by rendering. We evaluated our method in three experiments. The first experiment focused on introduction of the residual model, and it showed that the model parameters are estimated more accurately by 35.0 points. The second focused on timbral customization, and it showed that our method is more robust by 42.9 points in spectral distance compared with a conventional sound analysis-synthesis method, STRAIGHT. The third focused on the acoustic fidelity of customizing performance, and it showed that rendering phrase expression according to the note sequence leads to more accurate performance by 9.2 points in spectral distance in comparison with a rendering method that ignores the note sequence.

Original languageEnglish
Title of host publicationMM'09 - Proceedings of the 2009 ACM Multimedia Conference, with Co-located Workshops and Symposiums
Pages203-212
Number of pages10
DOIs
Publication statusPublished - 2009
Externally publishedYes
Event17th ACM International Conference on Multimedia, MM'09, with Co-located Workshops and Symposiums - Beijing
Duration: 2009 Oct 192009 Oct 24

Other

Other17th ACM International Conference on Multimedia, MM'09, with Co-located Workshops and Symposiums
CityBeijing
Period09/10/1909/10/24

Fingerprint

Acoustic waves
Crosstalk
Acoustics
Experiments

Keywords

  • Music manipulation
  • Performance rendering
  • Signal processing
  • Sound source extraction
  • Timbre mixing

ASJC Scopus subject areas

  • Computational Theory and Mathematics
  • Computer Science Applications
  • Computer Vision and Pattern Recognition
  • Software

Cite this

Yasuraoka, N., Abe, T., Itoyama, K., Takahashi, T., Ogata, T., & Okuno, H. G. (2009). Changing timbre and phrase in existing musical performances as you like - Manipulations of single part using harmonic and inharmonic models. In MM'09 - Proceedings of the 2009 ACM Multimedia Conference, with Co-located Workshops and Symposiums (pp. 203-212) https://doi.org/10.1145/1631272.1631302

Changing timbre and phrase in existing musical performances as you like - Manipulations of single part using harmonic and inharmonic models. / Yasuraoka, Naoki; Abe, Takehiro; Itoyama, Katsutoshi; Takahashi, Toru; Ogata, Tetsuya; Okuno, Hiroshi G.

MM'09 - Proceedings of the 2009 ACM Multimedia Conference, with Co-located Workshops and Symposiums. 2009. p. 203-212.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Yasuraoka, N, Abe, T, Itoyama, K, Takahashi, T, Ogata, T & Okuno, HG 2009, Changing timbre and phrase in existing musical performances as you like - Manipulations of single part using harmonic and inharmonic models. in MM'09 - Proceedings of the 2009 ACM Multimedia Conference, with Co-located Workshops and Symposiums. pp. 203-212, 17th ACM International Conference on Multimedia, MM'09, with Co-located Workshops and Symposiums, Beijing, 09/10/19. https://doi.org/10.1145/1631272.1631302
Yasuraoka N, Abe T, Itoyama K, Takahashi T, Ogata T, Okuno HG. Changing timbre and phrase in existing musical performances as you like - Manipulations of single part using harmonic and inharmonic models. In MM'09 - Proceedings of the 2009 ACM Multimedia Conference, with Co-located Workshops and Symposiums. 2009. p. 203-212 https://doi.org/10.1145/1631272.1631302
Yasuraoka, Naoki ; Abe, Takehiro ; Itoyama, Katsutoshi ; Takahashi, Toru ; Ogata, Tetsuya ; Okuno, Hiroshi G. / Changing timbre and phrase in existing musical performances as you like - Manipulations of single part using harmonic and inharmonic models. MM'09 - Proceedings of the 2009 ACM Multimedia Conference, with Co-located Workshops and Symposiums. 2009. pp. 203-212
@inproceedings{7bdc4312d2344ece99fdf3758d88a3e9,
title = "Changing timbre and phrase in existing musical performances as you like - Manipulations of single part using harmonic and inharmonic models",
abstract = "This paper presents a new music manipulation method that can change the timbre and phrases of an existing instrumental performance in a polyphonic sound mixture. This method consists of three primitive functions: 1) extracting and analyzing of a single instrumental part from polyphonic music signals, 2) mixing the instrument timbre with another, and 3) rendering a new phrase expression for another given score. The resulting customized part is re-mixed with the remaining parts of the original performance to generate new polyphonic music signals. A single instrumental part is extracted by using an integrated tone model that consists of harmonic and inharmonic tone models with the aid of the score of the single instrumental part. The extraction incorporates a residual model for the single instrumental part in order to avoid crosstalk between instrumental parts. The extracted model parameters are classified into their averages and deviations. The former is treated as instrument timbre and is customized by mixing, while the latter is treated as phrase expression and is customized by rendering. We evaluated our method in three experiments. The first experiment focused on introduction of the residual model, and it showed that the model parameters are estimated more accurately by 35.0 points. The second focused on timbral customization, and it showed that our method is more robust by 42.9 points in spectral distance compared with a conventional sound analysis-synthesis method, STRAIGHT. The third focused on the acoustic fidelity of customizing performance, and it showed that rendering phrase expression according to the note sequence leads to more accurate performance by 9.2 points in spectral distance in comparison with a rendering method that ignores the note sequence.",
keywords = "Music manipulation, Performance rendering, Signal processing, Sound source extraction, Timbre mixing",
author = "Naoki Yasuraoka and Takehiro Abe and Katsutoshi Itoyama and Toru Takahashi and Tetsuya Ogata and Okuno, {Hiroshi G.}",
year = "2009",
doi = "10.1145/1631272.1631302",
language = "English",
isbn = "9781605586083",
pages = "203--212",
booktitle = "MM'09 - Proceedings of the 2009 ACM Multimedia Conference, with Co-located Workshops and Symposiums",

}

TY - GEN

T1 - Changing timbre and phrase in existing musical performances as you like - Manipulations of single part using harmonic and inharmonic models

AU - Yasuraoka, Naoki

AU - Abe, Takehiro

AU - Itoyama, Katsutoshi

AU - Takahashi, Toru

AU - Ogata, Tetsuya

AU - Okuno, Hiroshi G.

PY - 2009

Y1 - 2009

N2 - This paper presents a new music manipulation method that can change the timbre and phrases of an existing instrumental performance in a polyphonic sound mixture. This method consists of three primitive functions: 1) extracting and analyzing of a single instrumental part from polyphonic music signals, 2) mixing the instrument timbre with another, and 3) rendering a new phrase expression for another given score. The resulting customized part is re-mixed with the remaining parts of the original performance to generate new polyphonic music signals. A single instrumental part is extracted by using an integrated tone model that consists of harmonic and inharmonic tone models with the aid of the score of the single instrumental part. The extraction incorporates a residual model for the single instrumental part in order to avoid crosstalk between instrumental parts. The extracted model parameters are classified into their averages and deviations. The former is treated as instrument timbre and is customized by mixing, while the latter is treated as phrase expression and is customized by rendering. We evaluated our method in three experiments. The first experiment focused on introduction of the residual model, and it showed that the model parameters are estimated more accurately by 35.0 points. The second focused on timbral customization, and it showed that our method is more robust by 42.9 points in spectral distance compared with a conventional sound analysis-synthesis method, STRAIGHT. The third focused on the acoustic fidelity of customizing performance, and it showed that rendering phrase expression according to the note sequence leads to more accurate performance by 9.2 points in spectral distance in comparison with a rendering method that ignores the note sequence.

AB - This paper presents a new music manipulation method that can change the timbre and phrases of an existing instrumental performance in a polyphonic sound mixture. This method consists of three primitive functions: 1) extracting and analyzing of a single instrumental part from polyphonic music signals, 2) mixing the instrument timbre with another, and 3) rendering a new phrase expression for another given score. The resulting customized part is re-mixed with the remaining parts of the original performance to generate new polyphonic music signals. A single instrumental part is extracted by using an integrated tone model that consists of harmonic and inharmonic tone models with the aid of the score of the single instrumental part. The extraction incorporates a residual model for the single instrumental part in order to avoid crosstalk between instrumental parts. The extracted model parameters are classified into their averages and deviations. The former is treated as instrument timbre and is customized by mixing, while the latter is treated as phrase expression and is customized by rendering. We evaluated our method in three experiments. The first experiment focused on introduction of the residual model, and it showed that the model parameters are estimated more accurately by 35.0 points. The second focused on timbral customization, and it showed that our method is more robust by 42.9 points in spectral distance compared with a conventional sound analysis-synthesis method, STRAIGHT. The third focused on the acoustic fidelity of customizing performance, and it showed that rendering phrase expression according to the note sequence leads to more accurate performance by 9.2 points in spectral distance in comparison with a rendering method that ignores the note sequence.

KW - Music manipulation

KW - Performance rendering

KW - Signal processing

KW - Sound source extraction

KW - Timbre mixing

UR - http://www.scopus.com/inward/record.url?scp=72449160099&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=72449160099&partnerID=8YFLogxK

U2 - 10.1145/1631272.1631302

DO - 10.1145/1631272.1631302

M3 - Conference contribution

AN - SCOPUS:72449160099

SN - 9781605586083

SP - 203

EP - 212

BT - MM'09 - Proceedings of the 2009 ACM Multimedia Conference, with Co-located Workshops and Symposiums

ER -