Automated violin fingering transcription through analysis of an audio recording

Akira Maezawa, Katsutoshi Itoyama, Kazunori Komatani, Tetsuya Ogata, Hiroshi G. Okuno

Research output: Contribution to journalArticle

5 Citations (Scopus)

Abstract

We present a method to recuperate fingerings for a given piece of violin music in order to recreate the timbre of a given audio recording of the piece. This is achieved by first analyzing an audio signal to determine the most likely sequence of two-dimensional fingerboard locations (string number and location along the string), which recovers elements of violin fingering relevant to timbre. This sequence is then used as a constraint for finding an ergonomic sequence of finger placements that satisfies both the sequence of notated pitch and the given fingerboard-location sequence. Fingerboard-location-sequence estimation is based on estimation of a hidden Markov model, each state of which represents a particular fingerboard location and emits a Gaussian mixture model of the relative strengths of harmonics. The relative strengths of harmonics are estimated from a polyphonic mixture using score-informed source segregation, and compensates for discrepancies between observed data and training data through mean normalization. Fingering estimation is based on the modeling of a cost function for a sequence of finger placements. We tailor our model to incorporate the playing practices of the violin. We evaluate the performance of the fingerboard-location estimator with a polyphonic mixture, and with recordings of a violin whose timbral characteristics differ significantly from that of the training data. We subjectively evaluate the fingering estimator and validate the effectiveness of tailoring the fingering model towards the violin.

Original languageEnglish
Pages (from-to)57-72
Number of pages16
JournalComputer Music Journal
Volume36
Issue number3
DOIs
Publication statusPublished - 2012 Sep
Externally publishedYes

Fingerprint

Audio recordings
Transcription
Hidden Markov models
Ergonomics
Cost functions
Violin
Fingering
Audio Recordings
Fingerboard

ASJC Scopus subject areas

  • Computer Science Applications
  • Media Technology
  • Music

Cite this

Automated violin fingering transcription through analysis of an audio recording. / Maezawa, Akira; Itoyama, Katsutoshi; Komatani, Kazunori; Ogata, Tetsuya; Okuno, Hiroshi G.

In: Computer Music Journal, Vol. 36, No. 3, 09.2012, p. 57-72.

Research output: Contribution to journalArticle

Maezawa, Akira ; Itoyama, Katsutoshi ; Komatani, Kazunori ; Ogata, Tetsuya ; Okuno, Hiroshi G. / Automated violin fingering transcription through analysis of an audio recording. In: Computer Music Journal. 2012 ; Vol. 36, No. 3. pp. 57-72.
@article{051070822b0a4463af91ed5fe3a9c088,
title = "Automated violin fingering transcription through analysis of an audio recording",
abstract = "We present a method to recuperate fingerings for a given piece of violin music in order to recreate the timbre of a given audio recording of the piece. This is achieved by first analyzing an audio signal to determine the most likely sequence of two-dimensional fingerboard locations (string number and location along the string), which recovers elements of violin fingering relevant to timbre. This sequence is then used as a constraint for finding an ergonomic sequence of finger placements that satisfies both the sequence of notated pitch and the given fingerboard-location sequence. Fingerboard-location-sequence estimation is based on estimation of a hidden Markov model, each state of which represents a particular fingerboard location and emits a Gaussian mixture model of the relative strengths of harmonics. The relative strengths of harmonics are estimated from a polyphonic mixture using score-informed source segregation, and compensates for discrepancies between observed data and training data through mean normalization. Fingering estimation is based on the modeling of a cost function for a sequence of finger placements. We tailor our model to incorporate the playing practices of the violin. We evaluate the performance of the fingerboard-location estimator with a polyphonic mixture, and with recordings of a violin whose timbral characteristics differ significantly from that of the training data. We subjectively evaluate the fingering estimator and validate the effectiveness of tailoring the fingering model towards the violin.",
author = "Akira Maezawa and Katsutoshi Itoyama and Kazunori Komatani and Tetsuya Ogata and Okuno, {Hiroshi G.}",
year = "2012",
month = "9",
doi = "10.1162/COMJ-a-00129",
language = "English",
volume = "36",
pages = "57--72",
journal = "Computer Music Journal",
issn = "0148-9267",
publisher = "MIT Press Journals",
number = "3",

}

TY - JOUR

T1 - Automated violin fingering transcription through analysis of an audio recording

AU - Maezawa, Akira

AU - Itoyama, Katsutoshi

AU - Komatani, Kazunori

AU - Ogata, Tetsuya

AU - Okuno, Hiroshi G.

PY - 2012/9

Y1 - 2012/9

N2 - We present a method to recuperate fingerings for a given piece of violin music in order to recreate the timbre of a given audio recording of the piece. This is achieved by first analyzing an audio signal to determine the most likely sequence of two-dimensional fingerboard locations (string number and location along the string), which recovers elements of violin fingering relevant to timbre. This sequence is then used as a constraint for finding an ergonomic sequence of finger placements that satisfies both the sequence of notated pitch and the given fingerboard-location sequence. Fingerboard-location-sequence estimation is based on estimation of a hidden Markov model, each state of which represents a particular fingerboard location and emits a Gaussian mixture model of the relative strengths of harmonics. The relative strengths of harmonics are estimated from a polyphonic mixture using score-informed source segregation, and compensates for discrepancies between observed data and training data through mean normalization. Fingering estimation is based on the modeling of a cost function for a sequence of finger placements. We tailor our model to incorporate the playing practices of the violin. We evaluate the performance of the fingerboard-location estimator with a polyphonic mixture, and with recordings of a violin whose timbral characteristics differ significantly from that of the training data. We subjectively evaluate the fingering estimator and validate the effectiveness of tailoring the fingering model towards the violin.

AB - We present a method to recuperate fingerings for a given piece of violin music in order to recreate the timbre of a given audio recording of the piece. This is achieved by first analyzing an audio signal to determine the most likely sequence of two-dimensional fingerboard locations (string number and location along the string), which recovers elements of violin fingering relevant to timbre. This sequence is then used as a constraint for finding an ergonomic sequence of finger placements that satisfies both the sequence of notated pitch and the given fingerboard-location sequence. Fingerboard-location-sequence estimation is based on estimation of a hidden Markov model, each state of which represents a particular fingerboard location and emits a Gaussian mixture model of the relative strengths of harmonics. The relative strengths of harmonics are estimated from a polyphonic mixture using score-informed source segregation, and compensates for discrepancies between observed data and training data through mean normalization. Fingering estimation is based on the modeling of a cost function for a sequence of finger placements. We tailor our model to incorporate the playing practices of the violin. We evaluate the performance of the fingerboard-location estimator with a polyphonic mixture, and with recordings of a violin whose timbral characteristics differ significantly from that of the training data. We subjectively evaluate the fingering estimator and validate the effectiveness of tailoring the fingering model towards the violin.

UR - http://www.scopus.com/inward/record.url?scp=84865446773&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84865446773&partnerID=8YFLogxK

U2 - 10.1162/COMJ-a-00129

DO - 10.1162/COMJ-a-00129

M3 - Article

AN - SCOPUS:84865446773

VL - 36

SP - 57

EP - 72

JO - Computer Music Journal

JF - Computer Music Journal

SN - 0148-9267

IS - 3

ER -