Abstract
This paper describes a system that can automatically synchronize polyphonic musical audio signals with their corresponding lyrics. Although methods for synchronizing monophonic speech signals and corresponding text transcriptions by using Viterbi alignment techniques have been proposed, these methods cannot be applied to vocals in CD recordings because vocals are often overlapped by accompaniment sounds. In addition to a conventional method for reducing the influence of the accompaniment sounds, we therefore developed four methods to overcome this problem: a method for detecting vocal sections, a method for constructing robust phoneme networks, a method for detecting fricative sounds, and a method for adapting a speech-recognizer phone model to segregated vocal signals. We then report experimental results for each of these methods and also describe our music playback interface that utilizes our system for synchronizing music and lyrics.
| Original language | English |
|---|---|
| Article number | 5876296 |
| Pages (from-to) | 1252-1261 |
| Number of pages | 10 |
| Journal | IEEE Journal on Selected Topics in Signal Processing |
| Volume | 5 |
| Issue number | 6 |
| DOIs | |
| Publication status | Published - 2011 Oct |
| Externally published | Yes |
Keywords
- Alignment
- Lyrics
- Singing voice
- Viterbi algorithm
- Vocal
ASJC Scopus subject areas
- Electrical and Electronic Engineering
- Signal Processing