Multi-class composite N-gram language model

Hirofumi Yamamoto, Shuntaro Isogai, Yoshinori Sagisaka

Research output: Contribution to journalArticle

35 Citations (Scopus)

Abstract

A new language model is proposed to cope with the scarcity of training data. The proposed multi-class N-gram achieves an accurate word prediction capability and high reliability with a small number of model parameters by clustering words multi-dimensionally into classes, where the left and right context are independently treated. Each multiple class is assigned by a grouping process based on the left and right neighboring characteristics. Furthermore, by introducing frequent word successions to partially include higher order statistics, multi-class N-grams are extended to more efficient multi-class composite N-grams. In comparison to conventional word tri-grams, the multi-class composite N-grams achieved 9.5% lower perplexity and a 16% lower word error rate in a speech recognition experiment with a 40% smaller parameter size.

Original languageEnglish
Pages (from-to)369-379
Number of pages11
JournalSpeech Communication
Volume41
Issue number2-3
DOIs
Publication statusPublished - 2003 Oct
Externally publishedYes

Fingerprint

N-gram
Language Model
Multi-class
Cluster Analysis
Language
Composite
Higher order statistics
Composite materials
language
Speech recognition
grouping
statistics
Higher-order Statistics
experiment
Speech Recognition
Small Parameter
Grouping
Error Rate
Experiments
Clustering

Keywords

  • Class N-gram
  • N-gram language model
  • Variable length N-gram
  • Word clustering

ASJC Scopus subject areas

  • Signal Processing
  • Electrical and Electronic Engineering
  • Experimental and Cognitive Psychology
  • Linguistics and Language

Cite this

Multi-class composite N-gram language model. / Yamamoto, Hirofumi; Isogai, Shuntaro; Sagisaka, Yoshinori.

In: Speech Communication, Vol. 41, No. 2-3, 10.2003, p. 369-379.

Research output: Contribution to journalArticle

Yamamoto, Hirofumi ; Isogai, Shuntaro ; Sagisaka, Yoshinori. / Multi-class composite N-gram language model. In: Speech Communication. 2003 ; Vol. 41, No. 2-3. pp. 369-379.
@article{e31d4e3d508e40cdaa148ee40c47e172,
title = "Multi-class composite N-gram language model",
abstract = "A new language model is proposed to cope with the scarcity of training data. The proposed multi-class N-gram achieves an accurate word prediction capability and high reliability with a small number of model parameters by clustering words multi-dimensionally into classes, where the left and right context are independently treated. Each multiple class is assigned by a grouping process based on the left and right neighboring characteristics. Furthermore, by introducing frequent word successions to partially include higher order statistics, multi-class N-grams are extended to more efficient multi-class composite N-grams. In comparison to conventional word tri-grams, the multi-class composite N-grams achieved 9.5{\%} lower perplexity and a 16{\%} lower word error rate in a speech recognition experiment with a 40{\%} smaller parameter size.",
keywords = "Class N-gram, N-gram language model, Variable length N-gram, Word clustering",
author = "Hirofumi Yamamoto and Shuntaro Isogai and Yoshinori Sagisaka",
year = "2003",
month = "10",
doi = "10.1016/S0167-6393(02)00179-6",
language = "English",
volume = "41",
pages = "369--379",
journal = "Speech Communication",
issn = "0167-6393",
publisher = "Elsevier",
number = "2-3",

}

TY - JOUR

T1 - Multi-class composite N-gram language model

AU - Yamamoto, Hirofumi

AU - Isogai, Shuntaro

AU - Sagisaka, Yoshinori

PY - 2003/10

Y1 - 2003/10

N2 - A new language model is proposed to cope with the scarcity of training data. The proposed multi-class N-gram achieves an accurate word prediction capability and high reliability with a small number of model parameters by clustering words multi-dimensionally into classes, where the left and right context are independently treated. Each multiple class is assigned by a grouping process based on the left and right neighboring characteristics. Furthermore, by introducing frequent word successions to partially include higher order statistics, multi-class N-grams are extended to more efficient multi-class composite N-grams. In comparison to conventional word tri-grams, the multi-class composite N-grams achieved 9.5% lower perplexity and a 16% lower word error rate in a speech recognition experiment with a 40% smaller parameter size.

AB - A new language model is proposed to cope with the scarcity of training data. The proposed multi-class N-gram achieves an accurate word prediction capability and high reliability with a small number of model parameters by clustering words multi-dimensionally into classes, where the left and right context are independently treated. Each multiple class is assigned by a grouping process based on the left and right neighboring characteristics. Furthermore, by introducing frequent word successions to partially include higher order statistics, multi-class N-grams are extended to more efficient multi-class composite N-grams. In comparison to conventional word tri-grams, the multi-class composite N-grams achieved 9.5% lower perplexity and a 16% lower word error rate in a speech recognition experiment with a 40% smaller parameter size.

KW - Class N-gram

KW - N-gram language model

KW - Variable length N-gram

KW - Word clustering

UR - http://www.scopus.com/inward/record.url?scp=0038373395&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=0038373395&partnerID=8YFLogxK

U2 - 10.1016/S0167-6393(02)00179-6

DO - 10.1016/S0167-6393(02)00179-6

M3 - Article

VL - 41

SP - 369

EP - 379

JO - Speech Communication

JF - Speech Communication

SN - 0167-6393

IS - 2-3

ER -