Adjusting occurrence probabilities of automatically-generated abbreviated words in spoken dialogue systems

Masaki Katsumaru, Kazunori Komatani, Tetsuya Ogata, Hiroshi G. Okuno

Research output: Chapter in Book/Report/Conference proceedingConference contribution

2 Citations (Scopus)

Abstract

Users often abbreviate long words when using spoken dialogue systems, which results in automatic speech recognition (ASR) errors. We define abbreviated words as sub-words of an original word and add them to the ASR dictionary. The first problem we face is that proper nouns cannot be correctly segmented by general morphological analyzers, although long and compound words need to be segmented in agglutinative languages such as Japanese. The second is that, as vocabulary size increases, adding many abbreviated words degrades the ASR accuracy. We have developed two methods, (1) to segment words by using conjunction probabilities between characters, and (2) to adjust occurrence probabilities of generated abbreviated words on the basis of the following two cues: phonological similarities between the abbreviated and original words and frequencies of abbreviated words in Web documents. Our method improves ASR accuracy by 34.9 points for utterances containing abbreviated words without degrading the accuracy for utterances containing original words.

Original languageEnglish
Title of host publicationLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Pages481-490
Number of pages10
Volume5579 LNAI
DOIs
Publication statusPublished - 2009
Externally publishedYes
Event22nd International Conference on Industrial, Engineering and Other Applications of Applied Intelligent Systems, IEA/AIE 2009 - Tainan
Duration: 2009 Jun 242009 Jun 27

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume5579 LNAI
ISSN (Print)03029743
ISSN (Electronic)16113349

Other

Other22nd International Conference on Industrial, Engineering and Other Applications of Applied Intelligent Systems, IEA/AIE 2009
CityTainan
Period09/6/2409/6/27

Fingerprint

Spoken Dialogue Systems
Automatic Speech Recognition
Speech recognition
Subword
Glossaries

ASJC Scopus subject areas

  • Computer Science(all)
  • Theoretical Computer Science

Cite this

Katsumaru, M., Komatani, K., Ogata, T., & Okuno, H. G. (2009). Adjusting occurrence probabilities of automatically-generated abbreviated words in spoken dialogue systems. In Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) (Vol. 5579 LNAI, pp. 481-490). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 5579 LNAI). https://doi.org/10.1007/978-3-642-02568-6_49

Adjusting occurrence probabilities of automatically-generated abbreviated words in spoken dialogue systems. / Katsumaru, Masaki; Komatani, Kazunori; Ogata, Tetsuya; Okuno, Hiroshi G.

Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics). Vol. 5579 LNAI 2009. p. 481-490 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 5579 LNAI).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Katsumaru, M, Komatani, K, Ogata, T & Okuno, HG 2009, Adjusting occurrence probabilities of automatically-generated abbreviated words in spoken dialogue systems. in Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics). vol. 5579 LNAI, Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 5579 LNAI, pp. 481-490, 22nd International Conference on Industrial, Engineering and Other Applications of Applied Intelligent Systems, IEA/AIE 2009, Tainan, 09/6/24. https://doi.org/10.1007/978-3-642-02568-6_49
Katsumaru M, Komatani K, Ogata T, Okuno HG. Adjusting occurrence probabilities of automatically-generated abbreviated words in spoken dialogue systems. In Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics). Vol. 5579 LNAI. 2009. p. 481-490. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). https://doi.org/10.1007/978-3-642-02568-6_49
Katsumaru, Masaki ; Komatani, Kazunori ; Ogata, Tetsuya ; Okuno, Hiroshi G. / Adjusting occurrence probabilities of automatically-generated abbreviated words in spoken dialogue systems. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics). Vol. 5579 LNAI 2009. pp. 481-490 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).
@inproceedings{47701c37975f42fb9c70df738a20ae0b,
title = "Adjusting occurrence probabilities of automatically-generated abbreviated words in spoken dialogue systems",
abstract = "Users often abbreviate long words when using spoken dialogue systems, which results in automatic speech recognition (ASR) errors. We define abbreviated words as sub-words of an original word and add them to the ASR dictionary. The first problem we face is that proper nouns cannot be correctly segmented by general morphological analyzers, although long and compound words need to be segmented in agglutinative languages such as Japanese. The second is that, as vocabulary size increases, adding many abbreviated words degrades the ASR accuracy. We have developed two methods, (1) to segment words by using conjunction probabilities between characters, and (2) to adjust occurrence probabilities of generated abbreviated words on the basis of the following two cues: phonological similarities between the abbreviated and original words and frequencies of abbreviated words in Web documents. Our method improves ASR accuracy by 34.9 points for utterances containing abbreviated words without degrading the accuracy for utterances containing original words.",
author = "Masaki Katsumaru and Kazunori Komatani and Tetsuya Ogata and Okuno, {Hiroshi G.}",
year = "2009",
doi = "10.1007/978-3-642-02568-6_49",
language = "English",
isbn = "3642025676",
volume = "5579 LNAI",
series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",
pages = "481--490",
booktitle = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",

}

TY - GEN

T1 - Adjusting occurrence probabilities of automatically-generated abbreviated words in spoken dialogue systems

AU - Katsumaru, Masaki

AU - Komatani, Kazunori

AU - Ogata, Tetsuya

AU - Okuno, Hiroshi G.

PY - 2009

Y1 - 2009

N2 - Users often abbreviate long words when using spoken dialogue systems, which results in automatic speech recognition (ASR) errors. We define abbreviated words as sub-words of an original word and add them to the ASR dictionary. The first problem we face is that proper nouns cannot be correctly segmented by general morphological analyzers, although long and compound words need to be segmented in agglutinative languages such as Japanese. The second is that, as vocabulary size increases, adding many abbreviated words degrades the ASR accuracy. We have developed two methods, (1) to segment words by using conjunction probabilities between characters, and (2) to adjust occurrence probabilities of generated abbreviated words on the basis of the following two cues: phonological similarities between the abbreviated and original words and frequencies of abbreviated words in Web documents. Our method improves ASR accuracy by 34.9 points for utterances containing abbreviated words without degrading the accuracy for utterances containing original words.

AB - Users often abbreviate long words when using spoken dialogue systems, which results in automatic speech recognition (ASR) errors. We define abbreviated words as sub-words of an original word and add them to the ASR dictionary. The first problem we face is that proper nouns cannot be correctly segmented by general morphological analyzers, although long and compound words need to be segmented in agglutinative languages such as Japanese. The second is that, as vocabulary size increases, adding many abbreviated words degrades the ASR accuracy. We have developed two methods, (1) to segment words by using conjunction probabilities between characters, and (2) to adjust occurrence probabilities of generated abbreviated words on the basis of the following two cues: phonological similarities between the abbreviated and original words and frequencies of abbreviated words in Web documents. Our method improves ASR accuracy by 34.9 points for utterances containing abbreviated words without degrading the accuracy for utterances containing original words.

UR - http://www.scopus.com/inward/record.url?scp=70350660538&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=70350660538&partnerID=8YFLogxK

U2 - 10.1007/978-3-642-02568-6_49

DO - 10.1007/978-3-642-02568-6_49

M3 - Conference contribution

AN - SCOPUS:70350660538

SN - 3642025676

SN - 9783642025679

VL - 5579 LNAI

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 481

EP - 490

BT - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

ER -