Improving End-to-end Speech Recognition with Pronunciation-assisted Sub-word Modeling

Hainan Xu; Shuoyang DIng; Shinji Watanabe

doi:10.1109/ICASSP.2019.8682494

Improving End-to-end Speech Recognition with Pronunciation-assisted Sub-word Modeling

Hainan Xu, Shuoyang DIng, Shinji Watanabe

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

21 Citations (Scopus)

Abstract

Most end-to-end speech recognition systems model text directly as a sequence of characters or sub-words. Current approaches to sub-word extraction only consider character sequence frequencies, which at times produce inferior sub-word segmentation that might lead to erroneous speech recognition output. We propose pronunciation-assisted sub-word modeling (PASM), a sub-word extraction method that leverages the pronunciation information of a word. Experiments show that the proposed method can greatly improve upon the character-based baseline, and also outperform commonly used byte-pair encoding methods.

Original language	English
Title of host publication	2019 IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP 2019 - Proceedings
Publisher	Institute of Electrical and Electronics Engineers Inc.
Pages	7110-7114
Number of pages	5
ISBN (Electronic)	9781479981311
DOIs	https://doi.org/10.1109/ICASSP.2019.8682494
Publication status	Published - 2019 May
Externally published	Yes
Event	44th IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP 2019 - Brighton, United Kingdom Duration: 2019 May 12 → 2019 May 17

Publication series

Name	ICASSP, IEEE International Conference on Acoustics, Speech and Signal Processing - Proceedings
Volume	2019-May
ISSN (Print)	1520-6149

Conference

Conference	44th IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP 2019
Country/Territory	United Kingdom
City	Brighton
Period	19/5/12 → 19/5/17

Keywords

end-to-end models
speech recognition
sub-word modeling

ASJC Scopus subject areas

Software
Signal Processing
Electrical and Electronic Engineering

Access to Document

10.1109/ICASSP.2019.8682494

Cite this

Xu, H., DIng, S., & Watanabe, S. (2019). Improving End-to-end Speech Recognition with Pronunciation-assisted Sub-word Modeling. In 2019 IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP 2019 - Proceedings (pp. 7110-7114). [8682494] (ICASSP, IEEE International Conference on Acoustics, Speech and Signal Processing - Proceedings; Vol. 2019-May). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/ICASSP.2019.8682494

Improving End-to-end Speech Recognition with Pronunciation-assisted Sub-word Modeling. / Xu, Hainan; DIng, Shuoyang; Watanabe, Shinji.

2019 IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP 2019 - Proceedings. Institute of Electrical and Electronics Engineers Inc., 2019. p. 7110-7114 8682494 (ICASSP, IEEE International Conference on Acoustics, Speech and Signal Processing - Proceedings; Vol. 2019-May).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Xu, H, DIng, S & Watanabe, S 2019, Improving End-to-end Speech Recognition with Pronunciation-assisted Sub-word Modeling. in 2019 IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP 2019 - Proceedings., 8682494, ICASSP, IEEE International Conference on Acoustics, Speech and Signal Processing - Proceedings, vol. 2019-May, Institute of Electrical and Electronics Engineers Inc., pp. 7110-7114, 44th IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP 2019, Brighton, United Kingdom, 19/5/12. https://doi.org/10.1109/ICASSP.2019.8682494

Xu H, DIng S, Watanabe S. Improving End-to-end Speech Recognition with Pronunciation-assisted Sub-word Modeling. In 2019 IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP 2019 - Proceedings. Institute of Electrical and Electronics Engineers Inc. 2019. p. 7110-7114. 8682494. (ICASSP, IEEE International Conference on Acoustics, Speech and Signal Processing - Proceedings). doi: 10.1109/ICASSP.2019.8682494

Xu, Hainan ; DIng, Shuoyang ; Watanabe, Shinji. / Improving End-to-end Speech Recognition with Pronunciation-assisted Sub-word Modeling. 2019 IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP 2019 - Proceedings. Institute of Electrical and Electronics Engineers Inc., 2019. pp. 7110-7114 (ICASSP, IEEE International Conference on Acoustics, Speech and Signal Processing - Proceedings).

@inproceedings{887de62071f740f48d36bbb7aa5991d7,

title = "Improving End-to-end Speech Recognition with Pronunciation-assisted Sub-word Modeling",

abstract = "Most end-to-end speech recognition systems model text directly as a sequence of characters or sub-words. Current approaches to sub-word extraction only consider character sequence frequencies, which at times produce inferior sub-word segmentation that might lead to erroneous speech recognition output. We propose pronunciation-assisted sub-word modeling (PASM), a sub-word extraction method that leverages the pronunciation information of a word. Experiments show that the proposed method can greatly improve upon the character-based baseline, and also outperform commonly used byte-pair encoding methods.",

keywords = "end-to-end models, speech recognition, sub-word modeling",

author = "Hainan Xu and Shuoyang DIng and Shinji Watanabe",

note = "Publisher Copyright: {\textcopyright} 2019 IEEE.; 44th IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP 2019 ; Conference date: 12-05-2019 Through 17-05-2019",

year = "2019",

month = may,

doi = "10.1109/ICASSP.2019.8682494",

language = "English",

series = "ICASSP, IEEE International Conference on Acoustics, Speech and Signal Processing - Proceedings",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

pages = "7110--7114",

booktitle = "2019 IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP 2019 - Proceedings",

}

TY - GEN

T1 - Improving End-to-end Speech Recognition with Pronunciation-assisted Sub-word Modeling

AU - Xu, Hainan

AU - DIng, Shuoyang

AU - Watanabe, Shinji

PY - 2019/5

Y1 - 2019/5

N2 - Most end-to-end speech recognition systems model text directly as a sequence of characters or sub-words. Current approaches to sub-word extraction only consider character sequence frequencies, which at times produce inferior sub-word segmentation that might lead to erroneous speech recognition output. We propose pronunciation-assisted sub-word modeling (PASM), a sub-word extraction method that leverages the pronunciation information of a word. Experiments show that the proposed method can greatly improve upon the character-based baseline, and also outperform commonly used byte-pair encoding methods.

AB - Most end-to-end speech recognition systems model text directly as a sequence of characters or sub-words. Current approaches to sub-word extraction only consider character sequence frequencies, which at times produce inferior sub-word segmentation that might lead to erroneous speech recognition output. We propose pronunciation-assisted sub-word modeling (PASM), a sub-word extraction method that leverages the pronunciation information of a word. Experiments show that the proposed method can greatly improve upon the character-based baseline, and also outperform commonly used byte-pair encoding methods.

KW - end-to-end models

KW - speech recognition

KW - sub-word modeling

UR - http://www.scopus.com/inward/record.url?scp=85068978138&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85068978138&partnerID=8YFLogxK

U2 - 10.1109/ICASSP.2019.8682494

DO - 10.1109/ICASSP.2019.8682494

M3 - Conference contribution

AN - SCOPUS:85068978138

T3 - ICASSP, IEEE International Conference on Acoustics, Speech and Signal Processing - Proceedings

SP - 7110

EP - 7114

BT - 2019 IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP 2019 - Proceedings

PB - Institute of Electrical and Electronics Engineers Inc.

T2 - 44th IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP 2019

Y2 - 12 May 2019 through 17 May 2019

ER -

Improving End-to-end Speech Recognition with Pronunciation-assisted Sub-word Modeling

Abstract

Publication series

Conference

Keywords

ASJC Scopus subject areas

Access to Document

Other files and links

Fingerprint

Cite this