Advances in joint CTC-attention based end-to-end speech recognition with a deep CNN encoder and RNN-LM

Takaaki Hori; Shinji Watanabe; Yu Zhang; William Chan

doi:10.21437/Interspeech.2017-1296

Advances in joint CTC-attention based end-to-end speech recognition with a deep CNN encoder and RNN-LM

Takaaki Hori, Shinji Watanabe, Yu Zhang, William Chan

Research output: Contribution to journal › Conference article › peer-review

151 Citations (Scopus)

Abstract

We present a state-of-the-art end-to-end Automatic Speech Recognition (ASR) model. We learn to listen and write characters with a joint Connectionist Temporal Classification (CTC) and attention-based encoder-decoder network. The encoder is a deep Convolutional Neural Network (CNN) based on the VGG network. The CTC network sits on top of the encoder and is jointly trained with the attention-based decoder. During the beam search process, we combine the CTC predictions, the attention-based decoder predictions and a separately trained LSTM language model. We achieve a 5-10% error reduction compared to prior systems on spontaneous Japanese and Chinese speech, and our end-to-end model beats out traditional hybrid ASR systems.

Original language	English
Pages (from-to)	949-953
Number of pages	5
Journal	Proceedings of the Annual Conference of the International Speech Communication Association, INTERSPEECH
Volume	2017-August
DOIs	https://doi.org/10.21437/Interspeech.2017-1296
Publication status	Published - 2017
Externally published	Yes
Event	18th Annual Conference of the International Speech Communication Association, INTERSPEECH 2017 - Stockholm, Sweden Duration: 2017 Aug 20 → 2017 Aug 24

Keywords

Attention model
Connectionist temporal classification
Encoder-decoder
End-to-end speech recognition

ASJC Scopus subject areas

Language and Linguistics
Human-Computer Interaction
Signal Processing
Software
Modelling and Simulation

Access to Document

10.21437/Interspeech.2017-1296

Cite this

Advances in joint CTC-attention based end-to-end speech recognition with a deep CNN encoder and RNN-LM. / Hori, Takaaki; Watanabe, Shinji; Zhang, Yu et al.

In: Proceedings of the Annual Conference of the International Speech Communication Association, INTERSPEECH, Vol. 2017-August, 2017, p. 949-953.

Research output: Contribution to journal › Conference article › peer-review

@article{3132409069354fa087182c81827e780d,

title = "Advances in joint CTC-attention based end-to-end speech recognition with a deep CNN encoder and RNN-LM",

abstract = "We present a state-of-the-art end-to-end Automatic Speech Recognition (ASR) model. We learn to listen and write characters with a joint Connectionist Temporal Classification (CTC) and attention-based encoder-decoder network. The encoder is a deep Convolutional Neural Network (CNN) based on the VGG network. The CTC network sits on top of the encoder and is jointly trained with the attention-based decoder. During the beam search process, we combine the CTC predictions, the attention-based decoder predictions and a separately trained LSTM language model. We achieve a 5-10% error reduction compared to prior systems on spontaneous Japanese and Chinese speech, and our end-to-end model beats out traditional hybrid ASR systems.",

keywords = "Attention model, Connectionist temporal classification, Encoder-decoder, End-to-end speech recognition",

author = "Takaaki Hori and Shinji Watanabe and Yu Zhang and William Chan",

note = "Publisher Copyright: Copyright {\textcopyright} 2017 ISCA.; 18th Annual Conference of the International Speech Communication Association, INTERSPEECH 2017 ; Conference date: 20-08-2017 Through 24-08-2017",

year = "2017",

doi = "10.21437/Interspeech.2017-1296",

language = "English",

volume = "2017-August",

pages = "949--953",

journal = "Proceedings of the Annual Conference of the International Speech Communication Association, INTERSPEECH",

issn = "2308-457X",

}

TY - JOUR

T1 - Advances in joint CTC-attention based end-to-end speech recognition with a deep CNN encoder and RNN-LM

AU - Hori, Takaaki

AU - Watanabe, Shinji

AU - Zhang, Yu

AU - Chan, William

PY - 2017

Y1 - 2017

N2 - We present a state-of-the-art end-to-end Automatic Speech Recognition (ASR) model. We learn to listen and write characters with a joint Connectionist Temporal Classification (CTC) and attention-based encoder-decoder network. The encoder is a deep Convolutional Neural Network (CNN) based on the VGG network. The CTC network sits on top of the encoder and is jointly trained with the attention-based decoder. During the beam search process, we combine the CTC predictions, the attention-based decoder predictions and a separately trained LSTM language model. We achieve a 5-10% error reduction compared to prior systems on spontaneous Japanese and Chinese speech, and our end-to-end model beats out traditional hybrid ASR systems.

AB - We present a state-of-the-art end-to-end Automatic Speech Recognition (ASR) model. We learn to listen and write characters with a joint Connectionist Temporal Classification (CTC) and attention-based encoder-decoder network. The encoder is a deep Convolutional Neural Network (CNN) based on the VGG network. The CTC network sits on top of the encoder and is jointly trained with the attention-based decoder. During the beam search process, we combine the CTC predictions, the attention-based decoder predictions and a separately trained LSTM language model. We achieve a 5-10% error reduction compared to prior systems on spontaneous Japanese and Chinese speech, and our end-to-end model beats out traditional hybrid ASR systems.

KW - Attention model

KW - Connectionist temporal classification

KW - Encoder-decoder

KW - End-to-end speech recognition

UR - http://www.scopus.com/inward/record.url?scp=85039169903&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85039169903&partnerID=8YFLogxK

U2 - 10.21437/Interspeech.2017-1296

DO - 10.21437/Interspeech.2017-1296

M3 - Conference article

AN - SCOPUS:85039169903

VL - 2017-August

SP - 949

EP - 953

JO - Proceedings of the Annual Conference of the International Speech Communication Association, INTERSPEECH

JF - Proceedings of the Annual Conference of the International Speech Communication Association, INTERSPEECH

SN - 2308-457X

T2 - 18th Annual Conference of the International Speech Communication Association, INTERSPEECH 2017

Y2 - 20 August 2017 through 24 August 2017

ER -

Advances in joint CTC-attention based end-to-end speech recognition with a deep CNN encoder and RNN-LM

Abstract

Keywords

ASJC Scopus subject areas

Access to Document

Other files and links

Fingerprint

Cite this