Automated structure discovery and parameter tuning of neural network language model based on evolution strategy

Tomohiro Tanaka, Takafumi Moriya, Takahiro Shinozaki, Shinji Watanabe, Takaaki Hori, Kevin Duh

研究成果: Conference contribution

8 被引用数 (Scopus)

抄録

Long short-term memory (LSTM) recurrent neural network based language models are known to improve speech recognition performance. However, significant effort is required to optimize network structures and training configurations. In this study, we automate the development process using evolutionary algorithms. In particular, we apply the covariance matrix adaptation-evolution strategy (CMA-ES), which has demonstrated robustness in other black box hyper-parameter optimization problems. By flexibly allowing optimization of various meta-parameters including layer wise unit types, our method automatically finds a configuration that gives improved recognition performance. Further, by using a Pareto based multi-objective CMA-ES, both WER and computational time were reduced jointly: after 10 generations, relative WER and computational time reductions for decoding were 4.1% and 22.7% respectively, compared to an initial baseline system whose WER was 8.7%.

本文言語English
ホスト出版物のタイトル2016 IEEE Workshop on Spoken Language Technology, SLT 2016 - Proceedings
出版社Institute of Electrical and Electronics Engineers Inc.
ページ665-671
ページ数7
ISBN(電子版)9781509049035
DOI
出版ステータスPublished - 2017 2 7
外部発表はい
イベント2016 IEEE Workshop on Spoken Language Technology, SLT 2016 - San Diego, United States
継続期間: 2016 12 132016 12 16

出版物シリーズ

名前2016 IEEE Workshop on Spoken Language Technology, SLT 2016 - Proceedings

Other

Other2016 IEEE Workshop on Spoken Language Technology, SLT 2016
国/地域United States
CitySan Diego
Period16/12/1316/12/16

ASJC Scopus subject areas

  • 人間とコンピュータの相互作用
  • 人工知能
  • 言語および言語学
  • コンピュータ ビジョンおよびパターン認識
  • コンピュータ サイエンスの応用

フィンガープリント

「Automated structure discovery and parameter tuning of neural network language model based on evolution strategy」の研究トピックを掘り下げます。これらがまとまってユニークなフィンガープリントを構成します。

引用スタイル