Sequence discriminative training for low-rank deep neural networks

Yuuki Tachioka, Shinji Watanabe, Jonathan Le Roux, John R. Hershey

研究成果: Conference contribution

1 被引用数 (Scopus)

抄録

Deep neural networks (DNNs) have proven very successful for automatic speech recognition but the number of parameters tends to be large, leading to high computational cost. To reduce the size of a DNN model, low-rank approximations of weight matrices, computed using singular value decomposition (SVD), have previously been applied. Previous studies only focused on clean speech, whereas the additional variability in noisy speech could make model reduction difficult. Thus we investigate the effectiveness of this SVD method on noisy reverberated speech. Furthermore, we combine the low-rank approximation with sequence discriminative training, which further improved the performance of the DNN, even though the original DNN was constructed using a discriminative criterion. We also investigated the effect of the order of application of the low-rank and sequence discriminative training. Our experiments show that low rank approximation is effective for noisy speech and the most effective combination of discriminative training with model reduction is to apply the low rank approximation to the base model first and then to perform discriminative training on the low-rank model. This low-rank discriminatively trained model outperformed the full discriminatively trained model.

本文言語English
ホスト出版物のタイトル2014 IEEE Global Conference on Signal and Information Processing, GlobalSIP 2014
出版社Institute of Electrical and Electronics Engineers Inc.
ページ572-576
ページ数5
ISBN(電子版)9781479970889
DOI
出版ステータスPublished - 2014 2 5
外部発表はい
イベント2014 IEEE Global Conference on Signal and Information Processing, GlobalSIP 2014 - Atlanta, United States
継続期間: 2014 12 32014 12 5

Other

Other2014 IEEE Global Conference on Signal and Information Processing, GlobalSIP 2014
国/地域United States
CityAtlanta
Period14/12/314/12/5

ASJC Scopus subject areas

  • 信号処理
  • 情報システム

フィンガープリント

「Sequence discriminative training for low-rank deep neural networks」の研究トピックを掘り下げます。これらがまとまってユニークなフィンガープリントを構成します。

引用スタイル