Fast-MD: Fast Multi-Decoder End-to-End Speech Translation with Non-Autoregressive Hidden Intermediates

Hirofumi Inaguma, Siddharth Dalmia, Brian Yan, Shinji Watanabe

研究成果: Conference contribution

1 被引用数 (Scopus)

抄録

The multi-decoder (MD) end-to-end speech translation model has demonstrated high translation quality by searching for better intermediate automatic speech recognition (ASR) decoder states as hidden intermediates (HI). It is a two-pass decoding model decomposing the overall task into ASR and machine translation sub-tasks. However, the decoding speed is not fast enough for real-world applications because it conducts beam search for both sub-tasks during inference. We propose Fast-MD, a fast MD model that generates HI by non-autoregressive (NAR) decoding based on connectionist temporal classification (CTC) outputs followed by an ASR decoder. We investigated two types of NAR HI: (1) parallel HI by using an autoregressive Transformer ASR decoder and (2) masked HI by using Mask-CTC, which combines CTC and the conditional masked language model. To reduce a mismatch in the ASR decoder between teacher-forcing during training and conditioning on CTC outputs during testing, we also propose sampling CTC outputs during training. Experimental evaluations on three corpora show that Fast-MD achieved about 2× and 4× faster decoding speed than that of the naïve MD model on GPU and CPU with comparable translation quality. Adopting the Conformer encoder and intermediate CTC loss further boosts its quality without sacrificing decoding speed.

本文言語English
ホスト出版物のタイトル2021 IEEE Automatic Speech Recognition and Understanding Workshop, ASRU 2021 - Proceedings
出版社Institute of Electrical and Electronics Engineers Inc.
ページ922-929
ページ数8
ISBN(電子版)9781665437394
DOI
出版ステータスPublished - 2021
外部発表はい
イベント2021 IEEE Automatic Speech Recognition and Understanding Workshop, ASRU 2021 - Cartagena, Colombia
継続期間: 2021 12月 132021 12月 17

出版物シリーズ

名前2021 IEEE Automatic Speech Recognition and Understanding Workshop, ASRU 2021 - Proceedings

Conference

Conference2021 IEEE Automatic Speech Recognition and Understanding Workshop, ASRU 2021
国/地域Colombia
CityCartagena
Period21/12/1321/12/17

ASJC Scopus subject areas

  • コンピュータ ビジョンおよびパターン認識
  • 信号処理
  • 言語学および言語

フィンガープリント

「Fast-MD: Fast Multi-Decoder End-to-End Speech Translation with Non-Autoregressive Hidden Intermediates」の研究トピックを掘り下げます。これらがまとまってユニークなフィンガープリントを構成します。

引用スタイル