Self-Guided Curriculum Learning for Neural Machine Translation

Lei Zhou*, Liang Ding, Kevin Duh, Shinji Watanabe, Ryohei Sasano, Koichi Takeda

*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceedingConference contribution

2 Citations (Scopus)

Abstract

In supervised learning, a well-trained model should be able to recover ground truth accurately, i.e. the predicted labels are expected to resemble the ground truth labels as much as possible. Inspired by this, we formulate a difficulty criterion based on the recovery degrees of training examples. Motivated by the intuition that after skimming through the training corpus, the neural machine translation (NMT) model “knows” how to schedule a suitable curriculum according to learning difficulty, we propose a self-guided curriculum learning strategy that encourages the NMT model to learn from easy to hard on the basis of recovery degrees. Specifically, we adopt sentence-level BLEU score as the proxy of recovery degree. Experimental results on translation benchmarks including WMT14 English?German and WMT17 Chinese?English demonstrate that our proposed method considerably improves the recovery degree, thus consistently improving the translation performance.

Original languageEnglish
Title of host publicationIWSLT 2021 - 18th International Conference on Spoken Language Translation, Proceedings
PublisherAssociation for Computational Linguistics (ACL)
Pages206-214
Number of pages9
ISBN (Electronic)9781954085749
Publication statusPublished - 2021
Externally publishedYes
Event18th International Conference on Spoken Language Translation, IWSLT 2021 - Virtual, Bangkok, Thailand
Duration: 2021 Aug 52021 Aug 6

Publication series

NameIWSLT 2021 - 18th International Conference on Spoken Language Translation, Proceedings

Conference

Conference18th International Conference on Spoken Language Translation, IWSLT 2021
Country/TerritoryThailand
CityVirtual, Bangkok
Period21/8/521/8/6

ASJC Scopus subject areas

  • Human-Computer Interaction
  • Language and Linguistics
  • Linguistics and Language

Fingerprint

Dive into the research topics of 'Self-Guided Curriculum Learning for Neural Machine Translation'. Together they form a unique fingerprint.

Cite this