End-to-End Integration of Speech Recognition, Speech Enhancement, and Self-Supervised Learning Representation

Xuankai Chang, Takashi Maekaku, Yuya Fujita, Shinji Watanabe

Research output: Contribution to journalConference articlepeer-review

Abstract

This work presents our end-to-end (E2E) automatic speech recognition (ASR) model targetting at robust speech recognition, called Integraded speech Recognition with enhanced speech Input for Self-supervised learning representation (IRIS). Compared with conventional E2E ASR models, the proposed E2E model integrates two important modules including a speech enhancement (SE) module and a self-supervised learning representation (SSLR) module. The SE module enhances the noisy speech. Then the SSLR module extracts features from enhanced speech to be used for speech recognition (ASR). To train the proposed model, we establish an efficient learning scheme. Evaluation results on the monaural CHiME-4 task show that the IRIS model achieves the best performance reported in the literature for the single-channel CHiME-4 benchmark (2.0% for the real development and 3.6% for the real test) thanks to the powerful pre-trained SSLR module and the fine-tuned SE module.

Original languageEnglish
Pages (from-to)3819-3823
Number of pages5
JournalProceedings of the Annual Conference of the International Speech Communication Association, INTERSPEECH
Volume2022-September
DOIs
Publication statusPublished - 2022
Externally publishedYes
Event23rd Annual Conference of the International Speech Communication Association, INTERSPEECH 2022 - Incheon, Korea, Republic of
Duration: 2022 Sep 182022 Sep 22

Keywords

  • deep learning
  • robust automatic speech recognition
  • self-supervised learning
  • speech enhancement

ASJC Scopus subject areas

  • Language and Linguistics
  • Human-Computer Interaction
  • Signal Processing
  • Software
  • Modelling and Simulation

Fingerprint

Dive into the research topics of 'End-to-End Integration of Speech Recognition, Speech Enhancement, and Self-Supervised Learning Representation'. Together they form a unique fingerprint.

Cite this