Autoencoder based multi-stream combination for noise robust speech recognition

Sri Harish Mallidi, Tetsuji Ogawa, Karel Vesely, Phani S. Nidadavolu, Hynek Hermansky

    Research output: Chapter in Book/Report/Conference proceedingConference contribution

    14 Citations (Scopus)

    Abstract

    Performances of automatic speech recognition (ASR) systems degrade rapidly when there is a mismatch between train and test acoustic conditions. Performance can be improved using a multi-stream framework, which involves combining posterior probabilities from several classifiers (often deep neural networks (DNNs)) trained on different features/streams. Knowledge about the confidence of each of these classifiers on a noisy test utterance can help in devising better techniques for posterior combination than simple sum and product rules [1]. In this work, we propose to use autoencoders which are multilayer feed forward neural networks, for estimating this confidence measure. During the training phase, for each stream, an autocoder is trained on TANDEM features extracted from the corresponding DNN. On employing the autoencoder during the testing phase, we show that the reconstruction error of the autoencoder is correlated to the robustness of the corresponding stream. These error estimates are then used as confidence measures to combine the posterior probabilities generated from each of the streams. Experiments on Aurora4 and BABEL databases indicate significant improvements, especially in the scenario of mismatch between train and test acoustic conditions.

    Original languageEnglish
    Title of host publicationProceedings of the Annual Conference of the International Speech Communication Association, INTERSPEECH
    PublisherInternational Speech and Communication Association
    Pages3551-3555
    Number of pages5
    Volume2015-January
    Publication statusPublished - 2015
    Event16th Annual Conference of the International Speech Communication Association, INTERSPEECH 2015 - Dresden, Germany
    Duration: 2015 Sep 62015 Sep 10

    Other

    Other16th Annual Conference of the International Speech Communication Association, INTERSPEECH 2015
    CountryGermany
    CityDresden
    Period15/9/615/9/10

    Keywords

    • Computational paralinguistics
    • Human-computer interaction
    • Speech recognition

    ASJC Scopus subject areas

    • Language and Linguistics
    • Human-Computer Interaction
    • Signal Processing
    • Software
    • Modelling and Simulation

    Fingerprint Dive into the research topics of 'Autoencoder based multi-stream combination for noise robust speech recognition'. Together they form a unique fingerprint.

  • Cite this

    Mallidi, S. H., Ogawa, T., Vesely, K., Nidadavolu, P. S., & Hermansky, H. (2015). Autoencoder based multi-stream combination for noise robust speech recognition. In Proceedings of the Annual Conference of the International Speech Communication Association, INTERSPEECH (Vol. 2015-January, pp. 3551-3555). International Speech and Communication Association.