Uncertainty estimation of DNN classifiers

Sri Harish Mallidi, Tetsuji Ogawa, Hynek Hermansky

    Research output: Chapter in Book/Report/Conference proceedingConference contribution

    16 Citations (Scopus)

    Abstract

    New efficient measures for estimating uncertainty of deep neural network (DNN) classifiers are proposed and successfully applied to multistream-based unsupervised adaptation of ASR systems to address uncertainty derived from noise. The proposed measure is the error from associative memory models trained on outputs of a DNN. In the present study, an attempt is made to use autoencoders for remembering the property of data. Another measure proposed is an extension of the M-measure, which computes the divergences of probability estimates spaced at specific time intervals. The extended measure results in an improved reliability by considering the latent information of phoneme duration. Experimental comparisons carried out in a multistream-based ASR paradigm demonstrates that the proposed measures yielded improvements over the multistyle trained system and system selected based on existing measures. Fusion of the proposed measures achieved almost the same performance as the oracle system selection.

    Original languageEnglish
    Title of host publication2015 IEEE Workshop on Automatic Speech Recognition and Understanding, ASRU 2015 - Proceedings
    PublisherInstitute of Electrical and Electronics Engineers Inc.
    Pages283-288
    Number of pages6
    ISBN (Print)9781479972913
    DOIs
    Publication statusPublished - 2016 Feb 10
    EventIEEE Workshop on Automatic Speech Recognition and Understanding, ASRU 2015 - Scottsdale, United States
    Duration: 2015 Dec 132015 Dec 17

    Other

    OtherIEEE Workshop on Automatic Speech Recognition and Understanding, ASRU 2015
    CountryUnited States
    CityScottsdale
    Period15/12/1315/12/17

    Fingerprint

    Classifiers
    Fusion reactions
    Data storage equipment
    Deep neural networks
    Uncertainty

    Keywords

    • Autoencoder
    • deep neural networks
    • M-delta measure
    • multistream ASR
    • uncertainty estimation

    ASJC Scopus subject areas

    • Artificial Intelligence
    • Computer Networks and Communications
    • Computer Vision and Pattern Recognition

    Cite this

    Mallidi, S. H., Ogawa, T., & Hermansky, H. (2016). Uncertainty estimation of DNN classifiers. In 2015 IEEE Workshop on Automatic Speech Recognition and Understanding, ASRU 2015 - Proceedings (pp. 283-288). [7404806] Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/ASRU.2015.7404806

    Uncertainty estimation of DNN classifiers. / Mallidi, Sri Harish; Ogawa, Tetsuji; Hermansky, Hynek.

    2015 IEEE Workshop on Automatic Speech Recognition and Understanding, ASRU 2015 - Proceedings. Institute of Electrical and Electronics Engineers Inc., 2016. p. 283-288 7404806.

    Research output: Chapter in Book/Report/Conference proceedingConference contribution

    Mallidi, SH, Ogawa, T & Hermansky, H 2016, Uncertainty estimation of DNN classifiers. in 2015 IEEE Workshop on Automatic Speech Recognition and Understanding, ASRU 2015 - Proceedings., 7404806, Institute of Electrical and Electronics Engineers Inc., pp. 283-288, IEEE Workshop on Automatic Speech Recognition and Understanding, ASRU 2015, Scottsdale, United States, 15/12/13. https://doi.org/10.1109/ASRU.2015.7404806
    Mallidi SH, Ogawa T, Hermansky H. Uncertainty estimation of DNN classifiers. In 2015 IEEE Workshop on Automatic Speech Recognition and Understanding, ASRU 2015 - Proceedings. Institute of Electrical and Electronics Engineers Inc. 2016. p. 283-288. 7404806 https://doi.org/10.1109/ASRU.2015.7404806
    Mallidi, Sri Harish ; Ogawa, Tetsuji ; Hermansky, Hynek. / Uncertainty estimation of DNN classifiers. 2015 IEEE Workshop on Automatic Speech Recognition and Understanding, ASRU 2015 - Proceedings. Institute of Electrical and Electronics Engineers Inc., 2016. pp. 283-288
    @inproceedings{5b92965af8ab4dbbae8278a741a10f17,
    title = "Uncertainty estimation of DNN classifiers",
    abstract = "New efficient measures for estimating uncertainty of deep neural network (DNN) classifiers are proposed and successfully applied to multistream-based unsupervised adaptation of ASR systems to address uncertainty derived from noise. The proposed measure is the error from associative memory models trained on outputs of a DNN. In the present study, an attempt is made to use autoencoders for remembering the property of data. Another measure proposed is an extension of the M-measure, which computes the divergences of probability estimates spaced at specific time intervals. The extended measure results in an improved reliability by considering the latent information of phoneme duration. Experimental comparisons carried out in a multistream-based ASR paradigm demonstrates that the proposed measures yielded improvements over the multistyle trained system and system selected based on existing measures. Fusion of the proposed measures achieved almost the same performance as the oracle system selection.",
    keywords = "Autoencoder, deep neural networks, M-delta measure, multistream ASR, uncertainty estimation",
    author = "Mallidi, {Sri Harish} and Tetsuji Ogawa and Hynek Hermansky",
    year = "2016",
    month = "2",
    day = "10",
    doi = "10.1109/ASRU.2015.7404806",
    language = "English",
    isbn = "9781479972913",
    pages = "283--288",
    booktitle = "2015 IEEE Workshop on Automatic Speech Recognition and Understanding, ASRU 2015 - Proceedings",
    publisher = "Institute of Electrical and Electronics Engineers Inc.",

    }

    TY - GEN

    T1 - Uncertainty estimation of DNN classifiers

    AU - Mallidi, Sri Harish

    AU - Ogawa, Tetsuji

    AU - Hermansky, Hynek

    PY - 2016/2/10

    Y1 - 2016/2/10

    N2 - New efficient measures for estimating uncertainty of deep neural network (DNN) classifiers are proposed and successfully applied to multistream-based unsupervised adaptation of ASR systems to address uncertainty derived from noise. The proposed measure is the error from associative memory models trained on outputs of a DNN. In the present study, an attempt is made to use autoencoders for remembering the property of data. Another measure proposed is an extension of the M-measure, which computes the divergences of probability estimates spaced at specific time intervals. The extended measure results in an improved reliability by considering the latent information of phoneme duration. Experimental comparisons carried out in a multistream-based ASR paradigm demonstrates that the proposed measures yielded improvements over the multistyle trained system and system selected based on existing measures. Fusion of the proposed measures achieved almost the same performance as the oracle system selection.

    AB - New efficient measures for estimating uncertainty of deep neural network (DNN) classifiers are proposed and successfully applied to multistream-based unsupervised adaptation of ASR systems to address uncertainty derived from noise. The proposed measure is the error from associative memory models trained on outputs of a DNN. In the present study, an attempt is made to use autoencoders for remembering the property of data. Another measure proposed is an extension of the M-measure, which computes the divergences of probability estimates spaced at specific time intervals. The extended measure results in an improved reliability by considering the latent information of phoneme duration. Experimental comparisons carried out in a multistream-based ASR paradigm demonstrates that the proposed measures yielded improvements over the multistyle trained system and system selected based on existing measures. Fusion of the proposed measures achieved almost the same performance as the oracle system selection.

    KW - Autoencoder

    KW - deep neural networks

    KW - M-delta measure

    KW - multistream ASR

    KW - uncertainty estimation

    UR - http://www.scopus.com/inward/record.url?scp=84964499350&partnerID=8YFLogxK

    UR - http://www.scopus.com/inward/citedby.url?scp=84964499350&partnerID=8YFLogxK

    U2 - 10.1109/ASRU.2015.7404806

    DO - 10.1109/ASRU.2015.7404806

    M3 - Conference contribution

    SN - 9781479972913

    SP - 283

    EP - 288

    BT - 2015 IEEE Workshop on Automatic Speech Recognition and Understanding, ASRU 2015 - Proceedings

    PB - Institute of Electrical and Electronics Engineers Inc.

    ER -