Joint separation, dereverberation and classification of multiple sources using multichannel variational autoencoder with auxiliary classifier

Shota Inoue, Hirokazu Kameoka, Li Li, Shoji Makino

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

This paper proposes a unified approach to jointly solving the separation, dereverberation, and classification of mixed sound sources from microphone array observations. The proposed method uses a frequency-wise convolutive mixture model to express the mixing process under highly reverberant environments and the auxiliary classifier conditional variational autoencoder (ACVAE) to model the complex spectrograms of underlying sources. Using an ACVAE as the source generative model allows us to estimate the latent vectors and the class index of each source in a test mixture by computing the outputs of the pretrained approximate posterior inference networks without using backpropagation. We experimentally confirmed that the proposed method outperformed conventional methods in terms of both computation time and source classification.

Original languageEnglish
Title of host publicationProceedings of the 23rd International Congress on Acoustics
Subtitle of host publicationIntegrating 4th EAA Euroregio 2019
EditorsMartin Ochmann, Vorlander Michael, Janina Fels
PublisherInternational Commission for Acoustics (ICA)
Pages6953-6960
Number of pages8
ISBN (Electronic)9783939296157
DOIs
Publication statusPublished - 2019
Externally publishedYes
Event23rd International Congress on Acoustics: Integrating 4th EAA Euroregio, ICA 2019 - Aachen, Germany
Duration: 2019 Sep 92019 Sep 23

Publication series

NameProceedings of the International Congress on Acoustics
Volume2019-September
ISSN (Print)2226-7808
ISSN (Electronic)2415-1599

Conference

Conference23rd International Congress on Acoustics: Integrating 4th EAA Euroregio, ICA 2019
Country/TerritoryGermany
CityAachen
Period19/9/919/9/23

Keywords

  • Blind dereverberation
  • Multichannel audio signal processing
  • Multichannel variational autoencoder (MVAE)
  • Source separation

ASJC Scopus subject areas

  • Mechanical Engineering
  • Acoustics and Ultrasonics

Fingerprint

Dive into the research topics of 'Joint separation, dereverberation and classification of multiple sources using multichannel variational autoencoder with auxiliary classifier'. Together they form a unique fingerprint.

Cite this