Sparse Bayesian Hierarchical Mixture of Experts and Variational Inference

Research output: Chapter in Book/Report/Conference proceedingConference contribution

1 Citation (Scopus)

Abstract

The hierarchical mixture of experts (HME) is a tree-structured probabilistic model for regression and classification. The HME has a considerable expression capability, however, the estimation of the parameters tends to overfit due to the complexity of the model. To avoid this problem, regularization techniques are widely used. In particular, it is known that a sparse solution can be obtained by L1 regularization. From a Bayesian point of view, regularization techniques are equivalent to assume that the parameters follow prior distributions and find the maximum a posteriori probability estimator. It is known that L1 regularization is equivalent to assuming Laplace distributions as prior distributions. However, it is difficult to compute the posterior distribution if Laplace distributions are assumed. In this paper, we assume that the parameters of the HME follow hierarchical prior distributions which are equivalent to Laplace distribution to promote sparse solutions. We propose a Bayesian estimation algorithm based on the variational method. Finally, the proposed algorithm is evaluated by computer simulations.

Original languageEnglish
Title of host publicationProceedings of 2018 International Symposium on Information Theory and Its Applications, ISITA 2018
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages60-64
Number of pages5
ISBN (Electronic)9784885523182
DOIs
Publication statusPublished - 2019 Mar 8
Event15th International Symposium on Information Theory and Its Applications, ISITA 2018 - Singapore, Singapore
Duration: 2018 Oct 282018 Oct 31

Publication series

NameProceedings of 2018 International Symposium on Information Theory and Its Applications, ISITA 2018

Conference

Conference15th International Symposium on Information Theory and Its Applications, ISITA 2018
CountrySingapore
CitySingapore
Period18/10/2818/10/31

    Fingerprint

ASJC Scopus subject areas

  • Computer Science Applications
  • Information Systems

Cite this

Iikubo, Y., Horii, S., & Matsushima, T. (2019). Sparse Bayesian Hierarchical Mixture of Experts and Variational Inference. In Proceedings of 2018 International Symposium on Information Theory and Its Applications, ISITA 2018 (pp. 60-64). [8664333] (Proceedings of 2018 International Symposium on Information Theory and Its Applications, ISITA 2018). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.23919/ISITA.2018.8664333