TY - GEN
T1 - Model selection of bayesian hierarchical mixture of experts based on variational inference
AU - Iikubo, Yuji
AU - Horii, Shunsuke
AU - Matsushima, Toshiyasu
PY - 2019/10
Y1 - 2019/10
N2 - We consider the model selection of the hierarchical mixture of experts (HME). The HME is a tree-structured probabilistic model for regression and classification. The HME model has high prediction accuracy and high interpretability, however, the estimation of the parameters tends to overfit due to the complexity of the model. In order to mitigate the overfitting problem, in previous studies, several Bayesian estimation methods for the HME parameters have been proposed. In these studies, the true model that generates data is fixed. In general, however, the true model is unknown. Model selection is one of the most important and difficult problems of regression and classification. For the Bayesian HME, the model is determined by the tree structure, the form of the prior distribution and its parameters, however, only the tree structure is considered as a model parameter in previous studies. In this paper, we consider all of these as model parameters and extend the model selection method. Then, we propose a maximum a posteriori (MAP) estimation method of the Bayesian HME model selection. The approximate posterior probability of each model is calculated by the variational lower bound. We show the effectiveness of the proposed method by numerical experiments and discuss the results applied to actual data sets.
AB - We consider the model selection of the hierarchical mixture of experts (HME). The HME is a tree-structured probabilistic model for regression and classification. The HME model has high prediction accuracy and high interpretability, however, the estimation of the parameters tends to overfit due to the complexity of the model. In order to mitigate the overfitting problem, in previous studies, several Bayesian estimation methods for the HME parameters have been proposed. In these studies, the true model that generates data is fixed. In general, however, the true model is unknown. Model selection is one of the most important and difficult problems of regression and classification. For the Bayesian HME, the model is determined by the tree structure, the form of the prior distribution and its parameters, however, only the tree structure is considered as a model parameter in previous studies. In this paper, we consider all of these as model parameters and extend the model selection method. Then, we propose a maximum a posteriori (MAP) estimation method of the Bayesian HME model selection. The approximate posterior probability of each model is calculated by the variational lower bound. We show the effectiveness of the proposed method by numerical experiments and discuss the results applied to actual data sets.
UR - http://www.scopus.com/inward/record.url?scp=85076744495&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85076744495&partnerID=8YFLogxK
U2 - 10.1109/SMC.2019.8913896
DO - 10.1109/SMC.2019.8913896
M3 - Conference contribution
AN - SCOPUS:85076744495
T3 - Conference Proceedings - IEEE International Conference on Systems, Man and Cybernetics
SP - 3474
EP - 3479
BT - 2019 IEEE International Conference on Systems, Man and Cybernetics, SMC 2019
PB - Institute of Electrical and Electronics Engineers Inc.
T2 - 2019 IEEE International Conference on Systems, Man and Cybernetics, SMC 2019
Y2 - 6 October 2019 through 9 October 2019
ER -