Tight lower bound of generalization error in ensemble learning

Masato Uchida

doi:10.1109/SCIS-ISIS.2014.7044723

Tight lower bound of generalization error in ensemble learning

Masato Uchida^*

^*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Abstract

A machine learning method that integrates multiple component predictors into one predictor is referred to as ensemble learning. In this paper, we derive a tight lower bound of the generalization error in ensemble learning based on asymptotic analysis using a mathematical model based on an exponential mixture of probability distributions and the Kullback-Leibler divergence. In addition, we derive the explicit expression of the weight parameter used in the exponential mixture of probability distributions that minimizes the tight lower bound and discuss the property of the derived weight parameter.

Original language	English
Title of host publication	2014 Joint 7th International Conference on Soft Computing and Intelligent Systems, SCIS 2014 and 15th International Symposium on Advanced Intelligent Systems, ISIS 2014
Publisher	Institute of Electrical and Electronics Engineers Inc.
Pages	1130-1133
Number of pages	4
ISBN (Electronic)	9781479959556
DOIs	https://doi.org/10.1109/SCIS-ISIS.2014.7044723
Publication status	Published - 2014 Feb 18
Externally published	Yes
Event	2014 Joint 7th International Conference on Soft Computing and Intelligent Systems, SCIS 2014 and 15th International Symposium on Advanced Intelligent Systems, ISIS 2014 - Kitakyushu, Japan Duration: 2014 Dec 3 → 2014 Dec 6

Publication series

Name	2014 Joint 7th International Conference on Soft Computing and Intelligent Systems, SCIS 2014 and 15th International Symposium on Advanced Intelligent Systems, ISIS 2014

Other

Other	2014 Joint 7th International Conference on Soft Computing and Intelligent Systems, SCIS 2014 and 15th International Symposium on Advanced Intelligent Systems, ISIS 2014
Country/Territory	Japan
City	Kitakyushu
Period	14/12/3 → 14/12/6

Keywords

asymptotic analysis
ensemble learning
exponential mixture model
generalization error
parameter estimation

ASJC Scopus subject areas

Software
Artificial Intelligence

Access to Document

10.1109/SCIS-ISIS.2014.7044723

Cite this

Uchida, M. (2014). Tight lower bound of generalization error in ensemble learning. In 2014 Joint 7th International Conference on Soft Computing and Intelligent Systems, SCIS 2014 and 15th International Symposium on Advanced Intelligent Systems, ISIS 2014 (pp. 1130-1133). [7044723] (2014 Joint 7th International Conference on Soft Computing and Intelligent Systems, SCIS 2014 and 15th International Symposium on Advanced Intelligent Systems, ISIS 2014). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/SCIS-ISIS.2014.7044723

Tight lower bound of generalization error in ensemble learning. / Uchida, Masato.

2014 Joint 7th International Conference on Soft Computing and Intelligent Systems, SCIS 2014 and 15th International Symposium on Advanced Intelligent Systems, ISIS 2014. Institute of Electrical and Electronics Engineers Inc., 2014. p. 1130-1133 7044723 (2014 Joint 7th International Conference on Soft Computing and Intelligent Systems, SCIS 2014 and 15th International Symposium on Advanced Intelligent Systems, ISIS 2014).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Uchida, M 2014, Tight lower bound of generalization error in ensemble learning. in 2014 Joint 7th International Conference on Soft Computing and Intelligent Systems, SCIS 2014 and 15th International Symposium on Advanced Intelligent Systems, ISIS 2014., 7044723, 2014 Joint 7th International Conference on Soft Computing and Intelligent Systems, SCIS 2014 and 15th International Symposium on Advanced Intelligent Systems, ISIS 2014, Institute of Electrical and Electronics Engineers Inc., pp. 1130-1133, 2014 Joint 7th International Conference on Soft Computing and Intelligent Systems, SCIS 2014 and 15th International Symposium on Advanced Intelligent Systems, ISIS 2014, Kitakyushu, Japan, 14/12/3. https://doi.org/10.1109/SCIS-ISIS.2014.7044723

Uchida M. Tight lower bound of generalization error in ensemble learning. In 2014 Joint 7th International Conference on Soft Computing and Intelligent Systems, SCIS 2014 and 15th International Symposium on Advanced Intelligent Systems, ISIS 2014. Institute of Electrical and Electronics Engineers Inc. 2014. p. 1130-1133. 7044723. (2014 Joint 7th International Conference on Soft Computing and Intelligent Systems, SCIS 2014 and 15th International Symposium on Advanced Intelligent Systems, ISIS 2014). doi: 10.1109/SCIS-ISIS.2014.7044723

Uchida, Masato. / Tight lower bound of generalization error in ensemble learning. 2014 Joint 7th International Conference on Soft Computing and Intelligent Systems, SCIS 2014 and 15th International Symposium on Advanced Intelligent Systems, ISIS 2014. Institute of Electrical and Electronics Engineers Inc., 2014. pp. 1130-1133 (2014 Joint 7th International Conference on Soft Computing and Intelligent Systems, SCIS 2014 and 15th International Symposium on Advanced Intelligent Systems, ISIS 2014).

@inproceedings{e3c5b7ef186c4dcea803ce81dbc099e3,

title = "Tight lower bound of generalization error in ensemble learning",

abstract = "A machine learning method that integrates multiple component predictors into one predictor is referred to as ensemble learning. In this paper, we derive a tight lower bound of the generalization error in ensemble learning based on asymptotic analysis using a mathematical model based on an exponential mixture of probability distributions and the Kullback-Leibler divergence. In addition, we derive the explicit expression of the weight parameter used in the exponential mixture of probability distributions that minimizes the tight lower bound and discuss the property of the derived weight parameter.",

keywords = "asymptotic analysis, ensemble learning, exponential mixture model, generalization error, parameter estimation",

author = "Masato Uchida",

note = "Publisher Copyright: {\textcopyright} 2014 IEEE.; 2014 Joint 7th International Conference on Soft Computing and Intelligent Systems, SCIS 2014 and 15th International Symposium on Advanced Intelligent Systems, ISIS 2014 ; Conference date: 03-12-2014 Through 06-12-2014",

year = "2014",

month = feb,

day = "18",

doi = "10.1109/SCIS-ISIS.2014.7044723",

language = "English",

series = "2014 Joint 7th International Conference on Soft Computing and Intelligent Systems, SCIS 2014 and 15th International Symposium on Advanced Intelligent Systems, ISIS 2014",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

pages = "1130--1133",

booktitle = "2014 Joint 7th International Conference on Soft Computing and Intelligent Systems, SCIS 2014 and 15th International Symposium on Advanced Intelligent Systems, ISIS 2014",

}

TY - GEN

T1 - Tight lower bound of generalization error in ensemble learning

AU - Uchida, Masato

PY - 2014/2/18

Y1 - 2014/2/18

N2 - A machine learning method that integrates multiple component predictors into one predictor is referred to as ensemble learning. In this paper, we derive a tight lower bound of the generalization error in ensemble learning based on asymptotic analysis using a mathematical model based on an exponential mixture of probability distributions and the Kullback-Leibler divergence. In addition, we derive the explicit expression of the weight parameter used in the exponential mixture of probability distributions that minimizes the tight lower bound and discuss the property of the derived weight parameter.

AB - A machine learning method that integrates multiple component predictors into one predictor is referred to as ensemble learning. In this paper, we derive a tight lower bound of the generalization error in ensemble learning based on asymptotic analysis using a mathematical model based on an exponential mixture of probability distributions and the Kullback-Leibler divergence. In addition, we derive the explicit expression of the weight parameter used in the exponential mixture of probability distributions that minimizes the tight lower bound and discuss the property of the derived weight parameter.

KW - asymptotic analysis

KW - ensemble learning

KW - exponential mixture model

KW - generalization error

KW - parameter estimation

UR - http://www.scopus.com/inward/record.url?scp=84988248499&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84988248499&partnerID=8YFLogxK

U2 - 10.1109/SCIS-ISIS.2014.7044723

DO - 10.1109/SCIS-ISIS.2014.7044723

M3 - Conference contribution

AN - SCOPUS:84988248499

T3 - 2014 Joint 7th International Conference on Soft Computing and Intelligent Systems, SCIS 2014 and 15th International Symposium on Advanced Intelligent Systems, ISIS 2014

SP - 1130

EP - 1133

BT - 2014 Joint 7th International Conference on Soft Computing and Intelligent Systems, SCIS 2014 and 15th International Symposium on Advanced Intelligent Systems, ISIS 2014

PB - Institute of Electrical and Electronics Engineers Inc.

T2 - 2014 Joint 7th International Conference on Soft Computing and Intelligent Systems, SCIS 2014 and 15th International Symposium on Advanced Intelligent Systems, ISIS 2014

Y2 - 3 December 2014 through 6 December 2014

ER -

Tight lower bound of generalization error in ensemble learning

Abstract

Publication series

Other

Keywords

ASJC Scopus subject areas

Access to Document

Other files and links

Fingerprint

Cite this