TY - JOUR

T1 - Geometry of EM and related iterative algorithms

AU - Hino, Hideitsu

AU - Akaho, Shotaro

AU - Murata, Noboru

N1 - Funding Information:
Part of this work is supported by JSPS KAKENHI No.JP22H03653 and JP22486199.
Publisher Copyright:
© 2022, The Author(s), under exclusive licence to Springer Nature Singapore Pte Ltd.

PY - 2022

Y1 - 2022

N2 - The Expectation–Maximization (EM) algorithm is a simple meta-algorithm that has been used for many years as a methodology for statistical inference when there are missing measurements in the observed data or when the data is composed of observables and unobservables. Its general properties are well studied, and also, there are countless ways to apply it to individual problems. In this paper, we introduce the em algorithm, an information geometric formulation of the EM algorithm, and its extensions and applications to various problems. Specifically, we will see that it is possible to formulate an outlier–robust inference algorithm, an algorithm for calculating channel capacity, parameter estimation methods on probability simplex, particular multivariate analysis methods such as principal component analysis in a space of probability models and modal regression, matrix factorization, and learning generative models, which have recently attracted attention in deep learning, from the geometric perspective provided by Amari.

AB - The Expectation–Maximization (EM) algorithm is a simple meta-algorithm that has been used for many years as a methodology for statistical inference when there are missing measurements in the observed data or when the data is composed of observables and unobservables. Its general properties are well studied, and also, there are countless ways to apply it to individual problems. In this paper, we introduce the em algorithm, an information geometric formulation of the EM algorithm, and its extensions and applications to various problems. Specifically, we will see that it is possible to formulate an outlier–robust inference algorithm, an algorithm for calculating channel capacity, parameter estimation methods on probability simplex, particular multivariate analysis methods such as principal component analysis in a space of probability models and modal regression, matrix factorization, and learning generative models, which have recently attracted attention in deep learning, from the geometric perspective provided by Amari.

KW - Bregman divergence

KW - EM algorithm

KW - Generative models

KW - Information geometry

KW - Information theory

KW - Robust statistics

KW - em algorithm

UR - http://www.scopus.com/inward/record.url?scp=85142249708&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85142249708&partnerID=8YFLogxK

U2 - 10.1007/s41884-022-00080-y

DO - 10.1007/s41884-022-00080-y

M3 - Article

AN - SCOPUS:85142249708

JO - Information Geometry

JF - Information Geometry

SN - 2511-2481

ER -