The α-EM algorithm: Surrogate likelihood maximization using α-logarithmic information measures

Yasuo Matsuyama

    Research output: Contribution to journalArticle

    36 Citations (Scopus)

    Abstract

    A new likelihood maximization algorithm called the α-EM algorithm (α-Expectation-Maximization algorithm) is presented. This algorithm outperforms the traditional or logarithmic EM algorithm in terms of convergence speed for an appropriate range of the design parameter α. The log-EM algorithm is a special case corresponding to α = -1. The main idea behind the α-EM algorithm is to search for an effective surrogate function or a minorizer for the maximization of the observed data's likelihood ratio. The surrogate function adopted in this paper is based upon the α-logarithm which is related to the convex divergence. The convergence speed of the α-EM algorithm is theoretically analyzed through α-dependent update matrices and illustrated by numerical simulations. Finally, general guidelines for using the α-logarithmic methods are given. The choice of alternative surrogate functions is also discussed.

    Original languageEnglish
    Pages (from-to)692-706
    Number of pages15
    JournalIEEE Transactions on Information Theory
    Volume49
    Issue number3
    DOIs
    Publication statusPublished - 2003 Mar

    Fingerprint

    divergence
    simulation
    Computer simulation

    Keywords

    • α-EM algorithm
    • α-logarithm
    • Convergence speed
    • Convex divergence
    • Exponential family
    • Independent component analysis
    • Minorization-maximization
    • Supervised and unsupervised learning
    • Surrogate function
    • Vector quantization

    ASJC Scopus subject areas

    • Information Systems
    • Electrical and Electronic Engineering

    Cite this

    The α-EM algorithm : Surrogate likelihood maximization using α-logarithmic information measures. / Matsuyama, Yasuo.

    In: IEEE Transactions on Information Theory, Vol. 49, No. 3, 03.2003, p. 692-706.

    Research output: Contribution to journalArticle

    @article{59b124ff0ffe4f08814d8f6dc19ea80f,
    title = "The α-EM algorithm: Surrogate likelihood maximization using α-logarithmic information measures",
    abstract = "A new likelihood maximization algorithm called the α-EM algorithm (α-Expectation-Maximization algorithm) is presented. This algorithm outperforms the traditional or logarithmic EM algorithm in terms of convergence speed for an appropriate range of the design parameter α. The log-EM algorithm is a special case corresponding to α = -1. The main idea behind the α-EM algorithm is to search for an effective surrogate function or a minorizer for the maximization of the observed data's likelihood ratio. The surrogate function adopted in this paper is based upon the α-logarithm which is related to the convex divergence. The convergence speed of the α-EM algorithm is theoretically analyzed through α-dependent update matrices and illustrated by numerical simulations. Finally, general guidelines for using the α-logarithmic methods are given. The choice of alternative surrogate functions is also discussed.",
    keywords = "α-EM algorithm, α-logarithm, Convergence speed, Convex divergence, Exponential family, Independent component analysis, Minorization-maximization, Supervised and unsupervised learning, Surrogate function, Vector quantization",
    author = "Yasuo Matsuyama",
    year = "2003",
    month = "3",
    doi = "10.1109/TIT.2002.808105",
    language = "English",
    volume = "49",
    pages = "692--706",
    journal = "IEEE Transactions on Information Theory",
    issn = "0018-9448",
    publisher = "Institute of Electrical and Electronics Engineers Inc.",
    number = "3",

    }

    TY - JOUR

    T1 - The α-EM algorithm

    T2 - Surrogate likelihood maximization using α-logarithmic information measures

    AU - Matsuyama, Yasuo

    PY - 2003/3

    Y1 - 2003/3

    N2 - A new likelihood maximization algorithm called the α-EM algorithm (α-Expectation-Maximization algorithm) is presented. This algorithm outperforms the traditional or logarithmic EM algorithm in terms of convergence speed for an appropriate range of the design parameter α. The log-EM algorithm is a special case corresponding to α = -1. The main idea behind the α-EM algorithm is to search for an effective surrogate function or a minorizer for the maximization of the observed data's likelihood ratio. The surrogate function adopted in this paper is based upon the α-logarithm which is related to the convex divergence. The convergence speed of the α-EM algorithm is theoretically analyzed through α-dependent update matrices and illustrated by numerical simulations. Finally, general guidelines for using the α-logarithmic methods are given. The choice of alternative surrogate functions is also discussed.

    AB - A new likelihood maximization algorithm called the α-EM algorithm (α-Expectation-Maximization algorithm) is presented. This algorithm outperforms the traditional or logarithmic EM algorithm in terms of convergence speed for an appropriate range of the design parameter α. The log-EM algorithm is a special case corresponding to α = -1. The main idea behind the α-EM algorithm is to search for an effective surrogate function or a minorizer for the maximization of the observed data's likelihood ratio. The surrogate function adopted in this paper is based upon the α-logarithm which is related to the convex divergence. The convergence speed of the α-EM algorithm is theoretically analyzed through α-dependent update matrices and illustrated by numerical simulations. Finally, general guidelines for using the α-logarithmic methods are given. The choice of alternative surrogate functions is also discussed.

    KW - α-EM algorithm

    KW - α-logarithm

    KW - Convergence speed

    KW - Convex divergence

    KW - Exponential family

    KW - Independent component analysis

    KW - Minorization-maximization

    KW - Supervised and unsupervised learning

    KW - Surrogate function

    KW - Vector quantization

    UR - http://www.scopus.com/inward/record.url?scp=0037355593&partnerID=8YFLogxK

    UR - http://www.scopus.com/inward/citedby.url?scp=0037355593&partnerID=8YFLogxK

    U2 - 10.1109/TIT.2002.808105

    DO - 10.1109/TIT.2002.808105

    M3 - Article

    AN - SCOPUS:0037355593

    VL - 49

    SP - 692

    EP - 706

    JO - IEEE Transactions on Information Theory

    JF - IEEE Transactions on Information Theory

    SN - 0018-9448

    IS - 3

    ER -