Nonparametric e-mixture estimation

Ken Takano, Hideitsu Hino, Shotaro Akaho, Noboru Murata

    Research output: Contribution to journalLetter

    2 Citations (Scopus)

    Abstract

    This study considers the common situation in data analysis when there are few observations of the distribution of interest or the target distribution, while abundant observations are available from auxiliary distributions. In this situation, it is natural to compensate for the lack of data from the target distribution by using data sets from these auxiliary distributions-in other words, approximating the target distribution in a subspace spanned by a set of auxiliary distributions. Mixture modeling is one of the simplest ways to integrate information from the target and auxiliary distributions in order to express the target distribution as accurately as possible. There are two typical mixtures in the context of information geometry: the m- and e-mixtures. The m-mixture is applied in a variety of research fields because of the presence of the well-known expectation-maximazation algorithm for parameter estimation, whereas the e-mixture is rarely used because of its difficulty of estimation, particularly for nonparametric models. The e-mixture, however, is a welltempered distribution that satisfies the principle of maximum entropy. To model a target distribution with scarce observations accurately, this letter proposes a novel framework for a nonparametric modeling of the emixture and a geometrically inspired estimation algorithm. As numerical examples of the proposed framework, a transfer learning setup is considered. The experimental results show that this framework works well for three types of synthetic data sets, as well as an EEG real-world data set.

    Original languageEnglish
    Pages (from-to)2687-2725
    Number of pages39
    JournalNeural Computation
    Volume28
    Issue number12
    DOIs
    Publication statusPublished - 2016 Dec 1

    Fingerprint

    Entropy
    Electroencephalography
    Research
    Datasets
    Transfer (Psychology)
    Modeling

    ASJC Scopus subject areas

    • Arts and Humanities (miscellaneous)
    • Cognitive Neuroscience

    Cite this

    Nonparametric e-mixture estimation. / Takano, Ken; Hino, Hideitsu; Akaho, Shotaro; Murata, Noboru.

    In: Neural Computation, Vol. 28, No. 12, 01.12.2016, p. 2687-2725.

    Research output: Contribution to journalLetter

    Takano, K, Hino, H, Akaho, S & Murata, N 2016, 'Nonparametric e-mixture estimation', Neural Computation, vol. 28, no. 12, pp. 2687-2725. https://doi.org/10.1162/NECO_a_00888
    Takano, Ken ; Hino, Hideitsu ; Akaho, Shotaro ; Murata, Noboru. / Nonparametric e-mixture estimation. In: Neural Computation. 2016 ; Vol. 28, No. 12. pp. 2687-2725.
    @article{0c02ec9d920644a9a6f5763919afc366,
    title = "Nonparametric e-mixture estimation",
    abstract = "This study considers the common situation in data analysis when there are few observations of the distribution of interest or the target distribution, while abundant observations are available from auxiliary distributions. In this situation, it is natural to compensate for the lack of data from the target distribution by using data sets from these auxiliary distributions-in other words, approximating the target distribution in a subspace spanned by a set of auxiliary distributions. Mixture modeling is one of the simplest ways to integrate information from the target and auxiliary distributions in order to express the target distribution as accurately as possible. There are two typical mixtures in the context of information geometry: the m- and e-mixtures. The m-mixture is applied in a variety of research fields because of the presence of the well-known expectation-maximazation algorithm for parameter estimation, whereas the e-mixture is rarely used because of its difficulty of estimation, particularly for nonparametric models. The e-mixture, however, is a welltempered distribution that satisfies the principle of maximum entropy. To model a target distribution with scarce observations accurately, this letter proposes a novel framework for a nonparametric modeling of the emixture and a geometrically inspired estimation algorithm. As numerical examples of the proposed framework, a transfer learning setup is considered. The experimental results show that this framework works well for three types of synthetic data sets, as well as an EEG real-world data set.",
    author = "Ken Takano and Hideitsu Hino and Shotaro Akaho and Noboru Murata",
    year = "2016",
    month = "12",
    day = "1",
    doi = "10.1162/NECO_a_00888",
    language = "English",
    volume = "28",
    pages = "2687--2725",
    journal = "Neural Computation",
    issn = "0899-7667",
    publisher = "MIT Press Journals",
    number = "12",

    }

    TY - JOUR

    T1 - Nonparametric e-mixture estimation

    AU - Takano, Ken

    AU - Hino, Hideitsu

    AU - Akaho, Shotaro

    AU - Murata, Noboru

    PY - 2016/12/1

    Y1 - 2016/12/1

    N2 - This study considers the common situation in data analysis when there are few observations of the distribution of interest or the target distribution, while abundant observations are available from auxiliary distributions. In this situation, it is natural to compensate for the lack of data from the target distribution by using data sets from these auxiliary distributions-in other words, approximating the target distribution in a subspace spanned by a set of auxiliary distributions. Mixture modeling is one of the simplest ways to integrate information from the target and auxiliary distributions in order to express the target distribution as accurately as possible. There are two typical mixtures in the context of information geometry: the m- and e-mixtures. The m-mixture is applied in a variety of research fields because of the presence of the well-known expectation-maximazation algorithm for parameter estimation, whereas the e-mixture is rarely used because of its difficulty of estimation, particularly for nonparametric models. The e-mixture, however, is a welltempered distribution that satisfies the principle of maximum entropy. To model a target distribution with scarce observations accurately, this letter proposes a novel framework for a nonparametric modeling of the emixture and a geometrically inspired estimation algorithm. As numerical examples of the proposed framework, a transfer learning setup is considered. The experimental results show that this framework works well for three types of synthetic data sets, as well as an EEG real-world data set.

    AB - This study considers the common situation in data analysis when there are few observations of the distribution of interest or the target distribution, while abundant observations are available from auxiliary distributions. In this situation, it is natural to compensate for the lack of data from the target distribution by using data sets from these auxiliary distributions-in other words, approximating the target distribution in a subspace spanned by a set of auxiliary distributions. Mixture modeling is one of the simplest ways to integrate information from the target and auxiliary distributions in order to express the target distribution as accurately as possible. There are two typical mixtures in the context of information geometry: the m- and e-mixtures. The m-mixture is applied in a variety of research fields because of the presence of the well-known expectation-maximazation algorithm for parameter estimation, whereas the e-mixture is rarely used because of its difficulty of estimation, particularly for nonparametric models. The e-mixture, however, is a welltempered distribution that satisfies the principle of maximum entropy. To model a target distribution with scarce observations accurately, this letter proposes a novel framework for a nonparametric modeling of the emixture and a geometrically inspired estimation algorithm. As numerical examples of the proposed framework, a transfer learning setup is considered. The experimental results show that this framework works well for three types of synthetic data sets, as well as an EEG real-world data set.

    UR - http://www.scopus.com/inward/record.url?scp=84997769361&partnerID=8YFLogxK

    UR - http://www.scopus.com/inward/citedby.url?scp=84997769361&partnerID=8YFLogxK

    U2 - 10.1162/NECO_a_00888

    DO - 10.1162/NECO_a_00888

    M3 - Letter

    AN - SCOPUS:84997769361

    VL - 28

    SP - 2687

    EP - 2725

    JO - Neural Computation

    JF - Neural Computation

    SN - 0899-7667

    IS - 12

    ER -