α-EM algorithm and α-ICA learning based upon extended logarithmic information measures

Yasuo Matsuyama, Takeshi Niimoto, Naoto Katsumata, Yoshitaka Suzuki, Satoshi Furukawa

    Research output: Chapter in Book/Report/Conference proceedingConference contribution

    7 Citations (Scopus)

    Abstract

    The α-logarithm extends the logarithm as the special case of α = -1. Usage of α-related information measures based upon this extended logarithm is expected to be effective to speedup of convergence, i.e., on the improvement of learning aptitude. In this paper, two typical cases are investigated. One is the α-EM algorithm (α-Expectation-Maximization algorithm) which is derived from the α-log-likelihood ratio. The other is the α-ICA (α-Independent Component Analysis) which is formulated as minimizing the α-mutual information. In the derivation of both algorithms, the α-divergence plays the main role. For the α-EM algorithm, the reason for the speedup is explained using Hessian and Jacobian matrices for learning. For the α-ICA learning, methods of exploiting the past and future information are presented. Examples are shown on single-loop α-EM's and sample-based α-ICA's. In all cases, effective speedups are observed. Thus, this paper's examples together with formerly reported ones are evidences that the speed improvement by the α-logarithm is a general property beyond individual problems.

    Original languageEnglish
    Title of host publicationProceedings of the International Joint Conference on Neural Networks
    Place of PublicationPiscataway, NJ, United States
    PublisherIEEE
    Pages351-356
    Number of pages6
    Volume3
    Publication statusPublished - 2000
    EventInternational Joint Conference on Neural Networks (IJCNN'2000) - Como, Italy
    Duration: 2000 Jul 242000 Jul 27

    Other

    OtherInternational Joint Conference on Neural Networks (IJCNN'2000)
    CityComo, Italy
    Period00/7/2400/7/27

    Fingerprint

    Independent component analysis
    Jacobian matrices

    ASJC Scopus subject areas

    • Software

    Cite this

    Matsuyama, Y., Niimoto, T., Katsumata, N., Suzuki, Y., & Furukawa, S. (2000). α-EM algorithm and α-ICA learning based upon extended logarithmic information measures. In Proceedings of the International Joint Conference on Neural Networks (Vol. 3, pp. 351-356). Piscataway, NJ, United States: IEEE.

    α-EM algorithm and α-ICA learning based upon extended logarithmic information measures. / Matsuyama, Yasuo; Niimoto, Takeshi; Katsumata, Naoto; Suzuki, Yoshitaka; Furukawa, Satoshi.

    Proceedings of the International Joint Conference on Neural Networks. Vol. 3 Piscataway, NJ, United States : IEEE, 2000. p. 351-356.

    Research output: Chapter in Book/Report/Conference proceedingConference contribution

    Matsuyama, Y, Niimoto, T, Katsumata, N, Suzuki, Y & Furukawa, S 2000, α-EM algorithm and α-ICA learning based upon extended logarithmic information measures. in Proceedings of the International Joint Conference on Neural Networks. vol. 3, IEEE, Piscataway, NJ, United States, pp. 351-356, International Joint Conference on Neural Networks (IJCNN'2000), Como, Italy, 00/7/24.
    Matsuyama Y, Niimoto T, Katsumata N, Suzuki Y, Furukawa S. α-EM algorithm and α-ICA learning based upon extended logarithmic information measures. In Proceedings of the International Joint Conference on Neural Networks. Vol. 3. Piscataway, NJ, United States: IEEE. 2000. p. 351-356
    Matsuyama, Yasuo ; Niimoto, Takeshi ; Katsumata, Naoto ; Suzuki, Yoshitaka ; Furukawa, Satoshi. / α-EM algorithm and α-ICA learning based upon extended logarithmic information measures. Proceedings of the International Joint Conference on Neural Networks. Vol. 3 Piscataway, NJ, United States : IEEE, 2000. pp. 351-356
    @inproceedings{3186c2fa0158480b8b88d33ae73b0849,
    title = "α-EM algorithm and α-ICA learning based upon extended logarithmic information measures",
    abstract = "The α-logarithm extends the logarithm as the special case of α = -1. Usage of α-related information measures based upon this extended logarithm is expected to be effective to speedup of convergence, i.e., on the improvement of learning aptitude. In this paper, two typical cases are investigated. One is the α-EM algorithm (α-Expectation-Maximization algorithm) which is derived from the α-log-likelihood ratio. The other is the α-ICA (α-Independent Component Analysis) which is formulated as minimizing the α-mutual information. In the derivation of both algorithms, the α-divergence plays the main role. For the α-EM algorithm, the reason for the speedup is explained using Hessian and Jacobian matrices for learning. For the α-ICA learning, methods of exploiting the past and future information are presented. Examples are shown on single-loop α-EM's and sample-based α-ICA's. In all cases, effective speedups are observed. Thus, this paper's examples together with formerly reported ones are evidences that the speed improvement by the α-logarithm is a general property beyond individual problems.",
    author = "Yasuo Matsuyama and Takeshi Niimoto and Naoto Katsumata and Yoshitaka Suzuki and Satoshi Furukawa",
    year = "2000",
    language = "English",
    volume = "3",
    pages = "351--356",
    booktitle = "Proceedings of the International Joint Conference on Neural Networks",
    publisher = "IEEE",

    }

    TY - GEN

    T1 - α-EM algorithm and α-ICA learning based upon extended logarithmic information measures

    AU - Matsuyama, Yasuo

    AU - Niimoto, Takeshi

    AU - Katsumata, Naoto

    AU - Suzuki, Yoshitaka

    AU - Furukawa, Satoshi

    PY - 2000

    Y1 - 2000

    N2 - The α-logarithm extends the logarithm as the special case of α = -1. Usage of α-related information measures based upon this extended logarithm is expected to be effective to speedup of convergence, i.e., on the improvement of learning aptitude. In this paper, two typical cases are investigated. One is the α-EM algorithm (α-Expectation-Maximization algorithm) which is derived from the α-log-likelihood ratio. The other is the α-ICA (α-Independent Component Analysis) which is formulated as minimizing the α-mutual information. In the derivation of both algorithms, the α-divergence plays the main role. For the α-EM algorithm, the reason for the speedup is explained using Hessian and Jacobian matrices for learning. For the α-ICA learning, methods of exploiting the past and future information are presented. Examples are shown on single-loop α-EM's and sample-based α-ICA's. In all cases, effective speedups are observed. Thus, this paper's examples together with formerly reported ones are evidences that the speed improvement by the α-logarithm is a general property beyond individual problems.

    AB - The α-logarithm extends the logarithm as the special case of α = -1. Usage of α-related information measures based upon this extended logarithm is expected to be effective to speedup of convergence, i.e., on the improvement of learning aptitude. In this paper, two typical cases are investigated. One is the α-EM algorithm (α-Expectation-Maximization algorithm) which is derived from the α-log-likelihood ratio. The other is the α-ICA (α-Independent Component Analysis) which is formulated as minimizing the α-mutual information. In the derivation of both algorithms, the α-divergence plays the main role. For the α-EM algorithm, the reason for the speedup is explained using Hessian and Jacobian matrices for learning. For the α-ICA learning, methods of exploiting the past and future information are presented. Examples are shown on single-loop α-EM's and sample-based α-ICA's. In all cases, effective speedups are observed. Thus, this paper's examples together with formerly reported ones are evidences that the speed improvement by the α-logarithm is a general property beyond individual problems.

    UR - http://www.scopus.com/inward/record.url?scp=0033717373&partnerID=8YFLogxK

    UR - http://www.scopus.com/inward/citedby.url?scp=0033717373&partnerID=8YFLogxK

    M3 - Conference contribution

    AN - SCOPUS:0033717373

    VL - 3

    SP - 351

    EP - 356

    BT - Proceedings of the International Joint Conference on Neural Networks

    PB - IEEE

    CY - Piscataway, NJ, United States

    ER -