### Abstract

The α-logarithm extends the logarithm as the special case of α = -1. Usage of α-related information measures based upon this extended logarithm is expected to be effective to speedup of convergence, i.e., on the improvement of learning aptitude. In this paper, two typical cases are investigated. One is the α-EM algorithm (α-Expectation-Maximization algorithm) which is derived from the α-log-likelihood ratio. The other is the α-ICA (α-Independent Component Analysis) which is formulated as minimizing the α-mutual information. In the derivation of both algorithms, the α-divergence plays the main role. For the α-EM algorithm, the reason for the speedup is explained using Hessian and Jacobian matrices for learning. For the α-ICA learning, methods of exploiting the past and future information are presented. Examples are shown on single-loop α-EM's and sample-based α-ICA's. In all cases, effective speedups are observed. Thus, this paper's examples together with formerly reported ones are evidences that the speed improvement by the α-logarithm is a general property beyond individual problems.

Original language | English |
---|---|

Title of host publication | Proceedings of the International Joint Conference on Neural Networks |

Place of Publication | Piscataway, NJ, United States |

Publisher | IEEE |

Pages | 351-356 |

Number of pages | 6 |

Volume | 3 |

Publication status | Published - 2000 |

Event | International Joint Conference on Neural Networks (IJCNN'2000) - Como, Italy Duration: 2000 Jul 24 → 2000 Jul 27 |

### Other

Other | International Joint Conference on Neural Networks (IJCNN'2000) |
---|---|

City | Como, Italy |

Period | 00/7/24 → 00/7/27 |

### Fingerprint

### ASJC Scopus subject areas

- Software

### Cite this

*Proceedings of the International Joint Conference on Neural Networks*(Vol. 3, pp. 351-356). Piscataway, NJ, United States: IEEE.

**α-EM algorithm and α-ICA learning based upon extended logarithmic information measures.** / Matsuyama, Yasuo; Niimoto, Takeshi; Katsumata, Naoto; Suzuki, Yoshitaka; Furukawa, Satoshi.

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

*Proceedings of the International Joint Conference on Neural Networks.*vol. 3, IEEE, Piscataway, NJ, United States, pp. 351-356, International Joint Conference on Neural Networks (IJCNN'2000), Como, Italy, 00/7/24.

}

TY - GEN

T1 - α-EM algorithm and α-ICA learning based upon extended logarithmic information measures

AU - Matsuyama, Yasuo

AU - Niimoto, Takeshi

AU - Katsumata, Naoto

AU - Suzuki, Yoshitaka

AU - Furukawa, Satoshi

PY - 2000

Y1 - 2000

N2 - The α-logarithm extends the logarithm as the special case of α = -1. Usage of α-related information measures based upon this extended logarithm is expected to be effective to speedup of convergence, i.e., on the improvement of learning aptitude. In this paper, two typical cases are investigated. One is the α-EM algorithm (α-Expectation-Maximization algorithm) which is derived from the α-log-likelihood ratio. The other is the α-ICA (α-Independent Component Analysis) which is formulated as minimizing the α-mutual information. In the derivation of both algorithms, the α-divergence plays the main role. For the α-EM algorithm, the reason for the speedup is explained using Hessian and Jacobian matrices for learning. For the α-ICA learning, methods of exploiting the past and future information are presented. Examples are shown on single-loop α-EM's and sample-based α-ICA's. In all cases, effective speedups are observed. Thus, this paper's examples together with formerly reported ones are evidences that the speed improvement by the α-logarithm is a general property beyond individual problems.

AB - The α-logarithm extends the logarithm as the special case of α = -1. Usage of α-related information measures based upon this extended logarithm is expected to be effective to speedup of convergence, i.e., on the improvement of learning aptitude. In this paper, two typical cases are investigated. One is the α-EM algorithm (α-Expectation-Maximization algorithm) which is derived from the α-log-likelihood ratio. The other is the α-ICA (α-Independent Component Analysis) which is formulated as minimizing the α-mutual information. In the derivation of both algorithms, the α-divergence plays the main role. For the α-EM algorithm, the reason for the speedup is explained using Hessian and Jacobian matrices for learning. For the α-ICA learning, methods of exploiting the past and future information are presented. Examples are shown on single-loop α-EM's and sample-based α-ICA's. In all cases, effective speedups are observed. Thus, this paper's examples together with formerly reported ones are evidences that the speed improvement by the α-logarithm is a general property beyond individual problems.

UR - http://www.scopus.com/inward/record.url?scp=0033717373&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=0033717373&partnerID=8YFLogxK

M3 - Conference contribution

AN - SCOPUS:0033717373

VL - 3

SP - 351

EP - 356

BT - Proceedings of the International Joint Conference on Neural Networks

PB - IEEE

CY - Piscataway, NJ, United States

ER -