Riemannian adaptive stochastic gradient algorithms on matrix manifolds

Hiroyuki Kasai, Pratik Jawanpuria, Bamdev Mishra

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Adaptive stochastic gradient algorithms in the Euclidean space have attracted much attention lately. Such explorations on Riemannian manifolds, on the other hand, are relatively new, limited, and challenging. This is because of the intrinsic nonlinear structure of the underlying manifold and the absence of a canonical coordinate system. In machine learning applications, however, most manifolds of interest are represented as matrices with notions of row and column subspaces. In addition, the implicit manifold-related constraints may also lie on such subspaces. For example, the Grassmann manifold is the set of column subspaces. To this end, such a rich structure should not be lost by transforming matrices to just a stack of vectors while developing optimization algorithms on manifolds. We propose novel stochastic gradient algorithms for problems on Riemannian matrix manifolds by adapting the row and column sub-spaces of gradients. Our algorithms are provably convergent and they achieve the convergence rate of order O(log(T)/√T), where T is the number of iterations. Our experiments illustrate the efficacy of the proposed algorithms on several applications.

Original languageEnglish
Title of host publication36th International Conference on Machine Learning, ICML 2019
PublisherInternational Machine Learning Society (IMLS)
Pages5699-5708
Number of pages10
ISBN (Electronic)9781510886988
Publication statusPublished - 2019
Externally publishedYes
Event36th International Conference on Machine Learning, ICML 2019 - Long Beach, United States
Duration: 2019 Jun 92019 Jun 15

Publication series

Name36th International Conference on Machine Learning, ICML 2019
Volume2019-June

Conference

Conference36th International Conference on Machine Learning, ICML 2019
CountryUnited States
CityLong Beach
Period19/6/919/6/15

ASJC Scopus subject areas

  • Education
  • Computer Science Applications
  • Human-Computer Interaction

Fingerprint Dive into the research topics of 'Riemannian adaptive stochastic gradient algorithms on matrix manifolds'. Together they form a unique fingerprint.

  • Cite this

    Kasai, H., Jawanpuria, P., & Mishra, B. (2019). Riemannian adaptive stochastic gradient algorithms on matrix manifolds. In 36th International Conference on Machine Learning, ICML 2019 (pp. 5699-5708). (36th International Conference on Machine Learning, ICML 2019; Vol. 2019-June). International Machine Learning Society (IMLS).