Theoretical Analysis of the Advantage of Deepening Neural Networks

研究成果: Conference contribution

抄録

We propose two new criteria to understand the advantage of deepening neural networks. It is important to know the expressivity of functions computable by deep neural networks in order to understand the advantage of deepening neural networks. Unless deep neural networks have enough expressivity, they cannot have good performance even though learning is successful. In this situation, the proposed criteria contribute to understanding the advantage of deepening neural networks since they can evaluate the expressivity independently from the efficiency of learning. The first criterion shows the approximation accuracy of deep neural networks to the target function. This criterion has the background that the goal of deep learning is approximating the target function by deep neural networks. The second criterion shows the property of linear regions of functions computable by deep neural networks. This criterion has the background that deep neural networks whose activation functions are piecewise linear are also piecewise linear. Furthermore, by the two criteria, we show that to increase layers is more effective than to increase units at each layer on improving the expressivity of deep neural networks.

本文言語English
ホスト出版物のタイトルProceedings - 19th IEEE International Conference on Machine Learning and Applications, ICMLA 2020
編集者M. Arif Wani, Feng Luo, Xiaolin Li, Dejing Dou, Francesco Bonchi
出版社Institute of Electrical and Electronics Engineers Inc.
ページ479-484
ページ数6
ISBN(電子版)9781728184708
DOI
出版ステータスPublished - 2020 12
イベント19th IEEE International Conference on Machine Learning and Applications, ICMLA 2020 - Virtual, Miami, United States
継続期間: 2020 12 142020 12 17

出版物シリーズ

名前Proceedings - 19th IEEE International Conference on Machine Learning and Applications, ICMLA 2020

Conference

Conference19th IEEE International Conference on Machine Learning and Applications, ICMLA 2020
国/地域United States
CityVirtual, Miami
Period20/12/1420/12/17

ASJC Scopus subject areas

  • 人工知能
  • コンピュータ サイエンスの応用
  • コンピュータ ビジョンおよびパターン認識
  • ハードウェアとアーキテクチャ

フィンガープリント

「Theoretical Analysis of the Advantage of Deepening Neural Networks」の研究トピックを掘り下げます。これらがまとまってユニークなフィンガープリントを構成します。

引用スタイル