Overfitting Measurement of Deep Neural Networks Using No Data

Satoru Watanabe, Hayato Yamana

研究成果: Conference contribution

2 被引用数 (Scopus)

抄録

Overfitting reduces the generalizability of deep neural networks (DNNs). Overfitting is generally detected by comparing the accuracies and losses of training and validation data; however, the detection method requires vast amounts of training data and is not always effective for forthcoming data due to the heterogeneity between training and forthcoming data. The dropout technique has been employed to prevent DNNs from overfitting, where the neurons in DNNs are invalidated randomly during their training. It has been hypothesized that this technique prevents DNNs from overfitting by restraining the co-adaptions among neurons. This hypothesis implies that overfitting of a DNN is a result of the co-adaptions among neurons and can be detected by investigating the inner representation of DNNs. Thus, we propose a method to detect overfitting of DNNs using no training and test data. The proposed method measures the degree of co-adaptions among neurons using persistent homology (PH). The proposed PH-based overfitting measure (PHOM) method constructs clique complexes on DNNs using the trained parameters of DNNs, and the one-dimensional PH investigates the co-adaptions among neurons. Thus, PHOM requires no training and test data to measure overfitting. We applied PHOM to convolutional neural networks trained for the classification problems of the CIFAR-10, SVHN, and Tiny ImageNet data sets. The experimental results demonstrate that PHOM reveals the degree of overfitting of DNNs to the training data, which suggests that PHOM enables us to filter overfitted DNNs without requiring the training and test data.

本文言語English
ホスト出版物のタイトル2021 IEEE 8th International Conference on Data Science and Advanced Analytics, DSAA 2021
出版社Institute of Electrical and Electronics Engineers Inc.
ISBN(電子版)9781665420990
DOI
出版ステータスPublished - 2021
イベント8th IEEE International Conference on Data Science and Advanced Analytics, DSAA 2021 - Virtual, Online, Portugal
継続期間: 2021 10月 62021 10月 9

出版物シリーズ

名前2021 IEEE 8th International Conference on Data Science and Advanced Analytics, DSAA 2021

Conference

Conference8th IEEE International Conference on Data Science and Advanced Analytics, DSAA 2021
国/地域Portugal
CityVirtual, Online
Period21/10/621/10/9

ASJC Scopus subject areas

  • コンピュータ ネットワークおよび通信
  • 信号処理
  • 情報システムおよび情報管理
  • 統計学、確率および不確実性

フィンガープリント

「Overfitting Measurement of Deep Neural Networks Using No Data」の研究トピックを掘り下げます。これらがまとまってユニークなフィンガープリントを構成します。

引用スタイル