Relation between weight initialization of neural networks and pruning algorithms case study on Mackey-Glass time series

W. Wan; K. Hirasawa; J. Hu; J. Murata

Relation between weight initialization of neural networks and pruning algorithms case study on Mackey-Glass time series

W. Wan^*, K. Hirasawa, J. Hu, J. Murata

^*Corresponding author for this work

Research output: Contribution to conference › Paper › peer-review

5 Citations (Scopus)

Abstract

The implementation of weight initialization is directly related to the convergence of learning algorithms. In this paper we made a case study on the famous Mackey-Glass time series problem in order to try to find some relations between weight initialization of neural networks and pruning algorithms. The pruning algorithm used in simulations is Laplace regularizer method, that is, the backpropagation algorithm with Laplace regularizer added to the criterion function. Simulation results show that different kinds of initialization weight matrices display almost the same generalization ability when using the pruning algorithm, at least for the Mackey-Glass time series.

Original language	English
Pages	1750-1755
Number of pages	6
Publication status	Published - 2001 Jan 1
Externally published	Yes
Event	International Joint Conference on Neural Networks (IJCNN'01) - Washington, DC, United States Duration: 2001 Jul 15 → 2001 Jul 19

Conference

Conference	International Joint Conference on Neural Networks (IJCNN'01)
Country/Territory	United States
City	Washington, DC
Period	01/7/15 → 01/7/19

Keywords

Mackey-Glass time series
Pruning methods
Weight initialization

ASJC Scopus subject areas

Software
Artificial Intelligence

Cite this

@conference{5531fd1eda5e44da95ca7166212be2d6,

title = "Relation between weight initialization of neural networks and pruning algorithms case study on Mackey-Glass time series",

abstract = "The implementation of weight initialization is directly related to the convergence of learning algorithms. In this paper we made a case study on the famous Mackey-Glass time series problem in order to try to find some relations between weight initialization of neural networks and pruning algorithms. The pruning algorithm used in simulations is Laplace regularizer method, that is, the backpropagation algorithm with Laplace regularizer added to the criterion function. Simulation results show that different kinds of initialization weight matrices display almost the same generalization ability when using the pruning algorithm, at least for the Mackey-Glass time series.",

keywords = "Mackey-Glass time series, Pruning methods, Weight initialization",

author = "W. Wan and K. Hirasawa and J. Hu and J. Murata",

year = "2001",

month = jan,

day = "1",

language = "English",

pages = "1750--1755",

note = "International Joint Conference on Neural Networks (IJCNN'01) ; Conference date: 15-07-2001 Through 19-07-2001",

}

TY - CONF

T1 - Relation between weight initialization of neural networks and pruning algorithms case study on Mackey-Glass time series

AU - Wan, W.

AU - Hirasawa, K.

AU - Hu, J.

AU - Murata, J.

PY - 2001/1/1

Y1 - 2001/1/1

N2 - The implementation of weight initialization is directly related to the convergence of learning algorithms. In this paper we made a case study on the famous Mackey-Glass time series problem in order to try to find some relations between weight initialization of neural networks and pruning algorithms. The pruning algorithm used in simulations is Laplace regularizer method, that is, the backpropagation algorithm with Laplace regularizer added to the criterion function. Simulation results show that different kinds of initialization weight matrices display almost the same generalization ability when using the pruning algorithm, at least for the Mackey-Glass time series.

AB - The implementation of weight initialization is directly related to the convergence of learning algorithms. In this paper we made a case study on the famous Mackey-Glass time series problem in order to try to find some relations between weight initialization of neural networks and pruning algorithms. The pruning algorithm used in simulations is Laplace regularizer method, that is, the backpropagation algorithm with Laplace regularizer added to the criterion function. Simulation results show that different kinds of initialization weight matrices display almost the same generalization ability when using the pruning algorithm, at least for the Mackey-Glass time series.

KW - Mackey-Glass time series

KW - Pruning methods

KW - Weight initialization

UR - http://www.scopus.com/inward/record.url?scp=0034876346&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=0034876346&partnerID=8YFLogxK

M3 - Paper

AN - SCOPUS:0034876346

SP - 1750

EP - 1755

T2 - International Joint Conference on Neural Networks (IJCNN'01)

Y2 - 15 July 2001 through 19 July 2001

ER -

Relation between weight initialization of neural networks and pruning algorithms case study on Mackey-Glass time series

Abstract

Conference

Keywords

ASJC Scopus subject areas

Other files and links

Fingerprint

Cite this