Neural networks with branch gates

Kenichi Goto, Kotaro Hirasawa, Takayuki Furuzuki

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

In this paper, a new architecture of Neural Networks (NNs) is proposed named Neural Networks with branch gates (NN-bg). It aims at improving the generalization ability of NNs by controlling the connectivity of neurons adaptively depending on the input information. To realize such architecture, we use a branch control system having low calculation costs. In the branch control system, the distance between the input values of the network and parameters of the branch control system is calculated. After normalized within 0 to 1, the outputs of the branch control system are multiplied to the branches of the NN. The parameters of the branch control system are trained by a random searching method, RasID, to realize an adaptive optimization with very small number of training steps. Through some simulations, the usefulness of the three-layered NN-bg is shown compared with conventional layered neural networks.

Original languageEnglish
Title of host publicationIEEE International Conference on Neural Networks - Conference Proceedings
Pages2331-2336
Number of pages6
Volume3
DOIs
Publication statusPublished - 2004
Event2004 IEEE International Joint Conference on Neural Networks - Proceedings - Budapest
Duration: 2004 Jul 252004 Jul 29

Other

Other2004 IEEE International Joint Conference on Neural Networks - Proceedings
CityBudapest
Period04/7/2504/7/29

Fingerprint

Neural networks
Control systems
Neurons
Costs

ASJC Scopus subject areas

  • Software

Cite this

Goto, K., Hirasawa, K., & Furuzuki, T. (2004). Neural networks with branch gates. In IEEE International Conference on Neural Networks - Conference Proceedings (Vol. 3, pp. 2331-2336) https://doi.org/10.1109/IJCNN.2004.1380990

Neural networks with branch gates. / Goto, Kenichi; Hirasawa, Kotaro; Furuzuki, Takayuki.

IEEE International Conference on Neural Networks - Conference Proceedings. Vol. 3 2004. p. 2331-2336.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Goto, K, Hirasawa, K & Furuzuki, T 2004, Neural networks with branch gates. in IEEE International Conference on Neural Networks - Conference Proceedings. vol. 3, pp. 2331-2336, 2004 IEEE International Joint Conference on Neural Networks - Proceedings, Budapest, 04/7/25. https://doi.org/10.1109/IJCNN.2004.1380990
Goto K, Hirasawa K, Furuzuki T. Neural networks with branch gates. In IEEE International Conference on Neural Networks - Conference Proceedings. Vol. 3. 2004. p. 2331-2336 https://doi.org/10.1109/IJCNN.2004.1380990
Goto, Kenichi ; Hirasawa, Kotaro ; Furuzuki, Takayuki. / Neural networks with branch gates. IEEE International Conference on Neural Networks - Conference Proceedings. Vol. 3 2004. pp. 2331-2336
@inproceedings{6fdb2a486df7438aacf3bbe158d8a4e3,
title = "Neural networks with branch gates",
abstract = "In this paper, a new architecture of Neural Networks (NNs) is proposed named Neural Networks with branch gates (NN-bg). It aims at improving the generalization ability of NNs by controlling the connectivity of neurons adaptively depending on the input information. To realize such architecture, we use a branch control system having low calculation costs. In the branch control system, the distance between the input values of the network and parameters of the branch control system is calculated. After normalized within 0 to 1, the outputs of the branch control system are multiplied to the branches of the NN. The parameters of the branch control system are trained by a random searching method, RasID, to realize an adaptive optimization with very small number of training steps. Through some simulations, the usefulness of the three-layered NN-bg is shown compared with conventional layered neural networks.",
author = "Kenichi Goto and Kotaro Hirasawa and Takayuki Furuzuki",
year = "2004",
doi = "10.1109/IJCNN.2004.1380990",
language = "English",
isbn = "0780383591",
volume = "3",
pages = "2331--2336",
booktitle = "IEEE International Conference on Neural Networks - Conference Proceedings",

}

TY - GEN

T1 - Neural networks with branch gates

AU - Goto, Kenichi

AU - Hirasawa, Kotaro

AU - Furuzuki, Takayuki

PY - 2004

Y1 - 2004

N2 - In this paper, a new architecture of Neural Networks (NNs) is proposed named Neural Networks with branch gates (NN-bg). It aims at improving the generalization ability of NNs by controlling the connectivity of neurons adaptively depending on the input information. To realize such architecture, we use a branch control system having low calculation costs. In the branch control system, the distance between the input values of the network and parameters of the branch control system is calculated. After normalized within 0 to 1, the outputs of the branch control system are multiplied to the branches of the NN. The parameters of the branch control system are trained by a random searching method, RasID, to realize an adaptive optimization with very small number of training steps. Through some simulations, the usefulness of the three-layered NN-bg is shown compared with conventional layered neural networks.

AB - In this paper, a new architecture of Neural Networks (NNs) is proposed named Neural Networks with branch gates (NN-bg). It aims at improving the generalization ability of NNs by controlling the connectivity of neurons adaptively depending on the input information. To realize such architecture, we use a branch control system having low calculation costs. In the branch control system, the distance between the input values of the network and parameters of the branch control system is calculated. After normalized within 0 to 1, the outputs of the branch control system are multiplied to the branches of the NN. The parameters of the branch control system are trained by a random searching method, RasID, to realize an adaptive optimization with very small number of training steps. Through some simulations, the usefulness of the three-layered NN-bg is shown compared with conventional layered neural networks.

UR - http://www.scopus.com/inward/record.url?scp=10844287938&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=10844287938&partnerID=8YFLogxK

U2 - 10.1109/IJCNN.2004.1380990

DO - 10.1109/IJCNN.2004.1380990

M3 - Conference contribution

AN - SCOPUS:10844287938

SN - 0780383591

VL - 3

SP - 2331

EP - 2336

BT - IEEE International Conference on Neural Networks - Conference Proceedings

ER -