Neural networks with branch gates

Kenichi Goto, Kotaro Hirasawa, Jinglu Hu

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

In this paper, a new architecture of Neural Networks (NNs) is proposed named Neural Networks with branch gates (NN-bg). It aims at improving the generalization ability of NNs by controlling the connectivity of neurons adaptively depending on the input information. To realize such architecture, we use a branch control system having low calculation costs. In the branch control system, the distance between the input values of the network and parameters of the branch control system is calculated. After normalized within 0 to 1, the outputs of the branch control system are multiplied to the branches of the NN. The parameters of the branch control system are trained by a random searching method, RasID, to realize an adaptive optimization with very small number of training steps. Through some simulations, the usefulness of the three-layered NN-bg is shown compared with conventional layered neural networks.

Original languageEnglish
Title of host publication2004 IEEE International Joint Conference on Neural Networks - Proceedings
Pages2331-2336
Number of pages6
DOIs
Publication statusPublished - 2004 Dec 1
Event2004 IEEE International Joint Conference on Neural Networks - Proceedings - Budapest, Hungary
Duration: 2004 Jul 252004 Jul 29

Publication series

NameIEEE International Conference on Neural Networks - Conference Proceedings
Volume3
ISSN (Print)1098-7576

Conference

Conference2004 IEEE International Joint Conference on Neural Networks - Proceedings
CountryHungary
CityBudapest
Period04/7/2504/7/29

ASJC Scopus subject areas

  • Software

Fingerprint Dive into the research topics of 'Neural networks with branch gates'. Together they form a unique fingerprint.

  • Cite this

    Goto, K., Hirasawa, K., & Hu, J. (2004). Neural networks with branch gates. In 2004 IEEE International Joint Conference on Neural Networks - Proceedings (pp. 2331-2336). (IEEE International Conference on Neural Networks - Conference Proceedings; Vol. 3). https://doi.org/10.1109/IJCNN.2004.1380990