TY - GEN
T1 - Neural networks with branch gates
AU - Goto, Kenichi
AU - Hirasawa, Kotaro
AU - Hu, Jinglu
PY - 2004/12/1
Y1 - 2004/12/1
N2 - In this paper, a new architecture of Neural Networks (NNs) is proposed named Neural Networks with branch gates (NN-bg). It aims at improving the generalization ability of NNs by controlling the connectivity of neurons adaptively depending on the input information. To realize such architecture, we use a branch control system having low calculation costs. In the branch control system, the distance between the input values of the network and parameters of the branch control system is calculated. After normalized within 0 to 1, the outputs of the branch control system are multiplied to the branches of the NN. The parameters of the branch control system are trained by a random searching method, RasID, to realize an adaptive optimization with very small number of training steps. Through some simulations, the usefulness of the three-layered NN-bg is shown compared with conventional layered neural networks.
AB - In this paper, a new architecture of Neural Networks (NNs) is proposed named Neural Networks with branch gates (NN-bg). It aims at improving the generalization ability of NNs by controlling the connectivity of neurons adaptively depending on the input information. To realize such architecture, we use a branch control system having low calculation costs. In the branch control system, the distance between the input values of the network and parameters of the branch control system is calculated. After normalized within 0 to 1, the outputs of the branch control system are multiplied to the branches of the NN. The parameters of the branch control system are trained by a random searching method, RasID, to realize an adaptive optimization with very small number of training steps. Through some simulations, the usefulness of the three-layered NN-bg is shown compared with conventional layered neural networks.
UR - http://www.scopus.com/inward/record.url?scp=10844287938&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=10844287938&partnerID=8YFLogxK
U2 - 10.1109/IJCNN.2004.1380990
DO - 10.1109/IJCNN.2004.1380990
M3 - Conference contribution
AN - SCOPUS:10844287938
SN - 0780383591
T3 - IEEE International Conference on Neural Networks - Conference Proceedings
SP - 2331
EP - 2336
BT - 2004 IEEE International Joint Conference on Neural Networks - Proceedings
T2 - 2004 IEEE International Joint Conference on Neural Networks - Proceedings
Y2 - 25 July 2004 through 29 July 2004
ER -