Multiplication units in feedforward neural networks and its training

Dazi Li, K. Hirasawa, Takayuki Furuzuki, J. Murata

Research output: Chapter in Book/Report/Conference proceedingConference contribution

6 Citations (Scopus)

Abstract

This paper proposes the application of neural networks with multiplication units to parity-N problem, mirror symmetry problem and a function approximation problem. It is clear that, higher-order terms in neural networks, such as sigma-pi unit, can improve the computational power of neural networks considerably. But how the real neurons do this is still unclear. We have used one multiplication unit to construct full higher-order terms of all the inputs, which was proved very efficient for parity-N problem. Our earlier work on applying multiplication units to other problems suffered from the drawback of gradient-based algorithm, such as backpropagation algorithms, for being easy to stuck at local minima due to the complexity of the network. In order to overcome this problem we consider a novel random search, RasID, for the training of neural networks with multiplication units, which does an intensified search where it is easy to find good solutions locally and a diversified search to escape from local minima under a pure random search scheme. The method shows its advantage on the training of neural networks with multiplication units.

Original languageEnglish
Title of host publicationICONIP 2002 - Proceedings of the 9th International Conference on Neural Information Processing: Computational Intelligence for the E-Age
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages75-79
Number of pages5
Volume1
ISBN (Print)9810475241, 9789810475246
DOIs
Publication statusPublished - 2002
Externally publishedYes
Event9th International Conference on Neural Information Processing, ICONIP 2002 - Singapore, Singapore
Duration: 2002 Nov 182002 Nov 22

Other

Other9th International Conference on Neural Information Processing, ICONIP 2002
CountrySingapore
CitySingapore
Period02/11/1802/11/22

ASJC Scopus subject areas

  • Computer Networks and Communications
  • Information Systems
  • Signal Processing

Fingerprint Dive into the research topics of 'Multiplication units in feedforward neural networks and its training'. Together they form a unique fingerprint.

  • Cite this

    Li, D., Hirasawa, K., Furuzuki, T., & Murata, J. (2002). Multiplication units in feedforward neural networks and its training. In ICONIP 2002 - Proceedings of the 9th International Conference on Neural Information Processing: Computational Intelligence for the E-Age (Vol. 1, pp. 75-79). [1202134] Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/ICONIP.2002.1202134