Training quasi-ARX neural network model by homotopy approach

Takayuki Furuzuki, Xibin Lu, Kotaro Hirasawa

Research output: Chapter in Book/Report/Conference proceedingConference contribution

2 Citations (Scopus)

Abstract

Quasi-ARX neural networks (NN) are NN based nonlinear models that not only have linear structures similar to linear ARX models, but also have useful interpretation in part of their parameters. However when applying an ordinary backpropagation (BP) for the training, it has potential risk that the BP algorithm is stuck at a local minimum, which results in a poorly trained model. In this paper, a homotopy continuation method is introduced to improve the convergence performance of BP training. The idea is to start the BP training with the criterion function for linear ARX model, which is gradually deformed first into one for quasi-ARX NN model with linear node functions, and then into the actual one for quasi-ARX NN with sigmoid node functions. By building the deformation into a usual recursive procedure for BP training of quasi-ARX NN model with adaptable node functions so that the proposed homotopy based BP algorithm is able to achieve improved convergence performance without much increase in the computation load. Numerical simulation results show that the proposed homotopy based BP has better performance than an ordinary BP.

Original languageEnglish
Title of host publicationProceedings of the SICE Annual Conference
Pages17-22
Number of pages6
Publication statusPublished - 2004
EventSICE Annual Conference 2004 - Sapporo
Duration: 2004 Aug 42004 Aug 6

Other

OtherSICE Annual Conference 2004
CitySapporo
Period04/8/404/8/6

Fingerprint

Backpropagation
Neural networks
Backpropagation algorithms
Computer simulation

Keywords

  • Backpropagation algorithm
  • Homotopy continuation method
  • Linear ARX model
  • Local minimum
  • Neural network

ASJC Scopus subject areas

  • Engineering(all)

Cite this

Furuzuki, T., Lu, X., & Hirasawa, K. (2004). Training quasi-ARX neural network model by homotopy approach. In Proceedings of the SICE Annual Conference (pp. 17-22). [WAI-1-4]

Training quasi-ARX neural network model by homotopy approach. / Furuzuki, Takayuki; Lu, Xibin; Hirasawa, Kotaro.

Proceedings of the SICE Annual Conference. 2004. p. 17-22 WAI-1-4.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Furuzuki, T, Lu, X & Hirasawa, K 2004, Training quasi-ARX neural network model by homotopy approach. in Proceedings of the SICE Annual Conference., WAI-1-4, pp. 17-22, SICE Annual Conference 2004, Sapporo, 04/8/4.
Furuzuki T, Lu X, Hirasawa K. Training quasi-ARX neural network model by homotopy approach. In Proceedings of the SICE Annual Conference. 2004. p. 17-22. WAI-1-4
Furuzuki, Takayuki ; Lu, Xibin ; Hirasawa, Kotaro. / Training quasi-ARX neural network model by homotopy approach. Proceedings of the SICE Annual Conference. 2004. pp. 17-22
@inproceedings{ad9ac24d84dd4170a65c0637d44f1945,
title = "Training quasi-ARX neural network model by homotopy approach",
abstract = "Quasi-ARX neural networks (NN) are NN based nonlinear models that not only have linear structures similar to linear ARX models, but also have useful interpretation in part of their parameters. However when applying an ordinary backpropagation (BP) for the training, it has potential risk that the BP algorithm is stuck at a local minimum, which results in a poorly trained model. In this paper, a homotopy continuation method is introduced to improve the convergence performance of BP training. The idea is to start the BP training with the criterion function for linear ARX model, which is gradually deformed first into one for quasi-ARX NN model with linear node functions, and then into the actual one for quasi-ARX NN with sigmoid node functions. By building the deformation into a usual recursive procedure for BP training of quasi-ARX NN model with adaptable node functions so that the proposed homotopy based BP algorithm is able to achieve improved convergence performance without much increase in the computation load. Numerical simulation results show that the proposed homotopy based BP has better performance than an ordinary BP.",
keywords = "Backpropagation algorithm, Homotopy continuation method, Linear ARX model, Local minimum, Neural network",
author = "Takayuki Furuzuki and Xibin Lu and Kotaro Hirasawa",
year = "2004",
language = "English",
pages = "17--22",
booktitle = "Proceedings of the SICE Annual Conference",

}

TY - GEN

T1 - Training quasi-ARX neural network model by homotopy approach

AU - Furuzuki, Takayuki

AU - Lu, Xibin

AU - Hirasawa, Kotaro

PY - 2004

Y1 - 2004

N2 - Quasi-ARX neural networks (NN) are NN based nonlinear models that not only have linear structures similar to linear ARX models, but also have useful interpretation in part of their parameters. However when applying an ordinary backpropagation (BP) for the training, it has potential risk that the BP algorithm is stuck at a local minimum, which results in a poorly trained model. In this paper, a homotopy continuation method is introduced to improve the convergence performance of BP training. The idea is to start the BP training with the criterion function for linear ARX model, which is gradually deformed first into one for quasi-ARX NN model with linear node functions, and then into the actual one for quasi-ARX NN with sigmoid node functions. By building the deformation into a usual recursive procedure for BP training of quasi-ARX NN model with adaptable node functions so that the proposed homotopy based BP algorithm is able to achieve improved convergence performance without much increase in the computation load. Numerical simulation results show that the proposed homotopy based BP has better performance than an ordinary BP.

AB - Quasi-ARX neural networks (NN) are NN based nonlinear models that not only have linear structures similar to linear ARX models, but also have useful interpretation in part of their parameters. However when applying an ordinary backpropagation (BP) for the training, it has potential risk that the BP algorithm is stuck at a local minimum, which results in a poorly trained model. In this paper, a homotopy continuation method is introduced to improve the convergence performance of BP training. The idea is to start the BP training with the criterion function for linear ARX model, which is gradually deformed first into one for quasi-ARX NN model with linear node functions, and then into the actual one for quasi-ARX NN with sigmoid node functions. By building the deformation into a usual recursive procedure for BP training of quasi-ARX NN model with adaptable node functions so that the proposed homotopy based BP algorithm is able to achieve improved convergence performance without much increase in the computation load. Numerical simulation results show that the proposed homotopy based BP has better performance than an ordinary BP.

KW - Backpropagation algorithm

KW - Homotopy continuation method

KW - Linear ARX model

KW - Local minimum

KW - Neural network

UR - http://www.scopus.com/inward/record.url?scp=12744263326&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=12744263326&partnerID=8YFLogxK

M3 - Conference contribution

AN - SCOPUS:12744263326

SP - 17

EP - 22

BT - Proceedings of the SICE Annual Conference

ER -