Recurrent neural networks with multi-branch structure

Takashi Yamashita, Shingo Mabu, Kotaro Hirasawa, Takayuki Furuzuki

Research output: Contribution to journalArticle

Abstract

Universal Learning Networks (ULNs) provide a generalized framework to many kinds of structures of neural networks with supervised learning. Multi-Branch Neural Networks (MBNNs) which use the framework of ULNs have been already shown that they have better representation ability in feedforward neural networks (FNNs). Multi-Branch structure of MBNNs can be easily extended to recurrent neural networks (RNNs) because the characteristics of ULNs include the connection of multiple branches with arbitrary time delays. In this paper, therefore, RNNs with Multi-Branch structure are proposed and they show that their representation ability is better than conventional RNNs. RNNs can represent dynamical systems and are useful for time series prediction. The performance evaluation of RNNs with Multi-Branch structure was carried out using a benchmark of time series prediction. Simulation results showed that RNNs with Multi-Branch structure could obtain better performance than conventional RNNs, and also showed that they could improve the representation ability even if they are smaller sized networks.

Original languageEnglish
JournalIEEJ Transactions on Electronics, Information and Systems
Volume127
Issue number9
DOIs
Publication statusPublished - 2007 Jan 1

Fingerprint

Recurrent neural networks
Neural networks
Time series
Feedforward neural networks
Supervised learning
Time delay
Dynamical systems

Keywords

  • Multi-branch
  • Recurrent neural networks
  • Time series prediction
  • Universal learning networks

ASJC Scopus subject areas

  • Electrical and Electronic Engineering

Cite this

Recurrent neural networks with multi-branch structure. / Yamashita, Takashi; Mabu, Shingo; Hirasawa, Kotaro; Furuzuki, Takayuki.

In: IEEJ Transactions on Electronics, Information and Systems, Vol. 127, No. 9, 01.01.2007.

Research output: Contribution to journalArticle

@article{f6093d2e35914f75a86563c8d7eb9e3e,
title = "Recurrent neural networks with multi-branch structure",
abstract = "Universal Learning Networks (ULNs) provide a generalized framework to many kinds of structures of neural networks with supervised learning. Multi-Branch Neural Networks (MBNNs) which use the framework of ULNs have been already shown that they have better representation ability in feedforward neural networks (FNNs). Multi-Branch structure of MBNNs can be easily extended to recurrent neural networks (RNNs) because the characteristics of ULNs include the connection of multiple branches with arbitrary time delays. In this paper, therefore, RNNs with Multi-Branch structure are proposed and they show that their representation ability is better than conventional RNNs. RNNs can represent dynamical systems and are useful for time series prediction. The performance evaluation of RNNs with Multi-Branch structure was carried out using a benchmark of time series prediction. Simulation results showed that RNNs with Multi-Branch structure could obtain better performance than conventional RNNs, and also showed that they could improve the representation ability even if they are smaller sized networks.",
keywords = "Multi-branch, Recurrent neural networks, Time series prediction, Universal learning networks",
author = "Takashi Yamashita and Shingo Mabu and Kotaro Hirasawa and Takayuki Furuzuki",
year = "2007",
month = "1",
day = "1",
doi = "10.1541/ieejeiss.127.1430",
language = "English",
volume = "127",
journal = "IEEJ Transactions on Electronics, Information and Systems",
issn = "0385-4221",
publisher = "The Institute of Electrical Engineers of Japan",
number = "9",

}

TY - JOUR

T1 - Recurrent neural networks with multi-branch structure

AU - Yamashita, Takashi

AU - Mabu, Shingo

AU - Hirasawa, Kotaro

AU - Furuzuki, Takayuki

PY - 2007/1/1

Y1 - 2007/1/1

N2 - Universal Learning Networks (ULNs) provide a generalized framework to many kinds of structures of neural networks with supervised learning. Multi-Branch Neural Networks (MBNNs) which use the framework of ULNs have been already shown that they have better representation ability in feedforward neural networks (FNNs). Multi-Branch structure of MBNNs can be easily extended to recurrent neural networks (RNNs) because the characteristics of ULNs include the connection of multiple branches with arbitrary time delays. In this paper, therefore, RNNs with Multi-Branch structure are proposed and they show that their representation ability is better than conventional RNNs. RNNs can represent dynamical systems and are useful for time series prediction. The performance evaluation of RNNs with Multi-Branch structure was carried out using a benchmark of time series prediction. Simulation results showed that RNNs with Multi-Branch structure could obtain better performance than conventional RNNs, and also showed that they could improve the representation ability even if they are smaller sized networks.

AB - Universal Learning Networks (ULNs) provide a generalized framework to many kinds of structures of neural networks with supervised learning. Multi-Branch Neural Networks (MBNNs) which use the framework of ULNs have been already shown that they have better representation ability in feedforward neural networks (FNNs). Multi-Branch structure of MBNNs can be easily extended to recurrent neural networks (RNNs) because the characteristics of ULNs include the connection of multiple branches with arbitrary time delays. In this paper, therefore, RNNs with Multi-Branch structure are proposed and they show that their representation ability is better than conventional RNNs. RNNs can represent dynamical systems and are useful for time series prediction. The performance evaluation of RNNs with Multi-Branch structure was carried out using a benchmark of time series prediction. Simulation results showed that RNNs with Multi-Branch structure could obtain better performance than conventional RNNs, and also showed that they could improve the representation ability even if they are smaller sized networks.

KW - Multi-branch

KW - Recurrent neural networks

KW - Time series prediction

KW - Universal learning networks

UR - http://www.scopus.com/inward/record.url?scp=34548750304&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=34548750304&partnerID=8YFLogxK

U2 - 10.1541/ieejeiss.127.1430

DO - 10.1541/ieejeiss.127.1430

M3 - Article

AN - SCOPUS:34548750304

VL - 127

JO - IEEJ Transactions on Electronics, Information and Systems

JF - IEEJ Transactions on Electronics, Information and Systems

SN - 0385-4221

IS - 9

ER -