High-speed, pipelined implementation of squashing functions in neural networks

Liangwei Ge, Song Chen, Takeshi Yoshimura

研究成果: Conference contribution

抄録

Neural networks are powerful tool to simulate nonlinear systems. However, obtaining reliable neural networks is usually a time-consuming task, which requires repeated training of the networks with the available data. Recently, some attempts to accelerate the neural network training by utilizing paralleled hardware have been proposed. One of the challenges in hardware acceleration is implementing the floating-point squashing functions, like sigmoid(x) and tanh(x), that have vast input domain. However, previous implementations of squashing functions either suffer from low speed and poor accuracy or require large area and lots of manual works. In this paper, we present an automatic method to implement the squashing functions. Based on the proposed domain partition algorithm and coefficient compression method, squashing functions with smaller size, faster speed, and higher precision are obtained. Experiment on sigmoid(x) shows that less memory usage, up to 20k times smaller error rate, 300 times synthesis speedup, and 50% reduction of LUTs and flop-flops usage are achieved than conventional method.

元の言語English
ホスト出版物のタイトルInternational Conference on Solid-State and Integrated Circuits Technology Proceedings, ICSICT
ページ2204-2207
ページ数4
DOI
出版物ステータスPublished - 2008
イベント2008 9th International Conference on Solid-State and Integrated-Circuit Technology, ICSICT 2008 - Beijing
継続期間: 2008 10 202008 10 23

Other

Other2008 9th International Conference on Solid-State and Integrated-Circuit Technology, ICSICT 2008
Beijing
期間08/10/2008/10/23

Fingerprint

high speed
Neural networks
hardware
education
Hardware
nonlinear systems
floating
low speed
Nonlinear systems
partitions
Data storage equipment
coefficients
synthesis
Experiments

ASJC Scopus subject areas

  • Electrical and Electronic Engineering
  • Condensed Matter Physics
  • Electronic, Optical and Magnetic Materials

これを引用

Ge, L., Chen, S., & Yoshimura, T. (2008). High-speed, pipelined implementation of squashing functions in neural networks. : International Conference on Solid-State and Integrated Circuits Technology Proceedings, ICSICT (pp. 2204-2207). [4735008] https://doi.org/10.1109/ICSICT.2008.4735008

High-speed, pipelined implementation of squashing functions in neural networks. / Ge, Liangwei; Chen, Song; Yoshimura, Takeshi.

International Conference on Solid-State and Integrated Circuits Technology Proceedings, ICSICT. 2008. p. 2204-2207 4735008.

研究成果: Conference contribution

Ge, L, Chen, S & Yoshimura, T 2008, High-speed, pipelined implementation of squashing functions in neural networks. : International Conference on Solid-State and Integrated Circuits Technology Proceedings, ICSICT., 4735008, pp. 2204-2207, 2008 9th International Conference on Solid-State and Integrated-Circuit Technology, ICSICT 2008, Beijing, 08/10/20. https://doi.org/10.1109/ICSICT.2008.4735008
Ge L, Chen S, Yoshimura T. High-speed, pipelined implementation of squashing functions in neural networks. : International Conference on Solid-State and Integrated Circuits Technology Proceedings, ICSICT. 2008. p. 2204-2207. 4735008 https://doi.org/10.1109/ICSICT.2008.4735008
Ge, Liangwei ; Chen, Song ; Yoshimura, Takeshi. / High-speed, pipelined implementation of squashing functions in neural networks. International Conference on Solid-State and Integrated Circuits Technology Proceedings, ICSICT. 2008. pp. 2204-2207
@inproceedings{2fb168d6e927402f8a7aa1ce2f612d8a,
title = "High-speed, pipelined implementation of squashing functions in neural networks",
abstract = "Neural networks are powerful tool to simulate nonlinear systems. However, obtaining reliable neural networks is usually a time-consuming task, which requires repeated training of the networks with the available data. Recently, some attempts to accelerate the neural network training by utilizing paralleled hardware have been proposed. One of the challenges in hardware acceleration is implementing the floating-point squashing functions, like sigmoid(x) and tanh(x), that have vast input domain. However, previous implementations of squashing functions either suffer from low speed and poor accuracy or require large area and lots of manual works. In this paper, we present an automatic method to implement the squashing functions. Based on the proposed domain partition algorithm and coefficient compression method, squashing functions with smaller size, faster speed, and higher precision are obtained. Experiment on sigmoid(x) shows that less memory usage, up to 20k times smaller error rate, 300 times synthesis speedup, and 50{\%} reduction of LUTs and flop-flops usage are achieved than conventional method.",
author = "Liangwei Ge and Song Chen and Takeshi Yoshimura",
year = "2008",
doi = "10.1109/ICSICT.2008.4735008",
language = "English",
isbn = "9781424421855",
pages = "2204--2207",
booktitle = "International Conference on Solid-State and Integrated Circuits Technology Proceedings, ICSICT",

}

TY - GEN

T1 - High-speed, pipelined implementation of squashing functions in neural networks

AU - Ge, Liangwei

AU - Chen, Song

AU - Yoshimura, Takeshi

PY - 2008

Y1 - 2008

N2 - Neural networks are powerful tool to simulate nonlinear systems. However, obtaining reliable neural networks is usually a time-consuming task, which requires repeated training of the networks with the available data. Recently, some attempts to accelerate the neural network training by utilizing paralleled hardware have been proposed. One of the challenges in hardware acceleration is implementing the floating-point squashing functions, like sigmoid(x) and tanh(x), that have vast input domain. However, previous implementations of squashing functions either suffer from low speed and poor accuracy or require large area and lots of manual works. In this paper, we present an automatic method to implement the squashing functions. Based on the proposed domain partition algorithm and coefficient compression method, squashing functions with smaller size, faster speed, and higher precision are obtained. Experiment on sigmoid(x) shows that less memory usage, up to 20k times smaller error rate, 300 times synthesis speedup, and 50% reduction of LUTs and flop-flops usage are achieved than conventional method.

AB - Neural networks are powerful tool to simulate nonlinear systems. However, obtaining reliable neural networks is usually a time-consuming task, which requires repeated training of the networks with the available data. Recently, some attempts to accelerate the neural network training by utilizing paralleled hardware have been proposed. One of the challenges in hardware acceleration is implementing the floating-point squashing functions, like sigmoid(x) and tanh(x), that have vast input domain. However, previous implementations of squashing functions either suffer from low speed and poor accuracy or require large area and lots of manual works. In this paper, we present an automatic method to implement the squashing functions. Based on the proposed domain partition algorithm and coefficient compression method, squashing functions with smaller size, faster speed, and higher precision are obtained. Experiment on sigmoid(x) shows that less memory usage, up to 20k times smaller error rate, 300 times synthesis speedup, and 50% reduction of LUTs and flop-flops usage are achieved than conventional method.

UR - http://www.scopus.com/inward/record.url?scp=60649083423&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=60649083423&partnerID=8YFLogxK

U2 - 10.1109/ICSICT.2008.4735008

DO - 10.1109/ICSICT.2008.4735008

M3 - Conference contribution

AN - SCOPUS:60649083423

SN - 9781424421855

SP - 2204

EP - 2207

BT - International Conference on Solid-State and Integrated Circuits Technology Proceedings, ICSICT

ER -