A Zero-Gating Processing Element Design for Low-Power Deep Convolutional Neural Networks

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Convolution neural networks (CNNs) have shown great success in many areas such as object detection and pattern recognition. However, the high computational complexity of state-of-the-art deep CNNs makes them extreme difficult to be run on resource-constrained mobile and wearable devices. To address this design challenge, in this paper we first analyzed the filters' weights of pre-trained models from four state-of-the-art CNNs. We found that in all the CNNs that we analyzed, from about 20% (AlexNet) to 43% (VGG-19) of the weights are zeros, which lead to redundant large amounts of computation. Then, based on this observation, a zero-gating processing element (PE) design was proposed for low-power deep CNNs, in which the vast number of zeros in both activation maps and filter weights are explored to eliminate redundant computation for power reduction. We implemented our proposal with VGG-16 using ImageNet dataset. Experiments were conducted for evaluations of area and total power consumption. Compared with the baseline PE design without zero-gating, overall the proposed zero-gating PE can achieve 37% power saving while the corresponding area overhead is less than 8%.

Original languageEnglish
Title of host publicationProceedings - APCCAS 2019
Subtitle of host publication2019 IEEE Asia Pacific Conference on Circuits and Systems: Innovative CAS Towards Sustainable Energy and Technology Disruption
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages317-320
Number of pages4
ISBN (Electronic)9781728129402
DOIs
Publication statusPublished - 2019 Nov
Event15th Annual IEEE Asia Pacific Conference on Circuits and Systems, APCCAS 2019 - Bangkok, Thailand
Duration: 2019 Nov 112019 Nov 14

Publication series

NameProceedings - APCCAS 2019: 2019 IEEE Asia Pacific Conference on Circuits and Systems: Innovative CAS Towards Sustainable Energy and Technology Disruption

Conference

Conference15th Annual IEEE Asia Pacific Conference on Circuits and Systems, APCCAS 2019
CountryThailand
CityBangkok
Period19/11/1119/11/14

Keywords

  • activation map
  • CNNs
  • filter
  • power consumption
  • zero-gating

ASJC Scopus subject areas

  • Energy Engineering and Power Technology
  • Renewable Energy, Sustainability and the Environment
  • Electrical and Electronic Engineering
  • Electronic, Optical and Magnetic Materials
  • Instrumentation

Fingerprint Dive into the research topics of 'A Zero-Gating Processing Element Design for Low-Power Deep Convolutional Neural Networks'. Together they form a unique fingerprint.

  • Cite this

    Ye, L., Ye, J., Yanagisawa, M., & Shi, Y. (2019). A Zero-Gating Processing Element Design for Low-Power Deep Convolutional Neural Networks. In Proceedings - APCCAS 2019: 2019 IEEE Asia Pacific Conference on Circuits and Systems: Innovative CAS Towards Sustainable Energy and Technology Disruption (pp. 317-320). [8953157] (Proceedings - APCCAS 2019: 2019 IEEE Asia Pacific Conference on Circuits and Systems: Innovative CAS Towards Sustainable Energy and Technology Disruption). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/APCCAS47518.2019.8953157