Full-HD 60 fps FPGA implementation of spatio-Temporal keypoint extraction based on gradient histogram and parallelization of keypoint connectivity

Takahiro Suzuki, Takeshi Ikenaga

研究成果: Article

1 引用 (Scopus)

抄録

Recently, cloud systems have started to be utilized for services which analyze user's data in the field of computer vision. In these services, keypoints are extracted from images or videos, and the data is identified by machine learning with a large database in the cloud. To reduce the number of keypoints which are sent to the cloud, Keypoints of Interest (KOI) extraction has been proposed. However, since its computational complexity is large, hardware implementation is required for realtime processing. Moreover, the hardware resource must be low because it is embedded in devices of users. This paper proposes a hardware-friendly KOI algorithm with low amount of computations and its real-Time hardware implementation based on dual threshold keypoint detection by gradient histogram and parallelization of connectivity of adjacent keypointutilizing register counters. The algorithm utilizes dual-histogram based detection and keypoint-matching based calculation of motion information and dense-clustering based keypoint smoothing. The hardware architecture is composed of a detection module utilizing descriptor, and grid-regionparallelization based density clustering. Finally, the evaluation results of hardware implementation show that the implemented hardware achieves Full-HD (1920x1080)-60 fps spatio-Temporal keypoint extraction. Further, it is 47 times faster than low complexity keypoint extraction on software and 12 times faster than spatio-Temporal keypoint extraction on software, and the hardware resources are almost the same as SIFT hardware implementation, maintaining accuracy.

元の言語English
ページ(範囲)1937-1946
ページ数10
ジャーナルIEICE Transactions on Fundamentals of Electronics, Communications and Computer Sciences
E99A
発行部数11
DOI
出版物ステータスPublished - 2016 11 1

Fingerprint

FPGA Implementation
Hardware Implementation
Parallelization
Histogram
Field programmable gate arrays (FPGA)
Connectivity
Hardware
Gradient
Clustering
Real-time Processing
Dual Algorithm
Resources
Software
Hardware Architecture
Scale Invariant Feature Transform
Computer Vision
Low Complexity
Descriptors
Smoothing
Machine Learning

ASJC Scopus subject areas

  • Signal Processing
  • Computer Graphics and Computer-Aided Design
  • Applied Mathematics
  • Electrical and Electronic Engineering

これを引用

@article{146ee007f35b468caa7388d1c9349fd1,
title = "Full-HD 60 fps FPGA implementation of spatio-Temporal keypoint extraction based on gradient histogram and parallelization of keypoint connectivity",
abstract = "Recently, cloud systems have started to be utilized for services which analyze user's data in the field of computer vision. In these services, keypoints are extracted from images or videos, and the data is identified by machine learning with a large database in the cloud. To reduce the number of keypoints which are sent to the cloud, Keypoints of Interest (KOI) extraction has been proposed. However, since its computational complexity is large, hardware implementation is required for realtime processing. Moreover, the hardware resource must be low because it is embedded in devices of users. This paper proposes a hardware-friendly KOI algorithm with low amount of computations and its real-Time hardware implementation based on dual threshold keypoint detection by gradient histogram and parallelization of connectivity of adjacent keypointutilizing register counters. The algorithm utilizes dual-histogram based detection and keypoint-matching based calculation of motion information and dense-clustering based keypoint smoothing. The hardware architecture is composed of a detection module utilizing descriptor, and grid-regionparallelization based density clustering. Finally, the evaluation results of hardware implementation show that the implemented hardware achieves Full-HD (1920x1080)-60 fps spatio-Temporal keypoint extraction. Further, it is 47 times faster than low complexity keypoint extraction on software and 12 times faster than spatio-Temporal keypoint extraction on software, and the hardware resources are almost the same as SIFT hardware implementation, maintaining accuracy.",
keywords = "Cloud, Hardware Architecture, Keypoint Extraction, SIFT, Video Recognition",
author = "Takahiro Suzuki and Takeshi Ikenaga",
year = "2016",
month = "11",
day = "1",
doi = "10.1587/transfun.E99.A.1937",
language = "English",
volume = "E99A",
pages = "1937--1946",
journal = "IEICE Transactions on Fundamentals of Electronics, Communications and Computer Sciences",
issn = "0916-8508",
publisher = "Maruzen Co., Ltd/Maruzen Kabushikikaisha",
number = "11",

}

TY - JOUR

T1 - Full-HD 60 fps FPGA implementation of spatio-Temporal keypoint extraction based on gradient histogram and parallelization of keypoint connectivity

AU - Suzuki, Takahiro

AU - Ikenaga, Takeshi

PY - 2016/11/1

Y1 - 2016/11/1

N2 - Recently, cloud systems have started to be utilized for services which analyze user's data in the field of computer vision. In these services, keypoints are extracted from images or videos, and the data is identified by machine learning with a large database in the cloud. To reduce the number of keypoints which are sent to the cloud, Keypoints of Interest (KOI) extraction has been proposed. However, since its computational complexity is large, hardware implementation is required for realtime processing. Moreover, the hardware resource must be low because it is embedded in devices of users. This paper proposes a hardware-friendly KOI algorithm with low amount of computations and its real-Time hardware implementation based on dual threshold keypoint detection by gradient histogram and parallelization of connectivity of adjacent keypointutilizing register counters. The algorithm utilizes dual-histogram based detection and keypoint-matching based calculation of motion information and dense-clustering based keypoint smoothing. The hardware architecture is composed of a detection module utilizing descriptor, and grid-regionparallelization based density clustering. Finally, the evaluation results of hardware implementation show that the implemented hardware achieves Full-HD (1920x1080)-60 fps spatio-Temporal keypoint extraction. Further, it is 47 times faster than low complexity keypoint extraction on software and 12 times faster than spatio-Temporal keypoint extraction on software, and the hardware resources are almost the same as SIFT hardware implementation, maintaining accuracy.

AB - Recently, cloud systems have started to be utilized for services which analyze user's data in the field of computer vision. In these services, keypoints are extracted from images or videos, and the data is identified by machine learning with a large database in the cloud. To reduce the number of keypoints which are sent to the cloud, Keypoints of Interest (KOI) extraction has been proposed. However, since its computational complexity is large, hardware implementation is required for realtime processing. Moreover, the hardware resource must be low because it is embedded in devices of users. This paper proposes a hardware-friendly KOI algorithm with low amount of computations and its real-Time hardware implementation based on dual threshold keypoint detection by gradient histogram and parallelization of connectivity of adjacent keypointutilizing register counters. The algorithm utilizes dual-histogram based detection and keypoint-matching based calculation of motion information and dense-clustering based keypoint smoothing. The hardware architecture is composed of a detection module utilizing descriptor, and grid-regionparallelization based density clustering. Finally, the evaluation results of hardware implementation show that the implemented hardware achieves Full-HD (1920x1080)-60 fps spatio-Temporal keypoint extraction. Further, it is 47 times faster than low complexity keypoint extraction on software and 12 times faster than spatio-Temporal keypoint extraction on software, and the hardware resources are almost the same as SIFT hardware implementation, maintaining accuracy.

KW - Cloud

KW - Hardware Architecture

KW - Keypoint Extraction

KW - SIFT

KW - Video Recognition

UR - http://www.scopus.com/inward/record.url?scp=84994559223&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84994559223&partnerID=8YFLogxK

U2 - 10.1587/transfun.E99.A.1937

DO - 10.1587/transfun.E99.A.1937

M3 - Article

AN - SCOPUS:84994559223

VL - E99A

SP - 1937

EP - 1946

JO - IEICE Transactions on Fundamentals of Electronics, Communications and Computer Sciences

JF - IEICE Transactions on Fundamentals of Electronics, Communications and Computer Sciences

SN - 0916-8508

IS - 11

ER -