An algorithm for tracking guitarists' fingertips based on CNN-segmentation and ROI associated particle filter

Zhao Wang, Jun Ohya

Research output: Contribution to journalArticle

Abstract

In this paper, we propose a new approach to tracking the fingertips of guitarists by embedding a CNN-based segmentation module and a temporal grouping-based ROI-association module combined with a particle filter. First, a CNN architecture is trained to segment hand area of each frame of input video. Then, four fingertip candidates (fore, middle, ring and little fingertips) on each frame are located by counting the vote number of template matching (TM) and reversed Hough transform (RHT). Furthermore, temporal grouping-based ROI association is applied to removal noise and group the fingertip candidates on consecutive frames. Finally, particles are distributed between associated fingertip candidates on every two adjacent frames for tracking the fingertips of guitarists. Experiments using videos containing multiple persons' guitar plays under different conditions demonstrate that the proposed method outperforms the current state-of-the-art tracking algorithm in terms of the hand area segmentation accuracy (98%) and the fingertip tracking mean error (5.16 pixel: 0.22 cm on the guitar neck) as well as computation efficiency.

Original languageEnglish
Article number020506-1
JournalJournal of Imaging Science and Technology
Volume63
Issue number2
DOIs
Publication statusPublished - 2019 Jan 1

Fingerprint

Template matching
Hough transforms
Pixels
filters
modules
Experiments
embedding
counting
templates
pixels
rings

ASJC Scopus subject areas

  • Electronic, Optical and Magnetic Materials
  • Chemistry(all)
  • Atomic and Molecular Physics, and Optics
  • Computer Science Applications

Cite this

An algorithm for tracking guitarists' fingertips based on CNN-segmentation and ROI associated particle filter. / Wang, Zhao; Ohya, Jun.

In: Journal of Imaging Science and Technology, Vol. 63, No. 2, 020506-1, 01.01.2019.

Research output: Contribution to journalArticle

@article{6b4142a5af4f437aa7cb375c7593aa00,
title = "An algorithm for tracking guitarists' fingertips based on CNN-segmentation and ROI associated particle filter",
abstract = "In this paper, we propose a new approach to tracking the fingertips of guitarists by embedding a CNN-based segmentation module and a temporal grouping-based ROI-association module combined with a particle filter. First, a CNN architecture is trained to segment hand area of each frame of input video. Then, four fingertip candidates (fore, middle, ring and little fingertips) on each frame are located by counting the vote number of template matching (TM) and reversed Hough transform (RHT). Furthermore, temporal grouping-based ROI association is applied to removal noise and group the fingertip candidates on consecutive frames. Finally, particles are distributed between associated fingertip candidates on every two adjacent frames for tracking the fingertips of guitarists. Experiments using videos containing multiple persons' guitar plays under different conditions demonstrate that the proposed method outperforms the current state-of-the-art tracking algorithm in terms of the hand area segmentation accuracy (98{\%}) and the fingertip tracking mean error (5.16 pixel: 0.22 cm on the guitar neck) as well as computation efficiency.",
author = "Zhao Wang and Jun Ohya",
year = "2019",
month = "1",
day = "1",
doi = "10.2352/J.ImagingSci.Technol.2019.63.2.020506",
language = "English",
volume = "63",
journal = "Journal of Imaging Science and Technology",
issn = "1062-3701",
publisher = "Society for Imaging Science and Technology",
number = "2",

}

TY - JOUR

T1 - An algorithm for tracking guitarists' fingertips based on CNN-segmentation and ROI associated particle filter

AU - Wang, Zhao

AU - Ohya, Jun

PY - 2019/1/1

Y1 - 2019/1/1

N2 - In this paper, we propose a new approach to tracking the fingertips of guitarists by embedding a CNN-based segmentation module and a temporal grouping-based ROI-association module combined with a particle filter. First, a CNN architecture is trained to segment hand area of each frame of input video. Then, four fingertip candidates (fore, middle, ring and little fingertips) on each frame are located by counting the vote number of template matching (TM) and reversed Hough transform (RHT). Furthermore, temporal grouping-based ROI association is applied to removal noise and group the fingertip candidates on consecutive frames. Finally, particles are distributed between associated fingertip candidates on every two adjacent frames for tracking the fingertips of guitarists. Experiments using videos containing multiple persons' guitar plays under different conditions demonstrate that the proposed method outperforms the current state-of-the-art tracking algorithm in terms of the hand area segmentation accuracy (98%) and the fingertip tracking mean error (5.16 pixel: 0.22 cm on the guitar neck) as well as computation efficiency.

AB - In this paper, we propose a new approach to tracking the fingertips of guitarists by embedding a CNN-based segmentation module and a temporal grouping-based ROI-association module combined with a particle filter. First, a CNN architecture is trained to segment hand area of each frame of input video. Then, four fingertip candidates (fore, middle, ring and little fingertips) on each frame are located by counting the vote number of template matching (TM) and reversed Hough transform (RHT). Furthermore, temporal grouping-based ROI association is applied to removal noise and group the fingertip candidates on consecutive frames. Finally, particles are distributed between associated fingertip candidates on every two adjacent frames for tracking the fingertips of guitarists. Experiments using videos containing multiple persons' guitar plays under different conditions demonstrate that the proposed method outperforms the current state-of-the-art tracking algorithm in terms of the hand area segmentation accuracy (98%) and the fingertip tracking mean error (5.16 pixel: 0.22 cm on the guitar neck) as well as computation efficiency.

UR - http://www.scopus.com/inward/record.url?scp=85074131450&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85074131450&partnerID=8YFLogxK

U2 - 10.2352/J.ImagingSci.Technol.2019.63.2.020506

DO - 10.2352/J.ImagingSci.Technol.2019.63.2.020506

M3 - Article

AN - SCOPUS:85074131450

VL - 63

JO - Journal of Imaging Science and Technology

JF - Journal of Imaging Science and Technology

SN - 1062-3701

IS - 2

M1 - 020506-1

ER -