A 3D guitar fingering assessing system based on CNN-Hand pose estimation and SVR-Assessment

Zhao Wang, Jun Ohya

    Research output: Contribution to journalConference article

    Abstract

    This paper proposes a guitar fingering assessing system based on CNN (Convolutional Neural Network) hand pose estimation and SVR (Support Vector Regression) evaluation. To spur our progress, first, a CNN architecture is proposed to estimate temporal 3D position of 16 joints of hand; then, based on a DCT (Discrete Cosine Transform) feature and SVR, fingering of guitarist is scored to interpret how well guitarist played. We also release a new dataset for professional guitar playing analysis with significant advantage in total number of video, professional judgement by expert of guitarist, accurate annotation for hand pose and score of guitar performance. Experiments using videos containing multiple persons' guitar plays under different conditions demonstrate that the proposed method outperforms the current state-of-art with (1) low mean error (Euclid distance of 6,1 mm) and high computation efficiency for hand pose estimation; (2) high rank correlation (0.68) for assessing the fingering (C major scale and symmetrical excise) of guitarists.

    Original languageEnglish
    Pages (from-to)2781-2785
    Number of pages5
    JournalIS and T International Symposium on Electronic Imaging Science and Technology
    VolumePart F138660
    DOIs
    Publication statusPublished - 2018 Jan 1
    EventIntelligent Robotics and Industrial Applications using Computer Vision 2018, IRIACV 2018 - Burlingame, United States
    Duration: 2018 Jan 282018 Feb 1

    ASJC Scopus subject areas

    • Computer Graphics and Computer-Aided Design
    • Computer Science Applications
    • Human-Computer Interaction
    • Software
    • Electrical and Electronic Engineering
    • Atomic and Molecular Physics, and Optics

    Fingerprint Dive into the research topics of 'A 3D guitar fingering assessing system based on CNN-Hand pose estimation and SVR-Assessment'. Together they form a unique fingerprint.

  • Cite this