A 3D guitar fingering assessing system based on CNN-Hand pose estimation and SVR-Assessment

Zhao Wang*, Jun Ohya

*この研究の対応する著者

    研究成果: Conference article査読

    抄録

    This paper proposes a guitar fingering assessing system based on CNN (Convolutional Neural Network) hand pose estimation and SVR (Support Vector Regression) evaluation. To spur our progress, first, a CNN architecture is proposed to estimate temporal 3D position of 16 joints of hand; then, based on a DCT (Discrete Cosine Transform) feature and SVR, fingering of guitarist is scored to interpret how well guitarist played. We also release a new dataset for professional guitar playing analysis with significant advantage in total number of video, professional judgement by expert of guitarist, accurate annotation for hand pose and score of guitar performance. Experiments using videos containing multiple persons' guitar plays under different conditions demonstrate that the proposed method outperforms the current state-of-art with (1) low mean error (Euclid distance of 6,1 mm) and high computation efficiency for hand pose estimation; (2) high rank correlation (0.68) for assessing the fingering (C major scale and symmetrical excise) of guitarists.

    本文言語English
    ページ(範囲)2781-2785
    ページ数5
    ジャーナルIS and T International Symposium on Electronic Imaging Science and Technology
    Part F138660
    DOI
    出版ステータスPublished - 2018 1 1
    イベントIntelligent Robotics and Industrial Applications using Computer Vision 2018, IRIACV 2018 - Burlingame, United States
    継続期間: 2018 1 282018 2 1

    ASJC Scopus subject areas

    • コンピュータ グラフィックスおよびコンピュータ支援設計
    • コンピュータ サイエンスの応用
    • 人間とコンピュータの相互作用
    • ソフトウェア
    • 電子工学および電気工学
    • 原子分子物理学および光学

    フィンガープリント

    「A 3D guitar fingering assessing system based on CNN-Hand pose estimation and SVR-Assessment」の研究トピックを掘り下げます。これらがまとまってユニークなフィンガープリントを構成します。

    引用スタイル