A 3D guitar fingering assessing system based on CNN-Hand pose estimation and SVR-Assessment

Zhao Wang, Jun Ohya

    研究成果: Conference article

    抄録

    This paper proposes a guitar fingering assessing system based on CNN (Convolutional Neural Network) hand pose estimation and SVR (Support Vector Regression) evaluation. To spur our progress, first, a CNN architecture is proposed to estimate temporal 3D position of 16 joints of hand; then, based on a DCT (Discrete Cosine Transform) feature and SVR, fingering of guitarist is scored to interpret how well guitarist played. We also release a new dataset for professional guitar playing analysis with significant advantage in total number of video, professional judgement by expert of guitarist, accurate annotation for hand pose and score of guitar performance. Experiments using videos containing multiple persons' guitar plays under different conditions demonstrate that the proposed method outperforms the current state-of-art with (1) low mean error (Euclid distance of 6,1 mm) and high computation efficiency for hand pose estimation; (2) high rank correlation (0.68) for assessing the fingering (C major scale and symmetrical excise) of guitarists.

    元の言語English
    ページ(範囲)2781-2785
    ページ数5
    ジャーナルIS and T International Symposium on Electronic Imaging Science and Technology
    Part F138660
    DOI
    出版物ステータスPublished - 2018 1 1
    イベントIntelligent Robotics and Industrial Applications using Computer Vision 2018, IRIACV 2018 - Burlingame, United States
    継続期間: 2018 1 282018 2 1

    Fingerprint

    regression analysis
    Neural networks
    annotations
    discrete cosine transform
    Discrete cosine transforms
    Network architecture
    evaluation
    estimates
    Experiments

    ASJC Scopus subject areas

    • Computer Graphics and Computer-Aided Design
    • Computer Science Applications
    • Human-Computer Interaction
    • Software
    • Electrical and Electronic Engineering
    • Atomic and Molecular Physics, and Optics

    これを引用

    @article{8837e0d777a0459c807dd9f8d672889d,
    title = "A 3D guitar fingering assessing system based on CNN-Hand pose estimation and SVR-Assessment",
    abstract = "This paper proposes a guitar fingering assessing system based on CNN (Convolutional Neural Network) hand pose estimation and SVR (Support Vector Regression) evaluation. To spur our progress, first, a CNN architecture is proposed to estimate temporal 3D position of 16 joints of hand; then, based on a DCT (Discrete Cosine Transform) feature and SVR, fingering of guitarist is scored to interpret how well guitarist played. We also release a new dataset for professional guitar playing analysis with significant advantage in total number of video, professional judgement by expert of guitarist, accurate annotation for hand pose and score of guitar performance. Experiments using videos containing multiple persons' guitar plays under different conditions demonstrate that the proposed method outperforms the current state-of-art with (1) low mean error (Euclid distance of 6,1 mm) and high computation efficiency for hand pose estimation; (2) high rank correlation (0.68) for assessing the fingering (C major scale and symmetrical excise) of guitarists.",
    author = "Zhao Wang and Jun Ohya",
    year = "2018",
    month = "1",
    day = "1",
    doi = "10.2352/ISSN.2470-1173.2018.09.IRIACV-204",
    language = "English",
    volume = "Part F138660",
    pages = "2781--2785",
    journal = "IS and T International Symposium on Electronic Imaging Science and Technology",
    issn = "2470-1173",

    }

    TY - JOUR

    T1 - A 3D guitar fingering assessing system based on CNN-Hand pose estimation and SVR-Assessment

    AU - Wang, Zhao

    AU - Ohya, Jun

    PY - 2018/1/1

    Y1 - 2018/1/1

    N2 - This paper proposes a guitar fingering assessing system based on CNN (Convolutional Neural Network) hand pose estimation and SVR (Support Vector Regression) evaluation. To spur our progress, first, a CNN architecture is proposed to estimate temporal 3D position of 16 joints of hand; then, based on a DCT (Discrete Cosine Transform) feature and SVR, fingering of guitarist is scored to interpret how well guitarist played. We also release a new dataset for professional guitar playing analysis with significant advantage in total number of video, professional judgement by expert of guitarist, accurate annotation for hand pose and score of guitar performance. Experiments using videos containing multiple persons' guitar plays under different conditions demonstrate that the proposed method outperforms the current state-of-art with (1) low mean error (Euclid distance of 6,1 mm) and high computation efficiency for hand pose estimation; (2) high rank correlation (0.68) for assessing the fingering (C major scale and symmetrical excise) of guitarists.

    AB - This paper proposes a guitar fingering assessing system based on CNN (Convolutional Neural Network) hand pose estimation and SVR (Support Vector Regression) evaluation. To spur our progress, first, a CNN architecture is proposed to estimate temporal 3D position of 16 joints of hand; then, based on a DCT (Discrete Cosine Transform) feature and SVR, fingering of guitarist is scored to interpret how well guitarist played. We also release a new dataset for professional guitar playing analysis with significant advantage in total number of video, professional judgement by expert of guitarist, accurate annotation for hand pose and score of guitar performance. Experiments using videos containing multiple persons' guitar plays under different conditions demonstrate that the proposed method outperforms the current state-of-art with (1) low mean error (Euclid distance of 6,1 mm) and high computation efficiency for hand pose estimation; (2) high rank correlation (0.68) for assessing the fingering (C major scale and symmetrical excise) of guitarists.

    UR - http://www.scopus.com/inward/record.url?scp=85052905106&partnerID=8YFLogxK

    UR - http://www.scopus.com/inward/citedby.url?scp=85052905106&partnerID=8YFLogxK

    U2 - 10.2352/ISSN.2470-1173.2018.09.IRIACV-204

    DO - 10.2352/ISSN.2470-1173.2018.09.IRIACV-204

    M3 - Conference article

    AN - SCOPUS:85052905106

    VL - Part F138660

    SP - 2781

    EP - 2785

    JO - IS and T International Symposium on Electronic Imaging Science and Technology

    JF - IS and T International Symposium on Electronic Imaging Science and Technology

    SN - 2470-1173

    ER -