An accurate and robust algorithm for tracking guitar neck in 3D based on modified RANSAC homography

Zhao Wang, Jun Ohya

    Research output: Contribution to journalConference article

    Abstract

    Towards the actualization of an automatic guitar teaching system that can supervise guitar players, this paper proposes an algorithm for accurately and robustly tracking the 3D position of the fretboard from the video of guitar plays. First, we detect the SIFT features within the guitar fretboard and then match the detected points using KD-tree searching based matching algorithm frame by frame to track the whole fretboard. However, during the guitar plays, due to movements of the guitar neck or occlusions caused by guitar players' fingers, the feature points on the fretboard cannot always be matched accurately even though applying traditional RANSAC homography. Therefore, by using our modified RANSAC algorithm to filter out the matching error of the feature points, perspective transformation matrix is obtained between the correctly matched feature points detected at the first and other frames. Consequently, the guitar neck is tracked correctly based on the perspective transformation matrix. Experiments show promising results such as high accuracy: the total mean tracking error of only 4.17 mm and variance of 1.5 for the four tracked corners of the fretboard. This indicates the proposed method outperforms related tracking works including state-of-art Fully-convolutional Network.

    Original languageEnglish
    Article number460
    JournalIS and T International Symposium on Electronic Imaging Science and Technology
    VolumePart F138651
    DOIs
    Publication statusPublished - 2018 Jan 1
    Event3D Image Processing, Measurement (3DIPM), and Applications 2018 - Burlingame, United States
    Duration: 2018 Jan 282018 Feb 1

    Fingerprint

    occlusion
    Teaching
    education
    filters
    Experiments

    ASJC Scopus subject areas

    • Computer Graphics and Computer-Aided Design
    • Computer Science Applications
    • Human-Computer Interaction
    • Software
    • Electrical and Electronic Engineering
    • Atomic and Molecular Physics, and Optics

    Cite this

    @article{741ed119b338442a864ac6708c106c46,
    title = "An accurate and robust algorithm for tracking guitar neck in 3D based on modified RANSAC homography",
    abstract = "Towards the actualization of an automatic guitar teaching system that can supervise guitar players, this paper proposes an algorithm for accurately and robustly tracking the 3D position of the fretboard from the video of guitar plays. First, we detect the SIFT features within the guitar fretboard and then match the detected points using KD-tree searching based matching algorithm frame by frame to track the whole fretboard. However, during the guitar plays, due to movements of the guitar neck or occlusions caused by guitar players' fingers, the feature points on the fretboard cannot always be matched accurately even though applying traditional RANSAC homography. Therefore, by using our modified RANSAC algorithm to filter out the matching error of the feature points, perspective transformation matrix is obtained between the correctly matched feature points detected at the first and other frames. Consequently, the guitar neck is tracked correctly based on the perspective transformation matrix. Experiments show promising results such as high accuracy: the total mean tracking error of only 4.17 mm and variance of 1.5 for the four tracked corners of the fretboard. This indicates the proposed method outperforms related tracking works including state-of-art Fully-convolutional Network.",
    author = "Zhao Wang and Jun Ohya",
    year = "2018",
    month = "1",
    day = "1",
    doi = "10.2352/ISSN.2470-1173.2018.18.3DIPM-460",
    language = "English",
    volume = "Part F138651",
    journal = "IS and T International Symposium on Electronic Imaging Science and Technology",
    issn = "2470-1173",

    }

    TY - JOUR

    T1 - An accurate and robust algorithm for tracking guitar neck in 3D based on modified RANSAC homography

    AU - Wang, Zhao

    AU - Ohya, Jun

    PY - 2018/1/1

    Y1 - 2018/1/1

    N2 - Towards the actualization of an automatic guitar teaching system that can supervise guitar players, this paper proposes an algorithm for accurately and robustly tracking the 3D position of the fretboard from the video of guitar plays. First, we detect the SIFT features within the guitar fretboard and then match the detected points using KD-tree searching based matching algorithm frame by frame to track the whole fretboard. However, during the guitar plays, due to movements of the guitar neck or occlusions caused by guitar players' fingers, the feature points on the fretboard cannot always be matched accurately even though applying traditional RANSAC homography. Therefore, by using our modified RANSAC algorithm to filter out the matching error of the feature points, perspective transformation matrix is obtained between the correctly matched feature points detected at the first and other frames. Consequently, the guitar neck is tracked correctly based on the perspective transformation matrix. Experiments show promising results such as high accuracy: the total mean tracking error of only 4.17 mm and variance of 1.5 for the four tracked corners of the fretboard. This indicates the proposed method outperforms related tracking works including state-of-art Fully-convolutional Network.

    AB - Towards the actualization of an automatic guitar teaching system that can supervise guitar players, this paper proposes an algorithm for accurately and robustly tracking the 3D position of the fretboard from the video of guitar plays. First, we detect the SIFT features within the guitar fretboard and then match the detected points using KD-tree searching based matching algorithm frame by frame to track the whole fretboard. However, during the guitar plays, due to movements of the guitar neck or occlusions caused by guitar players' fingers, the feature points on the fretboard cannot always be matched accurately even though applying traditional RANSAC homography. Therefore, by using our modified RANSAC algorithm to filter out the matching error of the feature points, perspective transformation matrix is obtained between the correctly matched feature points detected at the first and other frames. Consequently, the guitar neck is tracked correctly based on the perspective transformation matrix. Experiments show promising results such as high accuracy: the total mean tracking error of only 4.17 mm and variance of 1.5 for the four tracked corners of the fretboard. This indicates the proposed method outperforms related tracking works including state-of-art Fully-convolutional Network.

    UR - http://www.scopus.com/inward/record.url?scp=85052872734&partnerID=8YFLogxK

    UR - http://www.scopus.com/inward/citedby.url?scp=85052872734&partnerID=8YFLogxK

    U2 - 10.2352/ISSN.2470-1173.2018.18.3DIPM-460

    DO - 10.2352/ISSN.2470-1173.2018.18.3DIPM-460

    M3 - Conference article

    AN - SCOPUS:85052872734

    VL - Part F138651

    JO - IS and T International Symposium on Electronic Imaging Science and Technology

    JF - IS and T International Symposium on Electronic Imaging Science and Technology

    SN - 2470-1173

    M1 - 460

    ER -