Resolving occlusion for 3D object manipulation with hands in mixed reality

Qi Feng, Hubert P.H. Shum, Shigeo Morishima

    Research output: Chapter in Book/Report/Conference proceedingConference contribution

    Abstract

    Due to the need to interact with virtual objects, the hand-object interaction has become an important element in mixed reality (MR) applications. In this paper, we propose a novel approach to handle the occlusion of augmented 3D object manipulation with hands by exploiting the nature of hand poses combined with tracking-based and model-based methods, to achieve a complete mixed reality experience without necessities of heavy computations, complex manual segmentation processes or wearing special gloves. The experimental results show a frame rate faster than real-time and a great accuracy of rendered virtual appearances, and a user study verifies a more immersive experience compared to past approaches. We believe that the proposed method can improve a wide range of mixed reality applications that involve hand-object interactions.

    Original languageEnglish
    Title of host publicationProceedings - VRST 2018
    Subtitle of host publication24th ACM Symposium on Virtual Reality Software and Technology
    EditorsStephen N. Spencer
    PublisherAssociation for Computing Machinery
    ISBN (Electronic)9781450360869
    DOIs
    Publication statusPublished - 2018 Nov 28
    Event24th ACM Symposium on Virtual Reality Software and Technology, VRST 2018 - Tokyo, Japan
    Duration: 2018 Nov 282018 Dec 1

    Other

    Other24th ACM Symposium on Virtual Reality Software and Technology, VRST 2018
    CountryJapan
    CityTokyo
    Period18/11/2818/12/1

    Keywords

    • Hand tracking
    • Mixed Reality
    • Occlusion

    ASJC Scopus subject areas

    • Software

    Cite this

    Feng, Q., Shum, H. P. H., & Morishima, S. (2018). Resolving occlusion for 3D object manipulation with hands in mixed reality. In S. N. Spencer (Ed.), Proceedings - VRST 2018: 24th ACM Symposium on Virtual Reality Software and Technology [3283390] Association for Computing Machinery. https://doi.org/10.1145/3281505.3283390

    Resolving occlusion for 3D object manipulation with hands in mixed reality. / Feng, Qi; Shum, Hubert P.H.; Morishima, Shigeo.

    Proceedings - VRST 2018: 24th ACM Symposium on Virtual Reality Software and Technology. ed. / Stephen N. Spencer. Association for Computing Machinery, 2018. 3283390.

    Research output: Chapter in Book/Report/Conference proceedingConference contribution

    Feng, Q, Shum, HPH & Morishima, S 2018, Resolving occlusion for 3D object manipulation with hands in mixed reality. in SN Spencer (ed.), Proceedings - VRST 2018: 24th ACM Symposium on Virtual Reality Software and Technology., 3283390, Association for Computing Machinery, 24th ACM Symposium on Virtual Reality Software and Technology, VRST 2018, Tokyo, Japan, 18/11/28. https://doi.org/10.1145/3281505.3283390
    Feng Q, Shum HPH, Morishima S. Resolving occlusion for 3D object manipulation with hands in mixed reality. In Spencer SN, editor, Proceedings - VRST 2018: 24th ACM Symposium on Virtual Reality Software and Technology. Association for Computing Machinery. 2018. 3283390 https://doi.org/10.1145/3281505.3283390
    Feng, Qi ; Shum, Hubert P.H. ; Morishima, Shigeo. / Resolving occlusion for 3D object manipulation with hands in mixed reality. Proceedings - VRST 2018: 24th ACM Symposium on Virtual Reality Software and Technology. editor / Stephen N. Spencer. Association for Computing Machinery, 2018.
    @inproceedings{37e6723073c6496ca3619f391ac41168,
    title = "Resolving occlusion for 3D object manipulation with hands in mixed reality",
    abstract = "Due to the need to interact with virtual objects, the hand-object interaction has become an important element in mixed reality (MR) applications. In this paper, we propose a novel approach to handle the occlusion of augmented 3D object manipulation with hands by exploiting the nature of hand poses combined with tracking-based and model-based methods, to achieve a complete mixed reality experience without necessities of heavy computations, complex manual segmentation processes or wearing special gloves. The experimental results show a frame rate faster than real-time and a great accuracy of rendered virtual appearances, and a user study verifies a more immersive experience compared to past approaches. We believe that the proposed method can improve a wide range of mixed reality applications that involve hand-object interactions.",
    keywords = "Hand tracking, Mixed Reality, Occlusion",
    author = "Qi Feng and Shum, {Hubert P.H.} and Shigeo Morishima",
    year = "2018",
    month = "11",
    day = "28",
    doi = "10.1145/3281505.3283390",
    language = "English",
    editor = "Spencer, {Stephen N.}",
    booktitle = "Proceedings - VRST 2018",
    publisher = "Association for Computing Machinery",

    }

    TY - GEN

    T1 - Resolving occlusion for 3D object manipulation with hands in mixed reality

    AU - Feng, Qi

    AU - Shum, Hubert P.H.

    AU - Morishima, Shigeo

    PY - 2018/11/28

    Y1 - 2018/11/28

    N2 - Due to the need to interact with virtual objects, the hand-object interaction has become an important element in mixed reality (MR) applications. In this paper, we propose a novel approach to handle the occlusion of augmented 3D object manipulation with hands by exploiting the nature of hand poses combined with tracking-based and model-based methods, to achieve a complete mixed reality experience without necessities of heavy computations, complex manual segmentation processes or wearing special gloves. The experimental results show a frame rate faster than real-time and a great accuracy of rendered virtual appearances, and a user study verifies a more immersive experience compared to past approaches. We believe that the proposed method can improve a wide range of mixed reality applications that involve hand-object interactions.

    AB - Due to the need to interact with virtual objects, the hand-object interaction has become an important element in mixed reality (MR) applications. In this paper, we propose a novel approach to handle the occlusion of augmented 3D object manipulation with hands by exploiting the nature of hand poses combined with tracking-based and model-based methods, to achieve a complete mixed reality experience without necessities of heavy computations, complex manual segmentation processes or wearing special gloves. The experimental results show a frame rate faster than real-time and a great accuracy of rendered virtual appearances, and a user study verifies a more immersive experience compared to past approaches. We believe that the proposed method can improve a wide range of mixed reality applications that involve hand-object interactions.

    KW - Hand tracking

    KW - Mixed Reality

    KW - Occlusion

    UR - http://www.scopus.com/inward/record.url?scp=85060914303&partnerID=8YFLogxK

    UR - http://www.scopus.com/inward/citedby.url?scp=85060914303&partnerID=8YFLogxK

    U2 - 10.1145/3281505.3283390

    DO - 10.1145/3281505.3283390

    M3 - Conference contribution

    AN - SCOPUS:85060914303

    BT - Proceedings - VRST 2018

    A2 - Spencer, Stephen N.

    PB - Association for Computing Machinery

    ER -