Augmented Cross-modality: Translating the Physiological Responses, Knowledge and Impression to Audio-visual Information in Virtual Reality

Yutaro Hirao, Takashi Kawai

    Research output: Contribution to journalArticle

    1 Citation (Scopus)

    Abstract

    This paper proposes the method of interaction design to present haptic experience as intended in virtual reality (VR). The method that we named “Augmented Cross-Modality” is to translate the physiological responses, knowledge and impression about the experience in real world into audio-visual stimuli and add them to the interaction in VR. In this study, as expressions for presenting a haptic experience of gripping an object strongly and lifting a heavy object, we design hand tremor, strong gripping and increasing heart rate in VR. The objective is, at first, to enhance a sense of strain of a body with these augmented cross-modal expressions and then, change the quality of the total haptic experience and as a result, make it closer to the experience of lifting a heavy object. This method is evaluated by several rating scales, interviews and force sensors attached to a VR controller. The result suggests that the expressions of this method enhancing a haptic experience of strong gripping in almost all participants and the effectiveness were confirmed. c 2018 Society for Imaging Science and Technology.

    Original languageEnglish
    Article number060402
    JournalJournal of Imaging Science and Technology
    Volume62
    Issue number6
    DOIs
    Publication statusPublished - 2018 Nov 1

    Fingerprint

    physiological responses
    virtual reality
    translating
    Virtual reality
    visual stimuli
    tremors
    heart rate
    ratings
    Controllers
    controllers
    Sensors
    interactions
    sensors

    ASJC Scopus subject areas

    • Electronic, Optical and Magnetic Materials
    • Chemistry(all)
    • Atomic and Molecular Physics, and Optics
    • Computer Science Applications

    Cite this

    @article{0163033933c5459d848f27e78881d7dc,
    title = "Augmented Cross-modality: Translating the Physiological Responses, Knowledge and Impression to Audio-visual Information in Virtual Reality",
    abstract = "This paper proposes the method of interaction design to present haptic experience as intended in virtual reality (VR). The method that we named “Augmented Cross-Modality” is to translate the physiological responses, knowledge and impression about the experience in real world into audio-visual stimuli and add them to the interaction in VR. In this study, as expressions for presenting a haptic experience of gripping an object strongly and lifting a heavy object, we design hand tremor, strong gripping and increasing heart rate in VR. The objective is, at first, to enhance a sense of strain of a body with these augmented cross-modal expressions and then, change the quality of the total haptic experience and as a result, make it closer to the experience of lifting a heavy object. This method is evaluated by several rating scales, interviews and force sensors attached to a VR controller. The result suggests that the expressions of this method enhancing a haptic experience of strong gripping in almost all participants and the effectiveness were confirmed. c 2018 Society for Imaging Science and Technology.",
    author = "Yutaro Hirao and Takashi Kawai",
    year = "2018",
    month = "11",
    day = "1",
    doi = "10.2352/J.ImagingSci.Technol.2018.62.6.060402",
    language = "English",
    volume = "62",
    journal = "Journal of Imaging Science and Technology",
    issn = "1062-3701",
    publisher = "Society for Imaging Science and Technology",
    number = "6",

    }

    TY - JOUR

    T1 - Augmented Cross-modality

    T2 - Translating the Physiological Responses, Knowledge and Impression to Audio-visual Information in Virtual Reality

    AU - Hirao, Yutaro

    AU - Kawai, Takashi

    PY - 2018/11/1

    Y1 - 2018/11/1

    N2 - This paper proposes the method of interaction design to present haptic experience as intended in virtual reality (VR). The method that we named “Augmented Cross-Modality” is to translate the physiological responses, knowledge and impression about the experience in real world into audio-visual stimuli and add them to the interaction in VR. In this study, as expressions for presenting a haptic experience of gripping an object strongly and lifting a heavy object, we design hand tremor, strong gripping and increasing heart rate in VR. The objective is, at first, to enhance a sense of strain of a body with these augmented cross-modal expressions and then, change the quality of the total haptic experience and as a result, make it closer to the experience of lifting a heavy object. This method is evaluated by several rating scales, interviews and force sensors attached to a VR controller. The result suggests that the expressions of this method enhancing a haptic experience of strong gripping in almost all participants and the effectiveness were confirmed. c 2018 Society for Imaging Science and Technology.

    AB - This paper proposes the method of interaction design to present haptic experience as intended in virtual reality (VR). The method that we named “Augmented Cross-Modality” is to translate the physiological responses, knowledge and impression about the experience in real world into audio-visual stimuli and add them to the interaction in VR. In this study, as expressions for presenting a haptic experience of gripping an object strongly and lifting a heavy object, we design hand tremor, strong gripping and increasing heart rate in VR. The objective is, at first, to enhance a sense of strain of a body with these augmented cross-modal expressions and then, change the quality of the total haptic experience and as a result, make it closer to the experience of lifting a heavy object. This method is evaluated by several rating scales, interviews and force sensors attached to a VR controller. The result suggests that the expressions of this method enhancing a haptic experience of strong gripping in almost all participants and the effectiveness were confirmed. c 2018 Society for Imaging Science and Technology.

    UR - http://www.scopus.com/inward/record.url?scp=85059523955&partnerID=8YFLogxK

    UR - http://www.scopus.com/inward/citedby.url?scp=85059523955&partnerID=8YFLogxK

    U2 - 10.2352/J.ImagingSci.Technol.2018.62.6.060402

    DO - 10.2352/J.ImagingSci.Technol.2018.62.6.060402

    M3 - Article

    AN - SCOPUS:85059523955

    VL - 62

    JO - Journal of Imaging Science and Technology

    JF - Journal of Imaging Science and Technology

    SN - 1062-3701

    IS - 6

    M1 - 060402

    ER -