Interactive virtual reality speech simulation system using autonomous audience with natural non-verbal behavior

Public speaking anxiety (PSA) is a fear of speaking in front of others. Most people experience a certain amount of anxiety in public speaking situation. This study aims to help people overcome PSA using an interactive VR simulation system with real-life scenarios. We present a multimodal VR speech simulation system using autonomous audience with natural non-verbal behavior to enhance users' sense of presence. Additionally, real-time multimodal feedback is produced by virtual audience based on users' public speaking behavior which automatically analyzed by multimodal sensors (e.g. microphone, motion capture, heart rate monitor). We perform an evaluation based on self-assessment questionnaires and biometry to investigate three study conditions: (I) control condition (baseline), (II) interactive virtual audience, and (III) virtual audience with natural non-verbal behavior. We divided participants into two groups with different conditions: interactive virtual audience condition (n = 7) and virtual audience with nature non-verbal behavior condition (n = 9). The results indicate that the usage of a virtual audience with natural non-verbal behavior increased a higher sense of presence and more anxiety-provoking.