Virtual space teleconferencing: Real-time detection and reproduction of 3D face and body images

This paper describes a virtual space teleconferencing that proposed as an human-oriented telecommunication system. By using this system, participants at different sites can engage in a conference with the sensation of sharing the same space, and cooperative work can be performed among remotely located participants. More specifically, our system connects three different sites via a 1.5 Mbps ISDN now in commercial use. The system has two large screens that can provide real-time reproduction of 3-D whole body human images. Participants at the three different sites are able to feel as if they are all at the same site. In this experimental system, to achieve real-time reproduction, tape marks, a data glove and magnetic sensors are used. However, these tools are not appropriate for natural human communications. It is a final goal to develop a system that does not require such tools. We also describe a new facial expression detection method that does not need the tape marks, and real-time detection is realized by exploiting frequency domain data from the face image acquired by the CCD camera. Experiments show very promising results for accurate expression detection and for implementation of real-time hardware of the using propose method.