Face/gesture analysis/synthesis telchnologies for humanto-human communications through virtual environments

This paper describes the vutual metamorphosis system, whtch was originally proposed by ATR Media Integratton & Communicattons Research Laboratories, Kyoto, Japan. In the vntual metamorphosis system, people can change their forms mto any other form in a vutual scene. To realize. this changes, facial expressions and body postures are estimated by computer vision and reproduced m a computer graphics avatar m real-ume. We developed computer vision based technologies for automatic face trackmg, estimatmg deformations of factal components,and estimatmg body postures in real-time. We also developed real-time facial expresston reproduction method based on "Anatomy for artists". Usmg these technologies, prototyping systems for virtual metamorpho-SIS were constructed. This paper introduces two systems in order of their development: the Virtual Kabuki system and the "Shall We Dance" system.