3D human head geometry estimation from a speech

Akinobu Maejima, Shigeo Morishima

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

We can visualize acquaintances' appearance by just hearing their voice if we have met them in past few years. Thus, it would appear that some relationships exist in between voice and appearance. If 3D head geometry could be estimated from a voice, we can realize some applications (e.g, avatar generation, character modeling for video game, etc.). Previously, although many researchers have been reported about a relationship between acoustic features of a voice and its corresponding dynamical visual features including lip, tongue, and jaw movements or vocal articulation during a speech, however, there have been few reports about a relationship between acoustic features and static 3D head geometry. In this paper, we focus on estimating 3D head geometry from a voice. Acoustic features vary depending on a speech context and its intonation. Therefore we restrict a context to Japanese 5 vowels. Under this assumption, to estimate 3D head geometry, we use a Feedforward Neural Network (FNN) trained by using a correspondence between an individual acoustic features extracted from a Japanese vowel and 3D head geometry generated based on a 3D range scan. The performance of our method is shown by both closed and open tests. As a result, we found that 3D head geometry which is acoustically similar to an input voice could be estimated under the limited condition.

Original languageEnglish
Title of host publicationACM SIGGRAPH 2012 Posters, SIGGRAPH'12
DOIs
Publication statusPublished - 2012 Sep 6
EventACM Special Interest Group on Computer Graphics and Interactive Techniques Conference, SIGGRAPH'12 - Los Angeles, CA, United States
Duration: 2012 Aug 52012 Aug 9

Publication series

NameACM SIGGRAPH 2012 Posters, SIGGRAPH'12

Conference

ConferenceACM Special Interest Group on Computer Graphics and Interactive Techniques Conference, SIGGRAPH'12
CountryUnited States
CityLos Angeles, CA
Period12/8/512/8/9

    Fingerprint

ASJC Scopus subject areas

  • Computer Graphics and Computer-Aided Design
  • Computer Vision and Pattern Recognition
  • Software

Cite this

Maejima, A., & Morishima, S. (2012). 3D human head geometry estimation from a speech. In ACM SIGGRAPH 2012 Posters, SIGGRAPH'12 (ACM SIGGRAPH 2012 Posters, SIGGRAPH'12). https://doi.org/10.1145/2342896.2342997