Real-time facial action image synthesis system driven by speech and text

Shigeo Morishima, Kiyoharu Aizawa, Hiroshi Harashima

Research output: Chapter in Book/Report/Conference proceedingConference contribution

12 Citations (Scopus)

Abstract

Automatic facial motion image synthesis schemes and a real-time system design are presented. The purpose of this scheme is to realize an intelligent human-machine interface or intelligent communication system with talking head images. A human face is reconstructed with 3D surface model and texture mapping technique on the display of terminal. Facial motion images are synthesized naturally by transformation of the lattice points on wire frames. Two types of motion drive methods, text to image conversion and speech to image conversion are proposed in this paper. In the former manner, the synthesized head can speak some given texts naturally and in the latter case, some mouth and jaw motions can be synthesized in time to speech signal of behind speaker. These schemes were implemented to a parallel image computer and a real-time image synthesizer could output facial motion images to the display as fast as video rate.

Original languageEnglish
Title of host publicationProceedings of SPIE - The International Society for Optical Engineering
EditorsMurat Kunt
Place of PublicationBellingham, WA, United States
PublisherPubl by Int Soc for Optical Engineering
Pages1151-1158
Number of pages8
Volume1360 pt 2
ISBN (Print)0819404217
Publication statusPublished - 1990
Externally publishedYes
EventVisual Communications and Image Processing '90 - Lausanne, Switz
Duration: 1990 Oct 11990 Oct 4

Other

OtherVisual Communications and Image Processing '90
CityLausanne, Switz
Period90/10/190/10/4

Fingerprint

Display devices
synthesis
Real time systems
Communication systems
Textures
Systems analysis
Wire
talking
synthesizers
mouth
systems engineering
telecommunication
textures
wire
output

ASJC Scopus subject areas

  • Electrical and Electronic Engineering
  • Condensed Matter Physics

Cite this

Morishima, S., Aizawa, K., & Harashima, H. (1990). Real-time facial action image synthesis system driven by speech and text. In M. Kunt (Ed.), Proceedings of SPIE - The International Society for Optical Engineering (Vol. 1360 pt 2, pp. 1151-1158). Bellingham, WA, United States: Publ by Int Soc for Optical Engineering.

Real-time facial action image synthesis system driven by speech and text. / Morishima, Shigeo; Aizawa, Kiyoharu; Harashima, Hiroshi.

Proceedings of SPIE - The International Society for Optical Engineering. ed. / Murat Kunt. Vol. 1360 pt 2 Bellingham, WA, United States : Publ by Int Soc for Optical Engineering, 1990. p. 1151-1158.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Morishima, S, Aizawa, K & Harashima, H 1990, Real-time facial action image synthesis system driven by speech and text. in M Kunt (ed.), Proceedings of SPIE - The International Society for Optical Engineering. vol. 1360 pt 2, Publ by Int Soc for Optical Engineering, Bellingham, WA, United States, pp. 1151-1158, Visual Communications and Image Processing '90, Lausanne, Switz, 90/10/1.
Morishima S, Aizawa K, Harashima H. Real-time facial action image synthesis system driven by speech and text. In Kunt M, editor, Proceedings of SPIE - The International Society for Optical Engineering. Vol. 1360 pt 2. Bellingham, WA, United States: Publ by Int Soc for Optical Engineering. 1990. p. 1151-1158
Morishima, Shigeo ; Aizawa, Kiyoharu ; Harashima, Hiroshi. / Real-time facial action image synthesis system driven by speech and text. Proceedings of SPIE - The International Society for Optical Engineering. editor / Murat Kunt. Vol. 1360 pt 2 Bellingham, WA, United States : Publ by Int Soc for Optical Engineering, 1990. pp. 1151-1158
@inproceedings{f72a8a8a136d4b658417eeff0b70129e,
title = "Real-time facial action image synthesis system driven by speech and text",
abstract = "Automatic facial motion image synthesis schemes and a real-time system design are presented. The purpose of this scheme is to realize an intelligent human-machine interface or intelligent communication system with talking head images. A human face is reconstructed with 3D surface model and texture mapping technique on the display of terminal. Facial motion images are synthesized naturally by transformation of the lattice points on wire frames. Two types of motion drive methods, text to image conversion and speech to image conversion are proposed in this paper. In the former manner, the synthesized head can speak some given texts naturally and in the latter case, some mouth and jaw motions can be synthesized in time to speech signal of behind speaker. These schemes were implemented to a parallel image computer and a real-time image synthesizer could output facial motion images to the display as fast as video rate.",
author = "Shigeo Morishima and Kiyoharu Aizawa and Hiroshi Harashima",
year = "1990",
language = "English",
isbn = "0819404217",
volume = "1360 pt 2",
pages = "1151--1158",
editor = "Murat Kunt",
booktitle = "Proceedings of SPIE - The International Society for Optical Engineering",
publisher = "Publ by Int Soc for Optical Engineering",

}

TY - GEN

T1 - Real-time facial action image synthesis system driven by speech and text

AU - Morishima, Shigeo

AU - Aizawa, Kiyoharu

AU - Harashima, Hiroshi

PY - 1990

Y1 - 1990

N2 - Automatic facial motion image synthesis schemes and a real-time system design are presented. The purpose of this scheme is to realize an intelligent human-machine interface or intelligent communication system with talking head images. A human face is reconstructed with 3D surface model and texture mapping technique on the display of terminal. Facial motion images are synthesized naturally by transformation of the lattice points on wire frames. Two types of motion drive methods, text to image conversion and speech to image conversion are proposed in this paper. In the former manner, the synthesized head can speak some given texts naturally and in the latter case, some mouth and jaw motions can be synthesized in time to speech signal of behind speaker. These schemes were implemented to a parallel image computer and a real-time image synthesizer could output facial motion images to the display as fast as video rate.

AB - Automatic facial motion image synthesis schemes and a real-time system design are presented. The purpose of this scheme is to realize an intelligent human-machine interface or intelligent communication system with talking head images. A human face is reconstructed with 3D surface model and texture mapping technique on the display of terminal. Facial motion images are synthesized naturally by transformation of the lattice points on wire frames. Two types of motion drive methods, text to image conversion and speech to image conversion are proposed in this paper. In the former manner, the synthesized head can speak some given texts naturally and in the latter case, some mouth and jaw motions can be synthesized in time to speech signal of behind speaker. These schemes were implemented to a parallel image computer and a real-time image synthesizer could output facial motion images to the display as fast as video rate.

UR - http://www.scopus.com/inward/record.url?scp=0025541390&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=0025541390&partnerID=8YFLogxK

M3 - Conference contribution

AN - SCOPUS:0025541390

SN - 0819404217

VL - 1360 pt 2

SP - 1151

EP - 1158

BT - Proceedings of SPIE - The International Society for Optical Engineering

A2 - Kunt, Murat

PB - Publ by Int Soc for Optical Engineering

CY - Bellingham, WA, United States

ER -