Realtime face analysis and synthesis using neural network

Research output: Chapter in Book/Report/Conference proceedingChapter

5 Citations (Scopus)

Abstract

In this paper, we describe a recent research results about how to generate an avatar's face on a real-time process exactly copying a real person's face. It is very important for synthesis of a real avatar to duplicate emotion and impression precisely included in original face image and voice. Face fitting tool from multi-angle camera images is introduced to make a real 3D face model with real texture and geometry very close to the original. When avatar is speaking something, voice signal is very essential to decide a mouth shape feature. So real-time mouth shape control mechanism is proposed by conversion from speech parameters to lip shape parameters using multilayered neural network. For dynamic modeling of facial expression, muscle structure constraint is introduced to generate a facial expression naturally with a few parameters. We also tried to get muscle parameters automatically to decide an expression from local motion vector on face calculated by optical flow in video sequence. Finally an approach that enables the modeling emotions appearing on faces. A system with this approach helps to analyze, synthesize and code face images at the emotional level.

Original languageEnglish
Title of host publicationNeural Networks for Signal Processing - Proceedings of the IEEE Workshop
Place of PublicationPiscataway, NJ, United States
PublisherIEEE
Pages13-22
Number of pages10
Volume1
Publication statusPublished - 2000
Externally publishedYes
Event10th IEEE Workshop on Neural Netwoks for Signal Processing (NNSP2000) - Sydney, Australia
Duration: 2000 Dec 112000 Dec 13

Other

Other10th IEEE Workshop on Neural Netwoks for Signal Processing (NNSP2000)
CitySydney, Australia
Period00/12/1100/12/13

Fingerprint

Neural networks
Muscle
Copying
Optical flows
Textures
Cameras
Geometry

ASJC Scopus subject areas

  • Signal Processing
  • Software
  • Electrical and Electronic Engineering

Cite this

Morishima, S. (2000). Realtime face analysis and synthesis using neural network. In Neural Networks for Signal Processing - Proceedings of the IEEE Workshop (Vol. 1, pp. 13-22). Piscataway, NJ, United States: IEEE.

Realtime face analysis and synthesis using neural network. / Morishima, Shigeo.

Neural Networks for Signal Processing - Proceedings of the IEEE Workshop. Vol. 1 Piscataway, NJ, United States : IEEE, 2000. p. 13-22.

Research output: Chapter in Book/Report/Conference proceedingChapter

Morishima, S 2000, Realtime face analysis and synthesis using neural network. in Neural Networks for Signal Processing - Proceedings of the IEEE Workshop. vol. 1, IEEE, Piscataway, NJ, United States, pp. 13-22, 10th IEEE Workshop on Neural Netwoks for Signal Processing (NNSP2000), Sydney, Australia, 00/12/11.
Morishima S. Realtime face analysis and synthesis using neural network. In Neural Networks for Signal Processing - Proceedings of the IEEE Workshop. Vol. 1. Piscataway, NJ, United States: IEEE. 2000. p. 13-22
Morishima, Shigeo. / Realtime face analysis and synthesis using neural network. Neural Networks for Signal Processing - Proceedings of the IEEE Workshop. Vol. 1 Piscataway, NJ, United States : IEEE, 2000. pp. 13-22
@inbook{bb6bd355364d4b85862de08fd01c8adb,
title = "Realtime face analysis and synthesis using neural network",
abstract = "In this paper, we describe a recent research results about how to generate an avatar's face on a real-time process exactly copying a real person's face. It is very important for synthesis of a real avatar to duplicate emotion and impression precisely included in original face image and voice. Face fitting tool from multi-angle camera images is introduced to make a real 3D face model with real texture and geometry very close to the original. When avatar is speaking something, voice signal is very essential to decide a mouth shape feature. So real-time mouth shape control mechanism is proposed by conversion from speech parameters to lip shape parameters using multilayered neural network. For dynamic modeling of facial expression, muscle structure constraint is introduced to generate a facial expression naturally with a few parameters. We also tried to get muscle parameters automatically to decide an expression from local motion vector on face calculated by optical flow in video sequence. Finally an approach that enables the modeling emotions appearing on faces. A system with this approach helps to analyze, synthesize and code face images at the emotional level.",
author = "Shigeo Morishima",
year = "2000",
language = "English",
volume = "1",
pages = "13--22",
booktitle = "Neural Networks for Signal Processing - Proceedings of the IEEE Workshop",
publisher = "IEEE",

}

TY - CHAP

T1 - Realtime face analysis and synthesis using neural network

AU - Morishima, Shigeo

PY - 2000

Y1 - 2000

N2 - In this paper, we describe a recent research results about how to generate an avatar's face on a real-time process exactly copying a real person's face. It is very important for synthesis of a real avatar to duplicate emotion and impression precisely included in original face image and voice. Face fitting tool from multi-angle camera images is introduced to make a real 3D face model with real texture and geometry very close to the original. When avatar is speaking something, voice signal is very essential to decide a mouth shape feature. So real-time mouth shape control mechanism is proposed by conversion from speech parameters to lip shape parameters using multilayered neural network. For dynamic modeling of facial expression, muscle structure constraint is introduced to generate a facial expression naturally with a few parameters. We also tried to get muscle parameters automatically to decide an expression from local motion vector on face calculated by optical flow in video sequence. Finally an approach that enables the modeling emotions appearing on faces. A system with this approach helps to analyze, synthesize and code face images at the emotional level.

AB - In this paper, we describe a recent research results about how to generate an avatar's face on a real-time process exactly copying a real person's face. It is very important for synthesis of a real avatar to duplicate emotion and impression precisely included in original face image and voice. Face fitting tool from multi-angle camera images is introduced to make a real 3D face model with real texture and geometry very close to the original. When avatar is speaking something, voice signal is very essential to decide a mouth shape feature. So real-time mouth shape control mechanism is proposed by conversion from speech parameters to lip shape parameters using multilayered neural network. For dynamic modeling of facial expression, muscle structure constraint is introduced to generate a facial expression naturally with a few parameters. We also tried to get muscle parameters automatically to decide an expression from local motion vector on face calculated by optical flow in video sequence. Finally an approach that enables the modeling emotions appearing on faces. A system with this approach helps to analyze, synthesize and code face images at the emotional level.

UR - http://www.scopus.com/inward/record.url?scp=0034497415&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=0034497415&partnerID=8YFLogxK

M3 - Chapter

AN - SCOPUS:0034497415

VL - 1

SP - 13

EP - 22

BT - Neural Networks for Signal Processing - Proceedings of the IEEE Workshop

PB - IEEE

CY - Piscataway, NJ, United States

ER -