Face analysis and synthesis for interactive entertainment

Shoichiro Iwasawa, Tatsuo Yotsukura, Shigeo Morishima

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

A stand-in is a common technique for movies and TV programs in foreign languages. The current stand-in that only substitutes the voice channel results awkward matching to the mouth motion. Videophone with automatic voice translation are expected to be widely used in the near future, which may face the same problem without lip-synchronized speaking face image translation. In this paper, we propose a method to track motion of the face from the video image and then replace the face part or only mouth part with synthesized one which is synchronized with synthetic voice or spoken voice. This is one of the key technologies not only for speaking image translation and communication system, but also for an interactive entertainment system. Finally, an interactive movie system is introduced as an application of entertainment system.

Original languageEnglish
Title of host publicationIFIP Advances in Information and Communication Technology
PublisherSpringer New York LLC
Pages157-164
Number of pages8
Volume112
ISBN (Print)9781475751536
DOIs
Publication statusPublished - 2003
Externally publishedYes
EventIFIP 1st International Workshop on Entertainment Computing, IWEC 2002 - Makuhari
Duration: 2002 May 142002 May 17

Publication series

NameIFIP Advances in Information and Communication Technology
Volume112
ISSN (Print)18684238

Other

OtherIFIP 1st International Workshop on Entertainment Computing, IWEC 2002
CityMakuhari
Period02/5/1402/5/17

Fingerprint

Entertainment
Movies
Substitute
Communication systems
Language

Keywords

  • Face image synthesis
  • Face tracking
  • Interactive movie
  • Lip synchronization

ASJC Scopus subject areas

  • Information Systems and Management

Cite this

Iwasawa, S., Yotsukura, T., & Morishima, S. (2003). Face analysis and synthesis for interactive entertainment. In IFIP Advances in Information and Communication Technology (Vol. 112, pp. 157-164). (IFIP Advances in Information and Communication Technology; Vol. 112). Springer New York LLC. https://doi.org/10.1007/978-0-387-35660-0

Face analysis and synthesis for interactive entertainment. / Iwasawa, Shoichiro; Yotsukura, Tatsuo; Morishima, Shigeo.

IFIP Advances in Information and Communication Technology. Vol. 112 Springer New York LLC, 2003. p. 157-164 (IFIP Advances in Information and Communication Technology; Vol. 112).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Iwasawa, S, Yotsukura, T & Morishima, S 2003, Face analysis and synthesis for interactive entertainment. in IFIP Advances in Information and Communication Technology. vol. 112, IFIP Advances in Information and Communication Technology, vol. 112, Springer New York LLC, pp. 157-164, IFIP 1st International Workshop on Entertainment Computing, IWEC 2002, Makuhari, 02/5/14. https://doi.org/10.1007/978-0-387-35660-0
Iwasawa S, Yotsukura T, Morishima S. Face analysis and synthesis for interactive entertainment. In IFIP Advances in Information and Communication Technology. Vol. 112. Springer New York LLC. 2003. p. 157-164. (IFIP Advances in Information and Communication Technology). https://doi.org/10.1007/978-0-387-35660-0
Iwasawa, Shoichiro ; Yotsukura, Tatsuo ; Morishima, Shigeo. / Face analysis and synthesis for interactive entertainment. IFIP Advances in Information and Communication Technology. Vol. 112 Springer New York LLC, 2003. pp. 157-164 (IFIP Advances in Information and Communication Technology).
@inproceedings{11fc4b1d58f34616b921ed40bd2a6c77,
title = "Face analysis and synthesis for interactive entertainment",
abstract = "A stand-in is a common technique for movies and TV programs in foreign languages. The current stand-in that only substitutes the voice channel results awkward matching to the mouth motion. Videophone with automatic voice translation are expected to be widely used in the near future, which may face the same problem without lip-synchronized speaking face image translation. In this paper, we propose a method to track motion of the face from the video image and then replace the face part or only mouth part with synthesized one which is synchronized with synthetic voice or spoken voice. This is one of the key technologies not only for speaking image translation and communication system, but also for an interactive entertainment system. Finally, an interactive movie system is introduced as an application of entertainment system.",
keywords = "Face image synthesis, Face tracking, Interactive movie, Lip synchronization",
author = "Shoichiro Iwasawa and Tatsuo Yotsukura and Shigeo Morishima",
year = "2003",
doi = "10.1007/978-0-387-35660-0",
language = "English",
isbn = "9781475751536",
volume = "112",
series = "IFIP Advances in Information and Communication Technology",
publisher = "Springer New York LLC",
pages = "157--164",
booktitle = "IFIP Advances in Information and Communication Technology",

}

TY - GEN

T1 - Face analysis and synthesis for interactive entertainment

AU - Iwasawa, Shoichiro

AU - Yotsukura, Tatsuo

AU - Morishima, Shigeo

PY - 2003

Y1 - 2003

N2 - A stand-in is a common technique for movies and TV programs in foreign languages. The current stand-in that only substitutes the voice channel results awkward matching to the mouth motion. Videophone with automatic voice translation are expected to be widely used in the near future, which may face the same problem without lip-synchronized speaking face image translation. In this paper, we propose a method to track motion of the face from the video image and then replace the face part or only mouth part with synthesized one which is synchronized with synthetic voice or spoken voice. This is one of the key technologies not only for speaking image translation and communication system, but also for an interactive entertainment system. Finally, an interactive movie system is introduced as an application of entertainment system.

AB - A stand-in is a common technique for movies and TV programs in foreign languages. The current stand-in that only substitutes the voice channel results awkward matching to the mouth motion. Videophone with automatic voice translation are expected to be widely used in the near future, which may face the same problem without lip-synchronized speaking face image translation. In this paper, we propose a method to track motion of the face from the video image and then replace the face part or only mouth part with synthesized one which is synchronized with synthetic voice or spoken voice. This is one of the key technologies not only for speaking image translation and communication system, but also for an interactive entertainment system. Finally, an interactive movie system is introduced as an application of entertainment system.

KW - Face image synthesis

KW - Face tracking

KW - Interactive movie

KW - Lip synchronization

UR - http://www.scopus.com/inward/record.url?scp=84904339272&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84904339272&partnerID=8YFLogxK

U2 - 10.1007/978-0-387-35660-0

DO - 10.1007/978-0-387-35660-0

M3 - Conference contribution

SN - 9781475751536

VL - 112

T3 - IFIP Advances in Information and Communication Technology

SP - 157

EP - 164

BT - IFIP Advances in Information and Communication Technology

PB - Springer New York LLC

ER -