Automatic face tracking and model match-move automatic face tracking and model match-move in video sequence using 3D face model in video sequence using 3D face model

Takafumi Misawa, Kazumasa Murai, Satoshi Nakamura, Shigeo Morishima

Research output: Chapter in Book/Report/Conference proceedingConference contribution

1 Citation (Scopus)

Abstract

A stand-in is a common technique for movies and TV programs in foreign languages. The current stand-in that only substitutes the voice channel results awkward matching to the mouth motion. Videophone with automatic voice translation are expected to be widely used in the near future, which may face the same problem without lip-synchronized speaking face image translation. In this paper, we propose a method to track motion of the face from the video image, that is one of the key technologies for speaking image translation. Almost all the old tracking algorithms aim to detect feature points of the face. However, these algorithms had problems, such as blurring of a feature point between frames and occlusion of the hidden feature point by rotation of the head, and so on. In this paper, we propose a method which detects movement and rotation of the head given the three dimensional shape of the face, by template matching using a 3D personal face wireframe model. The evaluation experiments are carried out with the measured reference data of the head. The proposed method achieves 0.48 angle error in average. This result confirms effectiveness of the proposed method.

Original languageEnglish
Title of host publicationProceedings - IEEE International Conference on Multimedia and Expo
PublisherIEEE Computer Society
Pages234-236
Number of pages3
ISBN (Print)0769511988
DOIs
Publication statusPublished - 2001
Externally publishedYes
Event2001 IEEE International Conference on Multimedia and Expo, ICME 2001 - Tokyo
Duration: 2001 Aug 222001 Aug 25

Other

Other2001 IEEE International Conference on Multimedia and Expo, ICME 2001
CityTokyo
Period01/8/2201/8/25

Fingerprint

Template matching
Experiments

ASJC Scopus subject areas

  • Computer Networks and Communications
  • Computer Science Applications

Cite this

Misawa, T., Murai, K., Nakamura, S., & Morishima, S. (2001). Automatic face tracking and model match-move automatic face tracking and model match-move in video sequence using 3D face model in video sequence using 3D face model. In Proceedings - IEEE International Conference on Multimedia and Expo (pp. 234-236). [1237699] IEEE Computer Society. https://doi.org/10.1109/ICME.2001.1237699

Automatic face tracking and model match-move automatic face tracking and model match-move in video sequence using 3D face model in video sequence using 3D face model. / Misawa, Takafumi; Murai, Kazumasa; Nakamura, Satoshi; Morishima, Shigeo.

Proceedings - IEEE International Conference on Multimedia and Expo. IEEE Computer Society, 2001. p. 234-236 1237699.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Misawa, T, Murai, K, Nakamura, S & Morishima, S 2001, Automatic face tracking and model match-move automatic face tracking and model match-move in video sequence using 3D face model in video sequence using 3D face model. in Proceedings - IEEE International Conference on Multimedia and Expo., 1237699, IEEE Computer Society, pp. 234-236, 2001 IEEE International Conference on Multimedia and Expo, ICME 2001, Tokyo, 01/8/22. https://doi.org/10.1109/ICME.2001.1237699
Misawa T, Murai K, Nakamura S, Morishima S. Automatic face tracking and model match-move automatic face tracking and model match-move in video sequence using 3D face model in video sequence using 3D face model. In Proceedings - IEEE International Conference on Multimedia and Expo. IEEE Computer Society. 2001. p. 234-236. 1237699 https://doi.org/10.1109/ICME.2001.1237699
Misawa, Takafumi ; Murai, Kazumasa ; Nakamura, Satoshi ; Morishima, Shigeo. / Automatic face tracking and model match-move automatic face tracking and model match-move in video sequence using 3D face model in video sequence using 3D face model. Proceedings - IEEE International Conference on Multimedia and Expo. IEEE Computer Society, 2001. pp. 234-236
@inproceedings{be7f38eb2d654994b8c934a17eff6cab,
title = "Automatic face tracking and model match-move automatic face tracking and model match-move in video sequence using 3D face model in video sequence using 3D face model",
abstract = "A stand-in is a common technique for movies and TV programs in foreign languages. The current stand-in that only substitutes the voice channel results awkward matching to the mouth motion. Videophone with automatic voice translation are expected to be widely used in the near future, which may face the same problem without lip-synchronized speaking face image translation. In this paper, we propose a method to track motion of the face from the video image, that is one of the key technologies for speaking image translation. Almost all the old tracking algorithms aim to detect feature points of the face. However, these algorithms had problems, such as blurring of a feature point between frames and occlusion of the hidden feature point by rotation of the head, and so on. In this paper, we propose a method which detects movement and rotation of the head given the three dimensional shape of the face, by template matching using a 3D personal face wireframe model. The evaluation experiments are carried out with the measured reference data of the head. The proposed method achieves 0.48 angle error in average. This result confirms effectiveness of the proposed method.",
author = "Takafumi Misawa and Kazumasa Murai and Satoshi Nakamura and Shigeo Morishima",
year = "2001",
doi = "10.1109/ICME.2001.1237699",
language = "English",
isbn = "0769511988",
pages = "234--236",
booktitle = "Proceedings - IEEE International Conference on Multimedia and Expo",
publisher = "IEEE Computer Society",

}

TY - GEN

T1 - Automatic face tracking and model match-move automatic face tracking and model match-move in video sequence using 3D face model in video sequence using 3D face model

AU - Misawa, Takafumi

AU - Murai, Kazumasa

AU - Nakamura, Satoshi

AU - Morishima, Shigeo

PY - 2001

Y1 - 2001

N2 - A stand-in is a common technique for movies and TV programs in foreign languages. The current stand-in that only substitutes the voice channel results awkward matching to the mouth motion. Videophone with automatic voice translation are expected to be widely used in the near future, which may face the same problem without lip-synchronized speaking face image translation. In this paper, we propose a method to track motion of the face from the video image, that is one of the key technologies for speaking image translation. Almost all the old tracking algorithms aim to detect feature points of the face. However, these algorithms had problems, such as blurring of a feature point between frames and occlusion of the hidden feature point by rotation of the head, and so on. In this paper, we propose a method which detects movement and rotation of the head given the three dimensional shape of the face, by template matching using a 3D personal face wireframe model. The evaluation experiments are carried out with the measured reference data of the head. The proposed method achieves 0.48 angle error in average. This result confirms effectiveness of the proposed method.

AB - A stand-in is a common technique for movies and TV programs in foreign languages. The current stand-in that only substitutes the voice channel results awkward matching to the mouth motion. Videophone with automatic voice translation are expected to be widely used in the near future, which may face the same problem without lip-synchronized speaking face image translation. In this paper, we propose a method to track motion of the face from the video image, that is one of the key technologies for speaking image translation. Almost all the old tracking algorithms aim to detect feature points of the face. However, these algorithms had problems, such as blurring of a feature point between frames and occlusion of the hidden feature point by rotation of the head, and so on. In this paper, we propose a method which detects movement and rotation of the head given the three dimensional shape of the face, by template matching using a 3D personal face wireframe model. The evaluation experiments are carried out with the measured reference data of the head. The proposed method achieves 0.48 angle error in average. This result confirms effectiveness of the proposed method.

UR - http://www.scopus.com/inward/record.url?scp=10244262303&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=10244262303&partnerID=8YFLogxK

U2 - 10.1109/ICME.2001.1237699

DO - 10.1109/ICME.2001.1237699

M3 - Conference contribution

SN - 0769511988

SP - 234

EP - 236

BT - Proceedings - IEEE International Conference on Multimedia and Expo

PB - IEEE Computer Society

ER -