Video translation system using face tracking and lip synchronization

S. Morishima*, Shin Ogata, S. Nakamura

*この研究の対応する著者

研究成果

1 被引用数 (Scopus)

抄録

We introduce a multi-modal English-to-Japanese and Japanese-to-English translation system that also translates the speaker's speech motion while synchronizing it to the translated speech. To retain the speaker's facial expression, we substitute only the speech organ's image with the synthesized one, which is made by a three-dimensional wire-frame model that is adaptable to any speaker. Our approach enables image synthesis and translation with an extremely small database. Also, we propose a method to track motion of the face from the video image. In this system, movement and rotation of the head is detected by template matching using a 3D personal face wire-frame model. By this technique, an automatic video translation can be achieved.

本文言語English
ホスト出版物のタイトルProceedings - IEEE International Conference on Multimedia and Expo
出版社IEEE Computer Society
ページ649-652
ページ数4
ISBN(電子版)0769511988
DOI
出版ステータスPublished - 2001 1 1
外部発表はい
イベント2001 IEEE International Conference on Multimedia and Expo, ICME 2001 - Tokyo, Japan
継続期間: 2001 8 222001 8 25

出版物シリーズ

名前Proceedings - IEEE International Conference on Multimedia and Expo
ISSN(印刷版)1945-7871
ISSN(電子版)1945-788X

Other

Other2001 IEEE International Conference on Multimedia and Expo, ICME 2001
国/地域Japan
CityTokyo
Period01/8/2201/8/25

ASJC Scopus subject areas

  • コンピュータ ネットワークおよび通信
  • コンピュータ サイエンスの応用

フィンガープリント

「Video translation system using face tracking and lip synchronization」の研究トピックを掘り下げます。これらがまとまってユニークなフィンガープリントを構成します。

引用スタイル