Video translation system using face tracking and lip synchronization

S. Morishima, Shin Ogata, S. Nakamura

研究成果: Conference contribution

抄録

We introduce a multi-modal English-to-Japanese and Japanese-to-English translation system that also translates the speaker's speech motion while synchronizing it to the translated speech. To retain the speaker's facial expression, we substitute only the speech organ's image with the synthesized one, which is made by a three-dimensional wire-frame model that is adaptable to any speaker. Our approach enables image synthesis and translation with an extremely small database. Also, we propose a method to track motion of the face from the video image. In this system, movement and rotation of the head is detected by template matching using a 3D personal face wire-frame model. By this technique, an automatic video translation can be achieved.

本文言語English
ホスト出版物のタイトルProceedings - IEEE International Conference on Multimedia and Expo
出版社IEEE Computer Society
ページ649-652
ページ数4
ISBN(電子版)0769511988
DOI
出版ステータスPublished - 2001 1 1
外部発表はい
イベント2001 IEEE International Conference on Multimedia and Expo, ICME 2001 - Tokyo, Japan
継続期間: 2001 8 222001 8 25

出版物シリーズ

名前Proceedings - IEEE International Conference on Multimedia and Expo
ISSN(印刷版)1945-7871
ISSN(電子版)1945-788X

Other

Other2001 IEEE International Conference on Multimedia and Expo, ICME 2001
CountryJapan
CityTokyo
Period01/8/2201/8/25

ASJC Scopus subject areas

  • Computer Networks and Communications
  • Computer Science Applications

フィンガープリント 「Video translation system using face tracking and lip synchronization」の研究トピックを掘り下げます。これらがまとまってユニークなフィンガープリントを構成します。

引用スタイル