Model-based lip synchronization with automatically translated synthetic voice toward a multi-modal translation system

Shin Ogata, Kazumasa Murai, Satoshi Nakamura, Shigeo Morishima

Research output: Chapter in Book/Report/Conference proceedingConference contribution

7 Citations (Scopus)

Abstract

In this paper, we introduce a multi-modal English-to-Japanese and Japanese-to-English translation system that also translates the speaker's speech motion while synchronizing it to the translated speech. To retain the speaker's facial expression, we substitute only the speech organ's image with the synthesized one, which is made by a three-dimensional wire-frame model that is adaptable to any speaker. Our approach enables image synthesis and translation with an extremely small database.

Original languageEnglish
Title of host publicationProceedings - IEEE International Conference on Multimedia and Expo
PublisherIEEE Computer Society
Pages28-31
Number of pages4
ISBN (Print)0769511988
DOIs
Publication statusPublished - 2001
Externally publishedYes
Event2001 IEEE International Conference on Multimedia and Expo, ICME 2001 - Tokyo
Duration: 2001 Aug 222001 Aug 25

Other

Other2001 IEEE International Conference on Multimedia and Expo, ICME 2001
CityTokyo
Period01/8/2201/8/25

Fingerprint

Synchronization
Wire

ASJC Scopus subject areas

  • Computer Networks and Communications
  • Computer Science Applications

Cite this

Ogata, S., Murai, K., Nakamura, S., & Morishima, S. (2001). Model-based lip synchronization with automatically translated synthetic voice toward a multi-modal translation system. In Proceedings - IEEE International Conference on Multimedia and Expo (pp. 28-31). [1237647] IEEE Computer Society. https://doi.org/10.1109/ICME.2001.1237647

Model-based lip synchronization with automatically translated synthetic voice toward a multi-modal translation system. / Ogata, Shin; Murai, Kazumasa; Nakamura, Satoshi; Morishima, Shigeo.

Proceedings - IEEE International Conference on Multimedia and Expo. IEEE Computer Society, 2001. p. 28-31 1237647.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Ogata, S, Murai, K, Nakamura, S & Morishima, S 2001, Model-based lip synchronization with automatically translated synthetic voice toward a multi-modal translation system. in Proceedings - IEEE International Conference on Multimedia and Expo., 1237647, IEEE Computer Society, pp. 28-31, 2001 IEEE International Conference on Multimedia and Expo, ICME 2001, Tokyo, 01/8/22. https://doi.org/10.1109/ICME.2001.1237647
Ogata S, Murai K, Nakamura S, Morishima S. Model-based lip synchronization with automatically translated synthetic voice toward a multi-modal translation system. In Proceedings - IEEE International Conference on Multimedia and Expo. IEEE Computer Society. 2001. p. 28-31. 1237647 https://doi.org/10.1109/ICME.2001.1237647
Ogata, Shin ; Murai, Kazumasa ; Nakamura, Satoshi ; Morishima, Shigeo. / Model-based lip synchronization with automatically translated synthetic voice toward a multi-modal translation system. Proceedings - IEEE International Conference on Multimedia and Expo. IEEE Computer Society, 2001. pp. 28-31
@inproceedings{e33a68de05714e4dbbdffec9e17db3f2,
title = "Model-based lip synchronization with automatically translated synthetic voice toward a multi-modal translation system",
abstract = "In this paper, we introduce a multi-modal English-to-Japanese and Japanese-to-English translation system that also translates the speaker's speech motion while synchronizing it to the translated speech. To retain the speaker's facial expression, we substitute only the speech organ's image with the synthesized one, which is made by a three-dimensional wire-frame model that is adaptable to any speaker. Our approach enables image synthesis and translation with an extremely small database.",
author = "Shin Ogata and Kazumasa Murai and Satoshi Nakamura and Shigeo Morishima",
year = "2001",
doi = "10.1109/ICME.2001.1237647",
language = "English",
isbn = "0769511988",
pages = "28--31",
booktitle = "Proceedings - IEEE International Conference on Multimedia and Expo",
publisher = "IEEE Computer Society",

}

TY - GEN

T1 - Model-based lip synchronization with automatically translated synthetic voice toward a multi-modal translation system

AU - Ogata, Shin

AU - Murai, Kazumasa

AU - Nakamura, Satoshi

AU - Morishima, Shigeo

PY - 2001

Y1 - 2001

N2 - In this paper, we introduce a multi-modal English-to-Japanese and Japanese-to-English translation system that also translates the speaker's speech motion while synchronizing it to the translated speech. To retain the speaker's facial expression, we substitute only the speech organ's image with the synthesized one, which is made by a three-dimensional wire-frame model that is adaptable to any speaker. Our approach enables image synthesis and translation with an extremely small database.

AB - In this paper, we introduce a multi-modal English-to-Japanese and Japanese-to-English translation system that also translates the speaker's speech motion while synchronizing it to the translated speech. To retain the speaker's facial expression, we substitute only the speech organ's image with the synthesized one, which is made by a three-dimensional wire-frame model that is adaptable to any speaker. Our approach enables image synthesis and translation with an extremely small database.

UR - http://www.scopus.com/inward/record.url?scp=70350254571&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=70350254571&partnerID=8YFLogxK

U2 - 10.1109/ICME.2001.1237647

DO - 10.1109/ICME.2001.1237647

M3 - Conference contribution

AN - SCOPUS:70350254571

SN - 0769511988

SP - 28

EP - 31

BT - Proceedings - IEEE International Conference on Multimedia and Expo

PB - IEEE Computer Society

ER -