Voice animator: Automatic lip-synching in limited animation by audio

Shoichi Furukawa, Tsukasa Fukusato, Shugo Yamaguchi, Shigeo Morishima

    Research output: Chapter in Book/Report/Conference proceedingConference contribution

    Abstract

    Limited animation is one of the traditional techniques for producing cartoon animations. Owing to its expressive style, it has been enjoyed around the world. However, producing high quality animations using this limited style is time-consuming and costly for animators. Furthermore, proper synchronization between the voice-actor’s voice and the character’s mouth and lip motion requires well-experienced animators. This is essential because viewers are very sensitive to audio-lip discrepancies. In this paper, we propose a method that automatically creates high-quality limited-style lip-synched animations using audio tracks. Our system can be applied for creating not only the original animations but also dubbed ones independently of languages. Because our approach follows the standard workflow employed in cartoon animation production, our system can successfully assist animators. In addition, users can implement our system as a plug-in of a standard tool for creating animations (Adobe After Effects) and can easily arrange character lip motion to suit their own style. We visually evaluate our results both absolutely and relatively by comparing them with those of previous works. From the user evaluations, we confirm that our algorithms is able to successfully generate more natural audio-mouth synchronizations in limited-style lip-synched animations than previous algorithms.

    Original languageEnglish
    Title of host publicationAdvances in Computer Entertainment Technology - 14th International Conference, ACE 2017, Proceedings
    PublisherSpringer-Verlag
    Pages153-171
    Number of pages19
    ISBN (Print)9783319762692
    DOIs
    Publication statusPublished - 2018 Jan 1
    Event14th International Conference on Advances in Computer Entertainment Technology, ACE 2017 - London, United Kingdom
    Duration: 2017 Dec 142017 Dec 16

    Publication series

    NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
    Volume10714 LNCS
    ISSN (Print)0302-9743
    ISSN (Electronic)1611-3349

    Other

    Other14th International Conference on Advances in Computer Entertainment Technology, ACE 2017
    CountryUnited Kingdom
    CityLondon
    Period17/12/1417/12/16

    Fingerprint

    Animation
    Synchronization
    User Evaluation
    Motion
    Voice
    Plug-in
    Production Systems
    Work Flow
    Discrepancy
    Style
    Evaluate

    Keywords

    • Animation filtering
    • Limited animations
    • Lip-synching

    ASJC Scopus subject areas

    • Theoretical Computer Science
    • Computer Science(all)

    Cite this

    Furukawa, S., Fukusato, T., Yamaguchi, S., & Morishima, S. (2018). Voice animator: Automatic lip-synching in limited animation by audio. In Advances in Computer Entertainment Technology - 14th International Conference, ACE 2017, Proceedings (pp. 153-171). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 10714 LNCS). Springer-Verlag. https://doi.org/10.1007/978-3-319-76270-8_12

    Voice animator : Automatic lip-synching in limited animation by audio. / Furukawa, Shoichi; Fukusato, Tsukasa; Yamaguchi, Shugo; Morishima, Shigeo.

    Advances in Computer Entertainment Technology - 14th International Conference, ACE 2017, Proceedings. Springer-Verlag, 2018. p. 153-171 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 10714 LNCS).

    Research output: Chapter in Book/Report/Conference proceedingConference contribution

    Furukawa, S, Fukusato, T, Yamaguchi, S & Morishima, S 2018, Voice animator: Automatic lip-synching in limited animation by audio. in Advances in Computer Entertainment Technology - 14th International Conference, ACE 2017, Proceedings. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 10714 LNCS, Springer-Verlag, pp. 153-171, 14th International Conference on Advances in Computer Entertainment Technology, ACE 2017, London, United Kingdom, 17/12/14. https://doi.org/10.1007/978-3-319-76270-8_12
    Furukawa S, Fukusato T, Yamaguchi S, Morishima S. Voice animator: Automatic lip-synching in limited animation by audio. In Advances in Computer Entertainment Technology - 14th International Conference, ACE 2017, Proceedings. Springer-Verlag. 2018. p. 153-171. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). https://doi.org/10.1007/978-3-319-76270-8_12
    Furukawa, Shoichi ; Fukusato, Tsukasa ; Yamaguchi, Shugo ; Morishima, Shigeo. / Voice animator : Automatic lip-synching in limited animation by audio. Advances in Computer Entertainment Technology - 14th International Conference, ACE 2017, Proceedings. Springer-Verlag, 2018. pp. 153-171 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).
    @inproceedings{a3e2b502957d4b6889cef31293339ace,
    title = "Voice animator: Automatic lip-synching in limited animation by audio",
    abstract = "Limited animation is one of the traditional techniques for producing cartoon animations. Owing to its expressive style, it has been enjoyed around the world. However, producing high quality animations using this limited style is time-consuming and costly for animators. Furthermore, proper synchronization between the voice-actor’s voice and the character’s mouth and lip motion requires well-experienced animators. This is essential because viewers are very sensitive to audio-lip discrepancies. In this paper, we propose a method that automatically creates high-quality limited-style lip-synched animations using audio tracks. Our system can be applied for creating not only the original animations but also dubbed ones independently of languages. Because our approach follows the standard workflow employed in cartoon animation production, our system can successfully assist animators. In addition, users can implement our system as a plug-in of a standard tool for creating animations (Adobe After Effects) and can easily arrange character lip motion to suit their own style. We visually evaluate our results both absolutely and relatively by comparing them with those of previous works. From the user evaluations, we confirm that our algorithms is able to successfully generate more natural audio-mouth synchronizations in limited-style lip-synched animations than previous algorithms.",
    keywords = "Animation filtering, Limited animations, Lip-synching",
    author = "Shoichi Furukawa and Tsukasa Fukusato and Shugo Yamaguchi and Shigeo Morishima",
    year = "2018",
    month = "1",
    day = "1",
    doi = "10.1007/978-3-319-76270-8_12",
    language = "English",
    isbn = "9783319762692",
    series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",
    publisher = "Springer-Verlag",
    pages = "153--171",
    booktitle = "Advances in Computer Entertainment Technology - 14th International Conference, ACE 2017, Proceedings",

    }

    TY - GEN

    T1 - Voice animator

    T2 - Automatic lip-synching in limited animation by audio

    AU - Furukawa, Shoichi

    AU - Fukusato, Tsukasa

    AU - Yamaguchi, Shugo

    AU - Morishima, Shigeo

    PY - 2018/1/1

    Y1 - 2018/1/1

    N2 - Limited animation is one of the traditional techniques for producing cartoon animations. Owing to its expressive style, it has been enjoyed around the world. However, producing high quality animations using this limited style is time-consuming and costly for animators. Furthermore, proper synchronization between the voice-actor’s voice and the character’s mouth and lip motion requires well-experienced animators. This is essential because viewers are very sensitive to audio-lip discrepancies. In this paper, we propose a method that automatically creates high-quality limited-style lip-synched animations using audio tracks. Our system can be applied for creating not only the original animations but also dubbed ones independently of languages. Because our approach follows the standard workflow employed in cartoon animation production, our system can successfully assist animators. In addition, users can implement our system as a plug-in of a standard tool for creating animations (Adobe After Effects) and can easily arrange character lip motion to suit their own style. We visually evaluate our results both absolutely and relatively by comparing them with those of previous works. From the user evaluations, we confirm that our algorithms is able to successfully generate more natural audio-mouth synchronizations in limited-style lip-synched animations than previous algorithms.

    AB - Limited animation is one of the traditional techniques for producing cartoon animations. Owing to its expressive style, it has been enjoyed around the world. However, producing high quality animations using this limited style is time-consuming and costly for animators. Furthermore, proper synchronization between the voice-actor’s voice and the character’s mouth and lip motion requires well-experienced animators. This is essential because viewers are very sensitive to audio-lip discrepancies. In this paper, we propose a method that automatically creates high-quality limited-style lip-synched animations using audio tracks. Our system can be applied for creating not only the original animations but also dubbed ones independently of languages. Because our approach follows the standard workflow employed in cartoon animation production, our system can successfully assist animators. In addition, users can implement our system as a plug-in of a standard tool for creating animations (Adobe After Effects) and can easily arrange character lip motion to suit their own style. We visually evaluate our results both absolutely and relatively by comparing them with those of previous works. From the user evaluations, we confirm that our algorithms is able to successfully generate more natural audio-mouth synchronizations in limited-style lip-synched animations than previous algorithms.

    KW - Animation filtering

    KW - Limited animations

    KW - Lip-synching

    UR - http://www.scopus.com/inward/record.url?scp=85043526440&partnerID=8YFLogxK

    UR - http://www.scopus.com/inward/citedby.url?scp=85043526440&partnerID=8YFLogxK

    U2 - 10.1007/978-3-319-76270-8_12

    DO - 10.1007/978-3-319-76270-8_12

    M3 - Conference contribution

    AN - SCOPUS:85043526440

    SN - 9783319762692

    T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

    SP - 153

    EP - 171

    BT - Advances in Computer Entertainment Technology - 14th International Conference, ACE 2017, Proceedings

    PB - Springer-Verlag

    ER -