Choreographing amateur performers based on motion transfer between videos

Kenta Mizui, Makoto Okabe, Rikio Onai

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

We propose a technique for quickly and easily choreographing a video of an amateur performer by comparing it with a video of a corresponding professional performance. Our method allows the user to interactively edit the amateur performance in order to synchronize it with the professional performance in terms of timings and poses. In our system, the user first extracts the amateur and professional poses from every frame via semi-automatic video tracking. The system synchronizes the timings by computing dynamic time warping (DTW) between the two sets of video-tracking data, and then synchronizes the poses by applying image deformation to every frame. To eliminate unnatural vibrations, which often result from inaccurate video tracking, we apply an automatic motion-smoothing algorithm to the synthesized animation. We demonstrate that our method allows the user to successfully edit an amateur's performance into a more polished one, utilizing the Japanese sumo wrestling squat, the karate kick, and the moonwalk as examples.

Original languageEnglish
Title of host publicationAdvances in Multimedia Modeling - 19th International Conference, MMM 2013, Proceedings
Pages502-512
Number of pages11
Volume7732 LNCS
EditionPART 1
DOIs
Publication statusPublished - 2013
Externally publishedYes
Event19th International Conference on Advances in Multimedia Modeling, MMM 2013 - Huangshan
Duration: 2013 Jan 72013 Jan 9

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
NumberPART 1
Volume7732 LNCS
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Other

Other19th International Conference on Advances in Multimedia Modeling, MMM 2013
CityHuangshan
Period13/1/713/1/9

Fingerprint

Animation
Motion
Timing
Squat
Smoothing Algorithm
Dynamic Time Warping
Inaccurate
Eliminate
Vibration
Computing
Demonstrate

Keywords

  • Shape deformation
  • Video editing

ASJC Scopus subject areas

  • Theoretical Computer Science
  • Computer Science(all)

Cite this

Mizui, K., Okabe, M., & Onai, R. (2013). Choreographing amateur performers based on motion transfer between videos. In Advances in Multimedia Modeling - 19th International Conference, MMM 2013, Proceedings (PART 1 ed., Vol. 7732 LNCS, pp. 502-512). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 7732 LNCS, No. PART 1). https://doi.org/10.1007/978-3-642-35725-1_46

Choreographing amateur performers based on motion transfer between videos. / Mizui, Kenta; Okabe, Makoto; Onai, Rikio.

Advances in Multimedia Modeling - 19th International Conference, MMM 2013, Proceedings. Vol. 7732 LNCS PART 1. ed. 2013. p. 502-512 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 7732 LNCS, No. PART 1).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Mizui, K, Okabe, M & Onai, R 2013, Choreographing amateur performers based on motion transfer between videos. in Advances in Multimedia Modeling - 19th International Conference, MMM 2013, Proceedings. PART 1 edn, vol. 7732 LNCS, Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), no. PART 1, vol. 7732 LNCS, pp. 502-512, 19th International Conference on Advances in Multimedia Modeling, MMM 2013, Huangshan, 13/1/7. https://doi.org/10.1007/978-3-642-35725-1_46
Mizui K, Okabe M, Onai R. Choreographing amateur performers based on motion transfer between videos. In Advances in Multimedia Modeling - 19th International Conference, MMM 2013, Proceedings. PART 1 ed. Vol. 7732 LNCS. 2013. p. 502-512. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); PART 1). https://doi.org/10.1007/978-3-642-35725-1_46
Mizui, Kenta ; Okabe, Makoto ; Onai, Rikio. / Choreographing amateur performers based on motion transfer between videos. Advances in Multimedia Modeling - 19th International Conference, MMM 2013, Proceedings. Vol. 7732 LNCS PART 1. ed. 2013. pp. 502-512 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); PART 1).
@inproceedings{f414b9c07eac434f8de2bef248bd3e02,
title = "Choreographing amateur performers based on motion transfer between videos",
abstract = "We propose a technique for quickly and easily choreographing a video of an amateur performer by comparing it with a video of a corresponding professional performance. Our method allows the user to interactively edit the amateur performance in order to synchronize it with the professional performance in terms of timings and poses. In our system, the user first extracts the amateur and professional poses from every frame via semi-automatic video tracking. The system synchronizes the timings by computing dynamic time warping (DTW) between the two sets of video-tracking data, and then synchronizes the poses by applying image deformation to every frame. To eliminate unnatural vibrations, which often result from inaccurate video tracking, we apply an automatic motion-smoothing algorithm to the synthesized animation. We demonstrate that our method allows the user to successfully edit an amateur's performance into a more polished one, utilizing the Japanese sumo wrestling squat, the karate kick, and the moonwalk as examples.",
keywords = "Shape deformation, Video editing",
author = "Kenta Mizui and Makoto Okabe and Rikio Onai",
year = "2013",
doi = "10.1007/978-3-642-35725-1_46",
language = "English",
isbn = "9783642357244",
volume = "7732 LNCS",
series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",
number = "PART 1",
pages = "502--512",
booktitle = "Advances in Multimedia Modeling - 19th International Conference, MMM 2013, Proceedings",
edition = "PART 1",

}

TY - GEN

T1 - Choreographing amateur performers based on motion transfer between videos

AU - Mizui, Kenta

AU - Okabe, Makoto

AU - Onai, Rikio

PY - 2013

Y1 - 2013

N2 - We propose a technique for quickly and easily choreographing a video of an amateur performer by comparing it with a video of a corresponding professional performance. Our method allows the user to interactively edit the amateur performance in order to synchronize it with the professional performance in terms of timings and poses. In our system, the user first extracts the amateur and professional poses from every frame via semi-automatic video tracking. The system synchronizes the timings by computing dynamic time warping (DTW) between the two sets of video-tracking data, and then synchronizes the poses by applying image deformation to every frame. To eliminate unnatural vibrations, which often result from inaccurate video tracking, we apply an automatic motion-smoothing algorithm to the synthesized animation. We demonstrate that our method allows the user to successfully edit an amateur's performance into a more polished one, utilizing the Japanese sumo wrestling squat, the karate kick, and the moonwalk as examples.

AB - We propose a technique for quickly and easily choreographing a video of an amateur performer by comparing it with a video of a corresponding professional performance. Our method allows the user to interactively edit the amateur performance in order to synchronize it with the professional performance in terms of timings and poses. In our system, the user first extracts the amateur and professional poses from every frame via semi-automatic video tracking. The system synchronizes the timings by computing dynamic time warping (DTW) between the two sets of video-tracking data, and then synchronizes the poses by applying image deformation to every frame. To eliminate unnatural vibrations, which often result from inaccurate video tracking, we apply an automatic motion-smoothing algorithm to the synthesized animation. We demonstrate that our method allows the user to successfully edit an amateur's performance into a more polished one, utilizing the Japanese sumo wrestling squat, the karate kick, and the moonwalk as examples.

KW - Shape deformation

KW - Video editing

UR - http://www.scopus.com/inward/record.url?scp=84892853161&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84892853161&partnerID=8YFLogxK

U2 - 10.1007/978-3-642-35725-1_46

DO - 10.1007/978-3-642-35725-1_46

M3 - Conference contribution

SN - 9783642357244

VL - 7732 LNCS

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 502

EP - 512

BT - Advances in Multimedia Modeling - 19th International Conference, MMM 2013, Proceedings

ER -