DanceDJ: A 3D dance animation authoring system for live performance

Naoya Iwamoto, Takuya Kato, Hubert P.H. Shum, Ryo Kakitsuka, Kenta Hara, Shigeo Morishima

    Research output: Chapter in Book/Report/Conference proceedingConference contribution

    Abstract

    Dance is an important component of live performance for expressing emotion and presenting visual context. Human dance performances typically require expert knowledge of dance choreography and professional rehearsal, which are too costly for casual entertainment venues and clubs. Recent advancements in character animation and motion synthesis have made it possible to synthesize virtual 3D dance characters in real-time. The major problem in existing systems is a lack of an intuitive interfaces to control the animation for real-time dance controls. We propose a new system called the DanceDJ to solve this problem. Our system consists of two parts. The first part is an underlying motion analysis system that evaluates motion features including dance features such as the postures and movement tempo, as well as audio features such as the music tempo and structure. As a pre-process, given a dancing motion database, our system evaluates the quality of possible timings to connect and switch different dancing motions. During run-time, we propose a control interface that provides visual guidance. We observe that disk jockeys (DJs) effectively control the mixing of music using the DJ controller, and therefore propose a DJ controller for controlling dancing characters. This allows DJs to transfer their skills from music control to dance control using a similar hardware setup. We map different motion control functions onto the DJ controller, and visualize the timing of natural connection points, such that the DJ can effectively govern the synthesized dance motion. We conducted two user experiments to evaluate the user experience and the quality of the dance character. Quantitative analysis shows that our system performs well in both motion control and simulation quality.

    Original languageEnglish
    Title of host publicationAdvances in Computer Entertainment Technology - 14th International Conference, ACE 2017, Proceedings
    PublisherSpringer-Verlag
    Pages653-670
    Number of pages18
    ISBN (Print)9783319762692
    DOIs
    Publication statusPublished - 2018 Jan 1
    Event14th International Conference on Advances in Computer Entertainment Technology, ACE 2017 - London, United Kingdom
    Duration: 2017 Dec 142017 Dec 16

    Publication series

    NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
    Volume10714 LNCS
    ISSN (Print)0302-9743
    ISSN (Electronic)1611-3349

    Other

    Other14th International Conference on Advances in Computer Entertainment Technology, ACE 2017
    CountryUnited Kingdom
    CityLondon
    Period17/12/1417/12/16

    Fingerprint

    Authoring
    Animation
    Music
    Motion
    Motion control
    Motion Control
    Controllers
    Controller
    Evaluate
    Timing
    Character Animation
    Motion Simulation
    Choreography
    Real-time
    Real time control
    Human Performance
    Motion Analysis
    Control Function
    User Experience
    Database Systems

    Keywords

    • Character animation
    • Dance
    • DJ controllers
    • Human-computer interaction

    ASJC Scopus subject areas

    • Theoretical Computer Science
    • Computer Science(all)

    Cite this

    Iwamoto, N., Kato, T., Shum, H. P. H., Kakitsuka, R., Hara, K., & Morishima, S. (2018). DanceDJ: A 3D dance animation authoring system for live performance. In Advances in Computer Entertainment Technology - 14th International Conference, ACE 2017, Proceedings (pp. 653-670). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 10714 LNCS). Springer-Verlag. https://doi.org/10.1007/978-3-319-76270-8_46

    DanceDJ : A 3D dance animation authoring system for live performance. / Iwamoto, Naoya; Kato, Takuya; Shum, Hubert P.H.; Kakitsuka, Ryo; Hara, Kenta; Morishima, Shigeo.

    Advances in Computer Entertainment Technology - 14th International Conference, ACE 2017, Proceedings. Springer-Verlag, 2018. p. 653-670 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 10714 LNCS).

    Research output: Chapter in Book/Report/Conference proceedingConference contribution

    Iwamoto, N, Kato, T, Shum, HPH, Kakitsuka, R, Hara, K & Morishima, S 2018, DanceDJ: A 3D dance animation authoring system for live performance. in Advances in Computer Entertainment Technology - 14th International Conference, ACE 2017, Proceedings. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 10714 LNCS, Springer-Verlag, pp. 653-670, 14th International Conference on Advances in Computer Entertainment Technology, ACE 2017, London, United Kingdom, 17/12/14. https://doi.org/10.1007/978-3-319-76270-8_46
    Iwamoto N, Kato T, Shum HPH, Kakitsuka R, Hara K, Morishima S. DanceDJ: A 3D dance animation authoring system for live performance. In Advances in Computer Entertainment Technology - 14th International Conference, ACE 2017, Proceedings. Springer-Verlag. 2018. p. 653-670. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). https://doi.org/10.1007/978-3-319-76270-8_46
    Iwamoto, Naoya ; Kato, Takuya ; Shum, Hubert P.H. ; Kakitsuka, Ryo ; Hara, Kenta ; Morishima, Shigeo. / DanceDJ : A 3D dance animation authoring system for live performance. Advances in Computer Entertainment Technology - 14th International Conference, ACE 2017, Proceedings. Springer-Verlag, 2018. pp. 653-670 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).
    @inproceedings{e199502bbb454a6692d7ff1ff79a5ddd,
    title = "DanceDJ: A 3D dance animation authoring system for live performance",
    abstract = "Dance is an important component of live performance for expressing emotion and presenting visual context. Human dance performances typically require expert knowledge of dance choreography and professional rehearsal, which are too costly for casual entertainment venues and clubs. Recent advancements in character animation and motion synthesis have made it possible to synthesize virtual 3D dance characters in real-time. The major problem in existing systems is a lack of an intuitive interfaces to control the animation for real-time dance controls. We propose a new system called the DanceDJ to solve this problem. Our system consists of two parts. The first part is an underlying motion analysis system that evaluates motion features including dance features such as the postures and movement tempo, as well as audio features such as the music tempo and structure. As a pre-process, given a dancing motion database, our system evaluates the quality of possible timings to connect and switch different dancing motions. During run-time, we propose a control interface that provides visual guidance. We observe that disk jockeys (DJs) effectively control the mixing of music using the DJ controller, and therefore propose a DJ controller for controlling dancing characters. This allows DJs to transfer their skills from music control to dance control using a similar hardware setup. We map different motion control functions onto the DJ controller, and visualize the timing of natural connection points, such that the DJ can effectively govern the synthesized dance motion. We conducted two user experiments to evaluate the user experience and the quality of the dance character. Quantitative analysis shows that our system performs well in both motion control and simulation quality.",
    keywords = "Character animation, Dance, DJ controllers, Human-computer interaction",
    author = "Naoya Iwamoto and Takuya Kato and Shum, {Hubert P.H.} and Ryo Kakitsuka and Kenta Hara and Shigeo Morishima",
    year = "2018",
    month = "1",
    day = "1",
    doi = "10.1007/978-3-319-76270-8_46",
    language = "English",
    isbn = "9783319762692",
    series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",
    publisher = "Springer-Verlag",
    pages = "653--670",
    booktitle = "Advances in Computer Entertainment Technology - 14th International Conference, ACE 2017, Proceedings",

    }

    TY - GEN

    T1 - DanceDJ

    T2 - A 3D dance animation authoring system for live performance

    AU - Iwamoto, Naoya

    AU - Kato, Takuya

    AU - Shum, Hubert P.H.

    AU - Kakitsuka, Ryo

    AU - Hara, Kenta

    AU - Morishima, Shigeo

    PY - 2018/1/1

    Y1 - 2018/1/1

    N2 - Dance is an important component of live performance for expressing emotion and presenting visual context. Human dance performances typically require expert knowledge of dance choreography and professional rehearsal, which are too costly for casual entertainment venues and clubs. Recent advancements in character animation and motion synthesis have made it possible to synthesize virtual 3D dance characters in real-time. The major problem in existing systems is a lack of an intuitive interfaces to control the animation for real-time dance controls. We propose a new system called the DanceDJ to solve this problem. Our system consists of two parts. The first part is an underlying motion analysis system that evaluates motion features including dance features such as the postures and movement tempo, as well as audio features such as the music tempo and structure. As a pre-process, given a dancing motion database, our system evaluates the quality of possible timings to connect and switch different dancing motions. During run-time, we propose a control interface that provides visual guidance. We observe that disk jockeys (DJs) effectively control the mixing of music using the DJ controller, and therefore propose a DJ controller for controlling dancing characters. This allows DJs to transfer their skills from music control to dance control using a similar hardware setup. We map different motion control functions onto the DJ controller, and visualize the timing of natural connection points, such that the DJ can effectively govern the synthesized dance motion. We conducted two user experiments to evaluate the user experience and the quality of the dance character. Quantitative analysis shows that our system performs well in both motion control and simulation quality.

    AB - Dance is an important component of live performance for expressing emotion and presenting visual context. Human dance performances typically require expert knowledge of dance choreography and professional rehearsal, which are too costly for casual entertainment venues and clubs. Recent advancements in character animation and motion synthesis have made it possible to synthesize virtual 3D dance characters in real-time. The major problem in existing systems is a lack of an intuitive interfaces to control the animation for real-time dance controls. We propose a new system called the DanceDJ to solve this problem. Our system consists of two parts. The first part is an underlying motion analysis system that evaluates motion features including dance features such as the postures and movement tempo, as well as audio features such as the music tempo and structure. As a pre-process, given a dancing motion database, our system evaluates the quality of possible timings to connect and switch different dancing motions. During run-time, we propose a control interface that provides visual guidance. We observe that disk jockeys (DJs) effectively control the mixing of music using the DJ controller, and therefore propose a DJ controller for controlling dancing characters. This allows DJs to transfer their skills from music control to dance control using a similar hardware setup. We map different motion control functions onto the DJ controller, and visualize the timing of natural connection points, such that the DJ can effectively govern the synthesized dance motion. We conducted two user experiments to evaluate the user experience and the quality of the dance character. Quantitative analysis shows that our system performs well in both motion control and simulation quality.

    KW - Character animation

    KW - Dance

    KW - DJ controllers

    KW - Human-computer interaction

    UR - http://www.scopus.com/inward/record.url?scp=85043499069&partnerID=8YFLogxK

    UR - http://www.scopus.com/inward/citedby.url?scp=85043499069&partnerID=8YFLogxK

    U2 - 10.1007/978-3-319-76270-8_46

    DO - 10.1007/978-3-319-76270-8_46

    M3 - Conference contribution

    AN - SCOPUS:85043499069

    SN - 9783319762692

    T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

    SP - 653

    EP - 670

    BT - Advances in Computer Entertainment Technology - 14th International Conference, ACE 2017, Proceedings

    PB - Springer-Verlag

    ER -