A music video authoring system synchronizing climax of video clips and music via rearrangement of musical bars

Haruki Sato, Tatsunori Hirai, Tomoyasu Nakano, Masataka Goto, Shigeo Morishima

研究成果: Conference contribution

3 被引用数 (Scopus)

抄録

This paper presents a system that can automatically add a sound-track to a video clip by replacing and concatenating an existing song's musical bars considering a user's preference. Since a soundtrack makes a video clip attractive, adding a soundtrack to a clip is one of the most important processes in video editing. To make a video clip more attractive, an editor of the clip tends to add a soundtrack considering its timing and climax. For example, editors often add chorus sections to the climax of the clip by re-placing and concatenating musical bars in an existing song. How-ever, in the process, editors should take naturalness of rearranged soundtrack into account. Therefore, editors have to decide how to replace musical bars in a song considering its timing, climax, and naturalness of rearranged soundtrack simultaneously. In this case, editors are required to optimize the soundtrack by listening to the rearranged result as well as checking the naturalness and synchro-nization between the result and the video clip. However, this repe-titious work is time-consuming. [Feng et al. 2010] proposed an automatic soundtrack addition method. However, since this meth-od automatically adds soundtrack with data-driven approach, this method cannot consider timing and climax which a user prefers. Our system takes all the patterns of rearranged musical bars into account and finds the most natural soundtrack considering a user's preference of intention for an audio-visual alignment and a climax of the resulting soundtrack. Specifically, musical sections between user's specified points and the beginning and the ending of the song are automatically interpolated by replacing and concatenat-ing musical bars based on dynamic programming. To consider user's intention for a climax of the soundtrack, the system allows the user to specify the intended climax by an editing interface. The system immediately reflects the intention and the soundtrack will be interactively re-rearranged. These semi-automated pro-cesses of rearranging soundtrack for a video clip help the users adding songs without considering naturalness of the rearranged song by their own.

本文言語English
ホスト出版物のタイトルACM SIGGRAPH 2015 Posters, SIGGRAPH 2015
出版社Association for Computing Machinery, Inc
ISBN(電子版)9781450336321
DOI
出版ステータスPublished - 2015 7 31
イベントInternational Conference on Computer Graphics and Interactive Techniques, SIGGRAPH 2015 - Los Angeles, United States
継続期間: 2015 8 92015 8 13

出版物シリーズ

名前ACM SIGGRAPH 2015 Posters, SIGGRAPH 2015

Other

OtherInternational Conference on Computer Graphics and Interactive Techniques, SIGGRAPH 2015
CountryUnited States
CityLos Angeles
Period15/8/915/8/13

ASJC Scopus subject areas

  • Computer Graphics and Computer-Aided Design
  • Computer Vision and Pattern Recognition
  • Human-Computer Interaction

フィンガープリント 「A music video authoring system synchronizing climax of video clips and music via rearrangement of musical bars」の研究トピックを掘り下げます。これらがまとまってユニークなフィンガープリントを構成します。

引用スタイル