A soundtrack generation system to synchronize the climax of a video clip with music

Haruki Sato; Tatsunori Hirai; Tomoyasu Nakano; Masataka Goto; Shigeo Morishima

doi:10.1109/ICME.2016.7552868

A soundtrack generation system to synchronize the climax of a video clip with music

Haruki Sato, Tatsunori Hirai, Tomoyasu Nakano, Masataka Goto, Shigeo Morishima

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

2 Citations (Scopus)

Abstract

In this paper, we present a soundtrack generation system that can automatically add a soundtrack with the length and climax points aligned to those of a video clip. Adding a soundtrack to a video clip is an important process in video editing. Editors tend to add chorus sections to the climax points of the video clip by replacing and concatenating musical segments. However, this process is time-consuming. Our system automatically detects climaxes of both the video clips and music based on feature extraction and analysis. This enables the system to add a soundtrack in which the climax is synchronized to the climax of the video clip. We evaluated the generated soundtracks through a subjective evaluation.

Original language	English
Title of host publication	2016 IEEE International Conference on Multimedia and Expo, ICME 2016
Publisher	IEEE Computer Society
Volume	2016-August
ISBN (Electronic)	9781467372589
DOIs	https://doi.org/10.1109/ICME.2016.7552868
Publication status	Published - 2016 Aug 25
Event	2016 IEEE International Conference on Multimedia and Expo, ICME 2016 - Seattle, United States Duration: 2016 Jul 11 → 2016 Jul 15

Other

Other	2016 IEEE International Conference on Multimedia and Expo, ICME 2016
Country	United States
City	Seattle
Period	16/7/11 → 16/7/15

Fingerprint

Keywords

audio-visual feature extraction
climax synchronization
soundtrack generation

ASJC Scopus subject areas

Computer Networks and Communications
Computer Science Applications

Cite this

Sato, H., Hirai, T., Nakano, T., Goto, M., & Morishima, S. (2016). A soundtrack generation system to synchronize the climax of a video clip with music. In 2016 IEEE International Conference on Multimedia and Expo, ICME 2016 (Vol. 2016-August). [7552868] IEEE Computer Society. https://doi.org/10.1109/ICME.2016.7552868

A soundtrack generation system to synchronize the climax of a video clip with music. / Sato, Haruki; Hirai, Tatsunori; Nakano, Tomoyasu; Goto, Masataka; Morishima, Shigeo.

2016 IEEE International Conference on Multimedia and Expo, ICME 2016. Vol. 2016-August IEEE Computer Society, 2016. 7552868.

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Sato, H, Hirai, T, Nakano, T, Goto, M & Morishima, S 2016, A soundtrack generation system to synchronize the climax of a video clip with music. in 2016 IEEE International Conference on Multimedia and Expo, ICME 2016. vol. 2016-August, 7552868, IEEE Computer Society, 2016 IEEE International Conference on Multimedia and Expo, ICME 2016, Seattle, United States, 16/7/11. https://doi.org/10.1109/ICME.2016.7552868

@inproceedings{fefcefbe8c2f4f6bbb08fba1fa6f77a7,

title = "A soundtrack generation system to synchronize the climax of a video clip with music",

abstract = "In this paper, we present a soundtrack generation system that can automatically add a soundtrack with the length and climax points aligned to those of a video clip. Adding a soundtrack to a video clip is an important process in video editing. Editors tend to add chorus sections to the climax points of the video clip by replacing and concatenating musical segments. However, this process is time-consuming. Our system automatically detects climaxes of both the video clips and music based on feature extraction and analysis. This enables the system to add a soundtrack in which the climax is synchronized to the climax of the video clip. We evaluated the generated soundtracks through a subjective evaluation.",

keywords = "audio-visual feature extraction, climax synchronization, soundtrack generation",

author = "Haruki Sato and Tatsunori Hirai and Tomoyasu Nakano and Masataka Goto and Shigeo Morishima",

year = "2016",

month = aug

day = "25",

doi = "10.1109/ICME.2016.7552868",

language = "English",

volume = "2016-August",

booktitle = "2016 IEEE International Conference on Multimedia and Expo, ICME 2016",

publisher = "IEEE Computer Society",

address = "United States",

note = "2016 IEEE International Conference on Multimedia and Expo, ICME 2016 ; Conference date: 11-07-2016 Through 15-07-2016",

}

TY - GEN

T1 - A soundtrack generation system to synchronize the climax of a video clip with music

AU - Sato, Haruki

AU - Hirai, Tatsunori

AU - Nakano, Tomoyasu

AU - Goto, Masataka

AU - Morishima, Shigeo

PY - 2016/8/25

Y1 - 2016/8/25

N2 - In this paper, we present a soundtrack generation system that can automatically add a soundtrack with the length and climax points aligned to those of a video clip. Adding a soundtrack to a video clip is an important process in video editing. Editors tend to add chorus sections to the climax points of the video clip by replacing and concatenating musical segments. However, this process is time-consuming. Our system automatically detects climaxes of both the video clips and music based on feature extraction and analysis. This enables the system to add a soundtrack in which the climax is synchronized to the climax of the video clip. We evaluated the generated soundtracks through a subjective evaluation.

AB - In this paper, we present a soundtrack generation system that can automatically add a soundtrack with the length and climax points aligned to those of a video clip. Adding a soundtrack to a video clip is an important process in video editing. Editors tend to add chorus sections to the climax points of the video clip by replacing and concatenating musical segments. However, this process is time-consuming. Our system automatically detects climaxes of both the video clips and music based on feature extraction and analysis. This enables the system to add a soundtrack in which the climax is synchronized to the climax of the video clip. We evaluated the generated soundtracks through a subjective evaluation.

KW - audio-visual feature extraction

KW - climax synchronization

KW - soundtrack generation

UR - http://www.scopus.com/inward/record.url?scp=84987667513&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84987667513&partnerID=8YFLogxK

U2 - 10.1109/ICME.2016.7552868

DO - 10.1109/ICME.2016.7552868

M3 - Conference contribution

AN - SCOPUS:84987667513

VL - 2016-August

BT - 2016 IEEE International Conference on Multimedia and Expo, ICME 2016

PB - IEEE Computer Society

T2 - 2016 IEEE International Conference on Multimedia and Expo, ICME 2016

Y2 - 11 July 2016 through 15 July 2016

ER -

Access to Document

10.1109/ICME.2016.7552868