A selective fusion module for video super resolution with recurrent architecture

Zichen Gong, Toshiya Hori, Hiroshi Watanabe, Tomohiro Ikai, Takeshi Chujoh, Eiichi Sasaki, Norio Ito

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

As an important subtask of video restoration, video super-resolution has attracted a lot of attention in the community as it can eventually promote a wide range of technologies, e.g., video transmission system. Recent video super resolution model1 achieves cutting-edge performance. It efficiently utilizes recurrent architecture with neural networks to gradually aggregate details from previous frames. Nevertheless, this method faces a serious drawback that it is sensitive to occlusion, blur, and large motion changes since it only takes the previous generated output as recurrent input for the super resolution model. This will lead to undesirable rapid information loss during the recurrently generating process, and performance will therefore be dramatically decreased. Our works focus on addressing the issue of rapid information loss in video super-resolution model with recurrent architecture. By producing attention maps through selective fusion module, the recurrent model can adaptively aggregate necessary details across all previously generated high-resolution (HR) frames according to their informativeness. The proposed method is useful for preserving high frequency details collected progressively from each frame while being capable of removing noisy artifacts. This significantly improves the average quality of the super resolution video.

Original languageEnglish
Title of host publicationInternational Workshop on Advanced Imaging Technology, IWAIT 2020
EditorsPhooi Yee Lau, Mohammad Shobri
PublisherSPIE
ISBN (Electronic)9781510638358
DOIs
Publication statusPublished - 2020
EventInternational Workshop on Advanced Imaging Technology, IWAIT 2020 - Yogyakarta, Indonesia
Duration: 2020 Jan 52020 Jan 7

Publication series

NameProceedings of SPIE - The International Society for Optical Engineering
Volume11515
ISSN (Print)0277-786X
ISSN (Electronic)1996-756X

Conference

ConferenceInternational Workshop on Advanced Imaging Technology, IWAIT 2020
CountryIndonesia
CityYogyakarta
Period20/1/520/1/7

Keywords

  • Recurrent networks
  • Selective fusion
  • Video super resolution
  • Video transmission system

ASJC Scopus subject areas

  • Electronic, Optical and Magnetic Materials
  • Condensed Matter Physics
  • Computer Science Applications
  • Applied Mathematics
  • Electrical and Electronic Engineering

Fingerprint Dive into the research topics of 'A selective fusion module for video super resolution with recurrent architecture'. Together they form a unique fingerprint.

Cite this