Performance Analysis of Generated Predictive Frames Using PredNet Bi-directionally

Kanato Sakama*, Shunichi Sekiguchi, Wataru Kameyama

*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

For generating motion-compensated predictive frames, which is one of the video coding processes, there has been a lot of studies on using DNN without using motion vectors. Conventional methods of generating motion-compensated predictive frames using DNN use only the source frames for prediction in the forward direction. However, in the ever-standardized video coding schemes, it has been confirmed that the bi-directional prediction, e.g., B-picture, improves coding efficiency. Thus, for generating motion-compensated predictive frames to be used in video coding, we propose to apply PredNet bidirectionally, that is a future frame generation model using DNN based on the prediction process of visual input stimuli in brain. In this paper, the performance of the predictive frames generated by the proposed method is evaluated by using MSE and SSIM compared with the prediction accuracy applying PredNet only to the forward direction. In addition, we also investigate whether the prediction accuracy of the predicted frames can be improved by increasing the amount of training frames in videos chosen from YouTube-8M. The results show the effectiveness of the proposed method in terms of less prediction error compared with the forward-only PredNet, as well as the performance increasing by more training data.

Original languageEnglish
Title of host publicationInternational Workshop on Advanced Imaging Technology, IWAIT 2022
EditorsMasayuki Nakajima, Shogo Muramatsu, Jae-Gon Kim, Jing-Ming Guo, Qian Kemao
PublisherSPIE
ISBN (Electronic)9781510653313
DOIs
Publication statusPublished - 2022
Event2022 International Workshop on Advanced Imaging Technology, IWAIT 2022 - Hong Kong, China
Duration: 2022 Jan 42022 Jan 6

Publication series

NameProceedings of SPIE - The International Society for Optical Engineering
Volume12177
ISSN (Print)0277-786X
ISSN (Electronic)1996-756X

Conference

Conference2022 International Workshop on Advanced Imaging Technology, IWAIT 2022
Country/TerritoryChina
CityHong Kong
Period22/1/422/1/6

Keywords

  • Bi-directional prediction
  • DNN
  • Motion compensation
  • PredNet
  • Video coding

ASJC Scopus subject areas

  • Electronic, Optical and Magnetic Materials
  • Condensed Matter Physics
  • Computer Science Applications
  • Applied Mathematics
  • Electrical and Electronic Engineering

Fingerprint

Dive into the research topics of 'Performance Analysis of Generated Predictive Frames Using PredNet Bi-directionally'. Together they form a unique fingerprint.

Cite this