An integrated hole-filling algorithm for view synthesis

Wenxin Yu, Weichen Wang, Minghui Wang, Satoshi Goto

Research output: Contribution to journalArticle

Abstract

Multi-view video can provide users with three-dimensional (3-D) and virtual reality perception through multiple viewing angles. In recent years, depth image-based rendering (DIBR) has been generally used to synthesize virtual view images in free viewpoint television (FTV) and 3-D video. To conceal the zero-region more accurately and improve the quality of a virtual view synthesized frame, an integrated holefilling algorithm for view synthesis is proposed in this paper. The proposed algorithm contains five parts: an algorithm for distinguishing different regions, foreground and background boundary detection, texture image isophotes detection, a textural and structural isophote prediction algorithm, and an in-painting algorithm with gradient priority order. Based on the texture isophote prediction with a geometrical principle and the in-painting algorithm with a gradient priority order, the boundary information of the foreground is considerably clearer and the texture information in the zeroregion can be concealed much more accurately than in previous works. The vision quality mainly depends on the distortion of the structural information. Experimental results indicate that the proposed algorithm improves not only the objective quality of the virtual image, but also its subjective quality considerably; human vision is also clearly improved based on the subjective results. In particular, the algorithm ensures the boundary contours of the foreground objects and the textural and structural information.

Original languageEnglish
Pages (from-to)1306-1314
Number of pages9
JournalIEICE Transactions on Fundamentals of Electronics, Communications and Computer Sciences
VolumeE96-A
Issue number6
DOIs
Publication statusPublished - 2013 Jun

Fingerprint

Synthesis
Texture
Inpainting
Textures
Painting
Image-based Rendering
Gradient
3D Video
Boundary Detection
Human Vision
Prediction
Virtual Reality
Television
Virtual reality
Angle
Three-dimensional
Zero
Experimental Results

Keywords

  • DIBR
  • Hole-filling
  • In-painting
  • Virtual view

ASJC Scopus subject areas

  • Electrical and Electronic Engineering
  • Computer Graphics and Computer-Aided Design
  • Applied Mathematics
  • Signal Processing

Cite this

An integrated hole-filling algorithm for view synthesis. / Yu, Wenxin; Wang, Weichen; Wang, Minghui; Goto, Satoshi.

In: IEICE Transactions on Fundamentals of Electronics, Communications and Computer Sciences, Vol. E96-A, No. 6, 06.2013, p. 1306-1314.

Research output: Contribution to journalArticle

Yu, Wenxin ; Wang, Weichen ; Wang, Minghui ; Goto, Satoshi. / An integrated hole-filling algorithm for view synthesis. In: IEICE Transactions on Fundamentals of Electronics, Communications and Computer Sciences. 2013 ; Vol. E96-A, No. 6. pp. 1306-1314.
@article{0406c450969847b7a773725862ec409d,
title = "An integrated hole-filling algorithm for view synthesis",
abstract = "Multi-view video can provide users with three-dimensional (3-D) and virtual reality perception through multiple viewing angles. In recent years, depth image-based rendering (DIBR) has been generally used to synthesize virtual view images in free viewpoint television (FTV) and 3-D video. To conceal the zero-region more accurately and improve the quality of a virtual view synthesized frame, an integrated holefilling algorithm for view synthesis is proposed in this paper. The proposed algorithm contains five parts: an algorithm for distinguishing different regions, foreground and background boundary detection, texture image isophotes detection, a textural and structural isophote prediction algorithm, and an in-painting algorithm with gradient priority order. Based on the texture isophote prediction with a geometrical principle and the in-painting algorithm with a gradient priority order, the boundary information of the foreground is considerably clearer and the texture information in the zeroregion can be concealed much more accurately than in previous works. The vision quality mainly depends on the distortion of the structural information. Experimental results indicate that the proposed algorithm improves not only the objective quality of the virtual image, but also its subjective quality considerably; human vision is also clearly improved based on the subjective results. In particular, the algorithm ensures the boundary contours of the foreground objects and the textural and structural information.",
keywords = "DIBR, Hole-filling, In-painting, Virtual view",
author = "Wenxin Yu and Weichen Wang and Minghui Wang and Satoshi Goto",
year = "2013",
month = "6",
doi = "10.1587/transfun.E96.A.1306",
language = "English",
volume = "E96-A",
pages = "1306--1314",
journal = "IEICE Transactions on Fundamentals of Electronics, Communications and Computer Sciences",
issn = "0916-8508",
publisher = "Maruzen Co., Ltd/Maruzen Kabushikikaisha",
number = "6",

}

TY - JOUR

T1 - An integrated hole-filling algorithm for view synthesis

AU - Yu, Wenxin

AU - Wang, Weichen

AU - Wang, Minghui

AU - Goto, Satoshi

PY - 2013/6

Y1 - 2013/6

N2 - Multi-view video can provide users with three-dimensional (3-D) and virtual reality perception through multiple viewing angles. In recent years, depth image-based rendering (DIBR) has been generally used to synthesize virtual view images in free viewpoint television (FTV) and 3-D video. To conceal the zero-region more accurately and improve the quality of a virtual view synthesized frame, an integrated holefilling algorithm for view synthesis is proposed in this paper. The proposed algorithm contains five parts: an algorithm for distinguishing different regions, foreground and background boundary detection, texture image isophotes detection, a textural and structural isophote prediction algorithm, and an in-painting algorithm with gradient priority order. Based on the texture isophote prediction with a geometrical principle and the in-painting algorithm with a gradient priority order, the boundary information of the foreground is considerably clearer and the texture information in the zeroregion can be concealed much more accurately than in previous works. The vision quality mainly depends on the distortion of the structural information. Experimental results indicate that the proposed algorithm improves not only the objective quality of the virtual image, but also its subjective quality considerably; human vision is also clearly improved based on the subjective results. In particular, the algorithm ensures the boundary contours of the foreground objects and the textural and structural information.

AB - Multi-view video can provide users with three-dimensional (3-D) and virtual reality perception through multiple viewing angles. In recent years, depth image-based rendering (DIBR) has been generally used to synthesize virtual view images in free viewpoint television (FTV) and 3-D video. To conceal the zero-region more accurately and improve the quality of a virtual view synthesized frame, an integrated holefilling algorithm for view synthesis is proposed in this paper. The proposed algorithm contains five parts: an algorithm for distinguishing different regions, foreground and background boundary detection, texture image isophotes detection, a textural and structural isophote prediction algorithm, and an in-painting algorithm with gradient priority order. Based on the texture isophote prediction with a geometrical principle and the in-painting algorithm with a gradient priority order, the boundary information of the foreground is considerably clearer and the texture information in the zeroregion can be concealed much more accurately than in previous works. The vision quality mainly depends on the distortion of the structural information. Experimental results indicate that the proposed algorithm improves not only the objective quality of the virtual image, but also its subjective quality considerably; human vision is also clearly improved based on the subjective results. In particular, the algorithm ensures the boundary contours of the foreground objects and the textural and structural information.

KW - DIBR

KW - Hole-filling

KW - In-painting

KW - Virtual view

UR - http://www.scopus.com/inward/record.url?scp=84878621964&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84878621964&partnerID=8YFLogxK

U2 - 10.1587/transfun.E96.A.1306

DO - 10.1587/transfun.E96.A.1306

M3 - Article

AN - SCOPUS:84878621964

VL - E96-A

SP - 1306

EP - 1314

JO - IEICE Transactions on Fundamentals of Electronics, Communications and Computer Sciences

JF - IEICE Transactions on Fundamentals of Electronics, Communications and Computer Sciences

SN - 0916-8508

IS - 6

ER -