Multiple description coding of free viewpoint video for multi-path network streaming

Zhi Liu, Gene Cheung, Jacob Chakareski, Yusheng Ji

Research output: Chapter in Book/Report/Conference proceedingConference contribution

13 Citations (Scopus)

Abstract

By transmitting texture and depth videos from two adjacent captured viewpoints, a client can synthesize via depth-image-based rendering (DIBR) any intermediate virtual view of the scene, determined by the dynamic movement of the client's head. In so doing, depth perception of the 3D scene will be created through motion parallax. Due to the stringent playback deadline of interactive free viewpoint video, burst packet losses in the texture and depth video streams caused by transmission over unreliable channels are difficult to overcome and can severely degrade the synthesized view quality at the client. We propose a multiple description coding (MDC) of free viewpoint video in texture-plus-depth format that will be transmitted on two disjoint network paths. Specifically, we encode even frames of the left view and odd frames of the right view separately as one description and transmit it on path one. Similarly, we encode odd frames of the left view and even frames of the right view as the second description and transmit it on path two. Appropriate quantization parameters (QP) are selected for each description, such that its data rate matches optimally the available transmission bandwidth on each of the two paths. If the receiver receives one description but not the other due to burst loss on one of the paths, it can still partially reconstruct the missing frames in the loss-corrupted description using a computationally efficient DIBR-based recovery scheme that we design. Extensive experimental results show that our MDC streaming system can outperform the traditional single-path single-description transmission scheme by up to 7dB in Peak Signal-to-Noise Ratio (PSNR) of the synthesized intermediate view at the receiving client.

Original languageEnglish
Title of host publicationGLOBECOM - IEEE Global Telecommunications Conference
Pages2150-2155
Number of pages6
DOIs
Publication statusPublished - 2012
Externally publishedYes
Event2012 IEEE Global Communications Conference, GLOBECOM 2012 - Anaheim, CA, United States
Duration: 2012 Dec 32012 Dec 7

Other

Other2012 IEEE Global Communications Conference, GLOBECOM 2012
CountryUnited States
CityAnaheim, CA
Period12/12/312/12/7

Fingerprint

Textures
Depth perception
Packet loss
Signal to noise ratio
Bandwidth
Recovery

ASJC Scopus subject areas

  • Electrical and Electronic Engineering

Cite this

Liu, Z., Cheung, G., Chakareski, J., & Ji, Y. (2012). Multiple description coding of free viewpoint video for multi-path network streaming. In GLOBECOM - IEEE Global Telecommunications Conference (pp. 2150-2155). [6503434] https://doi.org/10.1109/GLOCOM.2012.6503434

Multiple description coding of free viewpoint video for multi-path network streaming. / Liu, Zhi; Cheung, Gene; Chakareski, Jacob; Ji, Yusheng.

GLOBECOM - IEEE Global Telecommunications Conference. 2012. p. 2150-2155 6503434.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Liu, Z, Cheung, G, Chakareski, J & Ji, Y 2012, Multiple description coding of free viewpoint video for multi-path network streaming. in GLOBECOM - IEEE Global Telecommunications Conference., 6503434, pp. 2150-2155, 2012 IEEE Global Communications Conference, GLOBECOM 2012, Anaheim, CA, United States, 12/12/3. https://doi.org/10.1109/GLOCOM.2012.6503434
Liu Z, Cheung G, Chakareski J, Ji Y. Multiple description coding of free viewpoint video for multi-path network streaming. In GLOBECOM - IEEE Global Telecommunications Conference. 2012. p. 2150-2155. 6503434 https://doi.org/10.1109/GLOCOM.2012.6503434
Liu, Zhi ; Cheung, Gene ; Chakareski, Jacob ; Ji, Yusheng. / Multiple description coding of free viewpoint video for multi-path network streaming. GLOBECOM - IEEE Global Telecommunications Conference. 2012. pp. 2150-2155
@inproceedings{7c2a596f6f22425eba2bababc18087b3,
title = "Multiple description coding of free viewpoint video for multi-path network streaming",
abstract = "By transmitting texture and depth videos from two adjacent captured viewpoints, a client can synthesize via depth-image-based rendering (DIBR) any intermediate virtual view of the scene, determined by the dynamic movement of the client's head. In so doing, depth perception of the 3D scene will be created through motion parallax. Due to the stringent playback deadline of interactive free viewpoint video, burst packet losses in the texture and depth video streams caused by transmission over unreliable channels are difficult to overcome and can severely degrade the synthesized view quality at the client. We propose a multiple description coding (MDC) of free viewpoint video in texture-plus-depth format that will be transmitted on two disjoint network paths. Specifically, we encode even frames of the left view and odd frames of the right view separately as one description and transmit it on path one. Similarly, we encode odd frames of the left view and even frames of the right view as the second description and transmit it on path two. Appropriate quantization parameters (QP) are selected for each description, such that its data rate matches optimally the available transmission bandwidth on each of the two paths. If the receiver receives one description but not the other due to burst loss on one of the paths, it can still partially reconstruct the missing frames in the loss-corrupted description using a computationally efficient DIBR-based recovery scheme that we design. Extensive experimental results show that our MDC streaming system can outperform the traditional single-path single-description transmission scheme by up to 7dB in Peak Signal-to-Noise Ratio (PSNR) of the synthesized intermediate view at the receiving client.",
author = "Zhi Liu and Gene Cheung and Jacob Chakareski and Yusheng Ji",
year = "2012",
doi = "10.1109/GLOCOM.2012.6503434",
language = "English",
isbn = "9781467309219",
pages = "2150--2155",
booktitle = "GLOBECOM - IEEE Global Telecommunications Conference",

}

TY - GEN

T1 - Multiple description coding of free viewpoint video for multi-path network streaming

AU - Liu, Zhi

AU - Cheung, Gene

AU - Chakareski, Jacob

AU - Ji, Yusheng

PY - 2012

Y1 - 2012

N2 - By transmitting texture and depth videos from two adjacent captured viewpoints, a client can synthesize via depth-image-based rendering (DIBR) any intermediate virtual view of the scene, determined by the dynamic movement of the client's head. In so doing, depth perception of the 3D scene will be created through motion parallax. Due to the stringent playback deadline of interactive free viewpoint video, burst packet losses in the texture and depth video streams caused by transmission over unreliable channels are difficult to overcome and can severely degrade the synthesized view quality at the client. We propose a multiple description coding (MDC) of free viewpoint video in texture-plus-depth format that will be transmitted on two disjoint network paths. Specifically, we encode even frames of the left view and odd frames of the right view separately as one description and transmit it on path one. Similarly, we encode odd frames of the left view and even frames of the right view as the second description and transmit it on path two. Appropriate quantization parameters (QP) are selected for each description, such that its data rate matches optimally the available transmission bandwidth on each of the two paths. If the receiver receives one description but not the other due to burst loss on one of the paths, it can still partially reconstruct the missing frames in the loss-corrupted description using a computationally efficient DIBR-based recovery scheme that we design. Extensive experimental results show that our MDC streaming system can outperform the traditional single-path single-description transmission scheme by up to 7dB in Peak Signal-to-Noise Ratio (PSNR) of the synthesized intermediate view at the receiving client.

AB - By transmitting texture and depth videos from two adjacent captured viewpoints, a client can synthesize via depth-image-based rendering (DIBR) any intermediate virtual view of the scene, determined by the dynamic movement of the client's head. In so doing, depth perception of the 3D scene will be created through motion parallax. Due to the stringent playback deadline of interactive free viewpoint video, burst packet losses in the texture and depth video streams caused by transmission over unreliable channels are difficult to overcome and can severely degrade the synthesized view quality at the client. We propose a multiple description coding (MDC) of free viewpoint video in texture-plus-depth format that will be transmitted on two disjoint network paths. Specifically, we encode even frames of the left view and odd frames of the right view separately as one description and transmit it on path one. Similarly, we encode odd frames of the left view and even frames of the right view as the second description and transmit it on path two. Appropriate quantization parameters (QP) are selected for each description, such that its data rate matches optimally the available transmission bandwidth on each of the two paths. If the receiver receives one description but not the other due to burst loss on one of the paths, it can still partially reconstruct the missing frames in the loss-corrupted description using a computationally efficient DIBR-based recovery scheme that we design. Extensive experimental results show that our MDC streaming system can outperform the traditional single-path single-description transmission scheme by up to 7dB in Peak Signal-to-Noise Ratio (PSNR) of the synthesized intermediate view at the receiving client.

UR - http://www.scopus.com/inward/record.url?scp=84877680100&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84877680100&partnerID=8YFLogxK

U2 - 10.1109/GLOCOM.2012.6503434

DO - 10.1109/GLOCOM.2012.6503434

M3 - Conference contribution

AN - SCOPUS:84877680100

SN - 9781467309219

SP - 2150

EP - 2155

BT - GLOBECOM - IEEE Global Telecommunications Conference

ER -