Multiple description coding and recovery of free viewpoint video for wireless multi-path streaming

Zhi Liu, Gene Cheung, Jacob Chakareski, Yusheng Ji

研究成果: Article査読

46 被引用数 (Scopus)


By transmitting texture and depth videos captured from two nearby camera viewpoints, a client can synthesize via depth-image-based rendering (DIBR) any freely chosen intermediate virtual view of the 3D scene, enhancing the user's perception of depth. During wireless network transmission, burst packet losses can corrupt the transmitted texture and depth videos and degrade the synthesized view quality at the client. In this paper, we propose a multiple description coding system for multi-path transmission of free-viewpoint video, with joint interview and temporal description recovery capability. In particular, we encode separately the even frames of the left view and the odd frames of the right view, and transmit them as one description on one path. The second description comprises the remaining frames in the two views and is transmitted over a second path. If the receiver receives only one description due to burst loss in the other path, the missing frames in the other description are partially reconstructed using our frame recovery procedure. First, we construct two recovery candidates for each lost pixel in a frame. The first candidate is generated via temporal super-resolution from its predecessor and successor frames in the same view. The second candidate is generated via DIBR fromthe received frame of the same time instance in the other view. Next, we select the best pixel candidates one patch at a time, where an image patch corresponds to a neighborhood of pixels with similar depth values in the 3D scene. Near-optimal source and channel coding rates for each description are selected using a branch-and-bound method, for given transmission bandwidth on each path. Experimental results show that our systemcan outperform a traditional single-description/single-path transmission scheme by up to 5.5 dB in Peak Signal-to-Noise Ratio (PSNR) of the synthesized intermediate view at the client.

ジャーナルIEEE Journal on Selected Topics in Signal Processing
出版ステータスPublished - 2015 2月 1

ASJC Scopus subject areas

  • 電子工学および電気工学
  • 信号処理


「Multiple description coding and recovery of free viewpoint video for wireless multi-path streaming」の研究トピックを掘り下げます。これらがまとまってユニークなフィンガープリントを構成します。