Generating Bird's Eye View from Egocentric RGB Videos

Vanita Jain, Qiming Wu, Shivam Grover, Kshitij Sidana, Gopal Chaudhary, San Hlaing Myint, Qiaozhi Hua*


研究成果: Article査読


In this paper, we present a method for generating bird's eye video from egocentric RGB videos. Working with egocentric views is tricky since such the view is highly warped and prone to occlusions. On the other hand, a bird's eye view has a consistent scaling in at least the two dimensions it shows. Moreover, most of the state-of-the-art systems for tasks such as path prediction are built for bird's eye views of the subjects. We present a deep learning-based approach that transfers the egocentric RGB images captured from a dashcam of a car to bird's eye view. This is a task of view translation, and we perform two experiments. The first one uses an image-to-image translation method, and the other uses a video-to-video translation. We compare the results of our work with homographic transformation, and our SSIM values are better by a margin of 77% and 14.4%, and the RMSE errors are lower by 40% and 14.6% for image-to-image translation and video-to-video translation, respectively. We also visually show the efficacy and limitations of each method with helpful insights for future research. Compared to previous works that use homography and LIDAR for 3D point clouds, our work is more generalizable and does not require any expensive equipment.

ジャーナルWireless Communications and Mobile Computing
出版ステータスPublished - 2021

ASJC Scopus subject areas

  • 情報システム
  • コンピュータ ネットワークおよび通信
  • 電子工学および電気工学


「Generating Bird's Eye View from Egocentric RGB Videos」の研究トピックを掘り下げます。これらがまとまってユニークなフィンガープリントを構成します。