Generating Bird's Eye View from Egocentric RGB Videos

Vanita Jain, Qiming Wu, Shivam Grover, Kshitij Sidana, Gopal Chaudhary, San Hlaing Myint, Qiaozhi Hua*

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

Abstract

In this paper, we present a method for generating bird's eye video from egocentric RGB videos. Working with egocentric views is tricky since such the view is highly warped and prone to occlusions. On the other hand, a bird's eye view has a consistent scaling in at least the two dimensions it shows. Moreover, most of the state-of-the-art systems for tasks such as path prediction are built for bird's eye views of the subjects. We present a deep learning-based approach that transfers the egocentric RGB images captured from a dashcam of a car to bird's eye view. This is a task of view translation, and we perform two experiments. The first one uses an image-to-image translation method, and the other uses a video-to-video translation. We compare the results of our work with homographic transformation, and our SSIM values are better by a margin of 77% and 14.4%, and the RMSE errors are lower by 40% and 14.6% for image-to-image translation and video-to-video translation, respectively. We also visually show the efficacy and limitations of each method with helpful insights for future research. Compared to previous works that use homography and LIDAR for 3D point clouds, our work is more generalizable and does not require any expensive equipment.

Original languageEnglish
Article number7479473
JournalWireless Communications and Mobile Computing
Volume2021
DOIs
Publication statusPublished - 2021

ASJC Scopus subject areas

  • Information Systems
  • Computer Networks and Communications
  • Electrical and Electronic Engineering

Fingerprint

Dive into the research topics of 'Generating Bird's Eye View from Egocentric RGB Videos'. Together they form a unique fingerprint.

Cite this