360 Depth Estimation in the Wild - The Depth360 Dataset and the SegFuse Network

Qi Feng, Hubert P.H. Shum, Shigeo Morishima

研究成果: Conference contribution

抄録

Single-view depth estimation from omnidirectional images has gained popularity with its wide range of applications such as autonomous driving and scene reconstruction. Although data-driven learning-based methods demonstrate significant potential in this field, scarce training data and ineffective 360 estimation algorithms are still two key limitations hindering accurate estimation across diverse domains. In this work, we first establish a large-scale dataset with varied settings called Depth360 to tackle the training data problem. This is achieved by exploring the use of a plenteous source of data, 360 videos from the internet, using a test-time training method that leverages unique information in each omnidirectional sequence. With novel geometric and temporal constraints, our method generates consistent and convincing depth samples to facilitate single-view estimation. We then propose an end-to-end two-branch multi-task learning network, SegFuse, that mimics the human eye to effectively learn from the dataset and estimate high-quality depth maps from diverse monocular RGB images. With a peripheral branch that uses equirectangular projection for depth estimation and a foveal branch that uses cubemap projection for semantic segmentation, our method predicts consistent global depth while maintaining sharp details at local regions. Experimental results show favorable performance against the state-of-the-art methods.

本文言語English
ホスト出版物のタイトルProceedings - 2022 IEEE Conference on Virtual Reality and 3D User Interfaces, VR 2022
出版社Institute of Electrical and Electronics Engineers Inc.
ページ664-673
ページ数10
ISBN(電子版)9781665496179
DOI
出版ステータスPublished - 2022
イベント29th IEEE Conference on Virtual Reality and 3D User Interfaces, VR 2022 - Virtual, Online, New Zealand
継続期間: 2022 3月 122022 3月 16

出版物シリーズ

名前Proceedings - 2022 IEEE Conference on Virtual Reality and 3D User Interfaces, VR 2022

Conference

Conference29th IEEE Conference on Virtual Reality and 3D User Interfaces, VR 2022
国/地域New Zealand
CityVirtual, Online
Period22/3/1222/3/16

ASJC Scopus subject areas

  • コンピュータ グラフィックスおよびコンピュータ支援設計
  • 人間とコンピュータの相互作用
  • メディア記述
  • モデリングとシミュレーション

フィンガープリント

「360 Depth Estimation in the Wild - The Depth360 Dataset and the SegFuse Network」の研究トピックを掘り下げます。これらがまとまってユニークなフィンガープリントを構成します。

引用スタイル