Bi-directional attention flow for video alignment

Reham Abobeah, Marwan Torki, Amin Shoukry, Jiro Katto

Research output: Chapter in Book/Report/Conference proceedingConference contribution

1 Citation (Scopus)

Abstract

In this paper, a novel technique is introduced to address the video alignment task which is one of the hot topics in computer vision. Specifically, we aim at finding the best possible correspondences between two overlapping videos without the restrictions imposed by previous techniques. The novelty of this work is that the video alignment problem is solved by drawing an analogy between it and the machine comprehension (MC) task in natural language processing (NLP). Simply, MC seeks to give the best answer to a question about a given paragraph. In our work, one of the two videos is considered as a query, while the other as a context. First, a pre-trained CNN is used to obtain high-level features from the frames of both the query and context videos. Then, the bidirectional attention flow mechanism; that has achieved considerable success in MC; is used to compute the query-context interactions in order to find the best mapping between the two input videos. The proposed model has been trained using 10k of collected video pairs from”YouTube”. The initial experimental results show that it is a promising solution for the video alignment task when compared to the state of the art techniques.

Original languageEnglish
Title of host publicationVISIGRAPP 2019 - Proceedings of the 14th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications
EditorsAndreas Kerren, Christophe Hurter, Jose Braz
PublisherSciTePress
Pages583-589
Number of pages7
ISBN (Electronic)9789897583544
Publication statusPublished - 2019 Jan 1
Event14th International Conference on Computer Vision Theory and Applications, VISAPP 2019 - Part of the 14th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications, VISIGRAPP 2019 - Prague, Czech Republic
Duration: 2019 Feb 252019 Feb 27

Publication series

NameVISIGRAPP 2019 - Proceedings of the 14th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications
Volume5

Conference

Conference14th International Conference on Computer Vision Theory and Applications, VISAPP 2019 - Part of the 14th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications, VISIGRAPP 2019
CountryCzech Republic
CityPrague
Period19/2/2519/2/27

Keywords

  • Attention Mechanisms
  • Bi-directional Attention
  • Synchronization
  • Temporal Alignment

ASJC Scopus subject areas

  • Computer Science Applications
  • Computer Vision and Pattern Recognition
  • Computer Graphics and Computer-Aided Design

Fingerprint Dive into the research topics of 'Bi-directional attention flow for video alignment'. Together they form a unique fingerprint.

  • Cite this

    Abobeah, R., Torki, M., Shoukry, A., & Katto, J. (2019). Bi-directional attention flow for video alignment. In A. Kerren, C. Hurter, & J. Braz (Eds.), VISIGRAPP 2019 - Proceedings of the 14th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (pp. 583-589). (VISIGRAPP 2019 - Proceedings of the 14th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications; Vol. 5). SciTePress.