Deep neural networks with mixture of experts layers for complex event recognition from images

Mingyao Li, Seiichiro Kamata

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

With the need for the real-world applications, event recognition from static images has become more and more popular in these years. Although there remain good achievements, recognizing events from images with a complex background like WIDER dataset is still very hard to get good results. In this paper, we show this gap is probably caused by the large discrepancy of data. Most of the existing methods choose to use various modifications on pre-trained CNN network model to solve the problem. Although we follow this thought, after a review of existing methods, we choose two other ways to solve this problem. Firstly, we reveal that a deep one-channel model with end-to-end structure is more suitable to this problem than other multi-channel or multi-task models, which leads we to propose a model under this rule by modifying on one single pre-trained ResNet channel. Secondly, we propose a Mixture of Experts (MoE) neural network layer to overcome the large discrepancy of data. To increase the performance and enhance the specialization of the MoE layer, we also involve a simple neural network transfer method, Elastic Weight Consolidation, to transfer knowledge from SocEID dataset. The result shows that we enhance the accuracy of the WIDER dataset from the state-of-the-art by 9.4% with lower computational time and memory consumption. And some experiments are also listed there to proof the validation of our method.

Original languageEnglish
Title of host publication2018 Joint 7th International Conference on Informatics, Electronics and Vision and 2nd International Conference on Imaging, Vision and Pattern Recognition, ICIEV-IVPR 2018
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages410-415
Number of pages6
ISBN (Electronic)9781538651612
DOIs
Publication statusPublished - 2019 Feb 12
EventJoint 7th International Conference on Informatics, Electronics and Vision and 2nd International Conference on Imaging, Vision and Pattern Recognition, ICIEV-IVPR 2018 - Kitakyushu, Japan
Duration: 2018 Jun 252018 Jun 28

Publication series

Name2018 Joint 7th International Conference on Informatics, Electronics and Vision and 2nd International Conference on Imaging, Vision and Pattern Recognition, ICIEV-IVPR 2018

Conference

ConferenceJoint 7th International Conference on Informatics, Electronics and Vision and 2nd International Conference on Imaging, Vision and Pattern Recognition, ICIEV-IVPR 2018
CountryJapan
CityKitakyushu
Period18/6/2518/6/28

Keywords

  • CNN
  • Complex Event Recognition
  • Deep Learning
  • Elastic Weight Consolidation
  • Event Recognition
  • Mixture of Experts

ASJC Scopus subject areas

  • Signal Processing
  • Control and Optimization
  • Artificial Intelligence
  • Computer Vision and Pattern Recognition
  • Information Systems

Fingerprint Dive into the research topics of 'Deep neural networks with mixture of experts layers for complex event recognition from images'. Together they form a unique fingerprint.

  • Cite this

    Li, M., & Kamata, S. (2019). Deep neural networks with mixture of experts layers for complex event recognition from images. In 2018 Joint 7th International Conference on Informatics, Electronics and Vision and 2nd International Conference on Imaging, Vision and Pattern Recognition, ICIEV-IVPR 2018 (pp. 410-415). [8641027] (2018 Joint 7th International Conference on Informatics, Electronics and Vision and 2nd International Conference on Imaging, Vision and Pattern Recognition, ICIEV-IVPR 2018). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/ICIEV.2018.8641027