Deep neural networks with mixture of experts layers for complex event recognition from images

Mingyao Li, Seiichiro Kamata

研究成果: Conference contribution

抄録

With the need for the real-world applications, event recognition from static images has become more and more popular in these years. Although there remain good achievements, recognizing events from images with a complex background like WIDER dataset is still very hard to get good results. In this paper, we show this gap is probably caused by the large discrepancy of data. Most of the existing methods choose to use various modifications on pre-trained CNN network model to solve the problem. Although we follow this thought, after a review of existing methods, we choose two other ways to solve this problem. Firstly, we reveal that a deep one-channel model with end-to-end structure is more suitable to this problem than other multi-channel or multi-task models, which leads we to propose a model under this rule by modifying on one single pre-trained ResNet channel. Secondly, we propose a Mixture of Experts (MoE) neural network layer to overcome the large discrepancy of data. To increase the performance and enhance the specialization of the MoE layer, we also involve a simple neural network transfer method, Elastic Weight Consolidation, to transfer knowledge from SocEID dataset. The result shows that we enhance the accuracy of the WIDER dataset from the state-of-the-art by 9.4% with lower computational time and memory consumption. And some experiments are also listed there to proof the validation of our method.

元の言語English
ホスト出版物のタイトル2018 Joint 7th International Conference on Informatics, Electronics and Vision and 2nd International Conference on Imaging, Vision and Pattern Recognition, ICIEV-IVPR 2018
出版者Institute of Electrical and Electronics Engineers Inc.
ページ410-415
ページ数6
ISBN(電子版)9781538651612
DOI
出版物ステータスPublished - 2019 2 12
イベントJoint 7th International Conference on Informatics, Electronics and Vision and 2nd International Conference on Imaging, Vision and Pattern Recognition, ICIEV-IVPR 2018 - Kitakyushu, Japan
継続期間: 2018 6 252018 6 28

出版物シリーズ

名前2018 Joint 7th International Conference on Informatics, Electronics and Vision and 2nd International Conference on Imaging, Vision and Pattern Recognition, ICIEV-IVPR 2018

Conference

ConferenceJoint 7th International Conference on Informatics, Electronics and Vision and 2nd International Conference on Imaging, Vision and Pattern Recognition, ICIEV-IVPR 2018
Japan
Kitakyushu
期間18/6/2518/6/28

Fingerprint

Mixture of Experts
Neural Networks
Discrepancy
Choose
Neural networks
Task Model
Knowledge Transfer
Network layers
Channel Model
Consolidation
Specialization
Real-world Applications
Network Model
Data storage equipment
Deep neural networks
Experiment
Experiments

ASJC Scopus subject areas

  • Signal Processing
  • Control and Optimization
  • Artificial Intelligence
  • Computer Vision and Pattern Recognition
  • Information Systems

これを引用

Li, M., & Kamata, S. (2019). Deep neural networks with mixture of experts layers for complex event recognition from images. : 2018 Joint 7th International Conference on Informatics, Electronics and Vision and 2nd International Conference on Imaging, Vision and Pattern Recognition, ICIEV-IVPR 2018 (pp. 410-415). [8641027] (2018 Joint 7th International Conference on Informatics, Electronics and Vision and 2nd International Conference on Imaging, Vision and Pattern Recognition, ICIEV-IVPR 2018). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/ICIEV.2018.8641027

Deep neural networks with mixture of experts layers for complex event recognition from images. / Li, Mingyao; Kamata, Seiichiro.

2018 Joint 7th International Conference on Informatics, Electronics and Vision and 2nd International Conference on Imaging, Vision and Pattern Recognition, ICIEV-IVPR 2018. Institute of Electrical and Electronics Engineers Inc., 2019. p. 410-415 8641027 (2018 Joint 7th International Conference on Informatics, Electronics and Vision and 2nd International Conference on Imaging, Vision and Pattern Recognition, ICIEV-IVPR 2018).

研究成果: Conference contribution

Li, M & Kamata, S 2019, Deep neural networks with mixture of experts layers for complex event recognition from images. : 2018 Joint 7th International Conference on Informatics, Electronics and Vision and 2nd International Conference on Imaging, Vision and Pattern Recognition, ICIEV-IVPR 2018., 8641027, 2018 Joint 7th International Conference on Informatics, Electronics and Vision and 2nd International Conference on Imaging, Vision and Pattern Recognition, ICIEV-IVPR 2018, Institute of Electrical and Electronics Engineers Inc., pp. 410-415, Joint 7th International Conference on Informatics, Electronics and Vision and 2nd International Conference on Imaging, Vision and Pattern Recognition, ICIEV-IVPR 2018, Kitakyushu, Japan, 18/6/25. https://doi.org/10.1109/ICIEV.2018.8641027
Li M, Kamata S. Deep neural networks with mixture of experts layers for complex event recognition from images. : 2018 Joint 7th International Conference on Informatics, Electronics and Vision and 2nd International Conference on Imaging, Vision and Pattern Recognition, ICIEV-IVPR 2018. Institute of Electrical and Electronics Engineers Inc. 2019. p. 410-415. 8641027. (2018 Joint 7th International Conference on Informatics, Electronics and Vision and 2nd International Conference on Imaging, Vision and Pattern Recognition, ICIEV-IVPR 2018). https://doi.org/10.1109/ICIEV.2018.8641027
Li, Mingyao ; Kamata, Seiichiro. / Deep neural networks with mixture of experts layers for complex event recognition from images. 2018 Joint 7th International Conference on Informatics, Electronics and Vision and 2nd International Conference on Imaging, Vision and Pattern Recognition, ICIEV-IVPR 2018. Institute of Electrical and Electronics Engineers Inc., 2019. pp. 410-415 (2018 Joint 7th International Conference on Informatics, Electronics and Vision and 2nd International Conference on Imaging, Vision and Pattern Recognition, ICIEV-IVPR 2018).
@inproceedings{9b5ab6ba92dc4b56a2c76d13323288f5,
title = "Deep neural networks with mixture of experts layers for complex event recognition from images",
abstract = "With the need for the real-world applications, event recognition from static images has become more and more popular in these years. Although there remain good achievements, recognizing events from images with a complex background like WIDER dataset is still very hard to get good results. In this paper, we show this gap is probably caused by the large discrepancy of data. Most of the existing methods choose to use various modifications on pre-trained CNN network model to solve the problem. Although we follow this thought, after a review of existing methods, we choose two other ways to solve this problem. Firstly, we reveal that a deep one-channel model with end-to-end structure is more suitable to this problem than other multi-channel or multi-task models, which leads we to propose a model under this rule by modifying on one single pre-trained ResNet channel. Secondly, we propose a Mixture of Experts (MoE) neural network layer to overcome the large discrepancy of data. To increase the performance and enhance the specialization of the MoE layer, we also involve a simple neural network transfer method, Elastic Weight Consolidation, to transfer knowledge from SocEID dataset. The result shows that we enhance the accuracy of the WIDER dataset from the state-of-the-art by 9.4{\%} with lower computational time and memory consumption. And some experiments are also listed there to proof the validation of our method.",
keywords = "CNN, Complex Event Recognition, Deep Learning, Elastic Weight Consolidation, Event Recognition, Mixture of Experts",
author = "Mingyao Li and Seiichiro Kamata",
year = "2019",
month = "2",
day = "12",
doi = "10.1109/ICIEV.2018.8641027",
language = "English",
series = "2018 Joint 7th International Conference on Informatics, Electronics and Vision and 2nd International Conference on Imaging, Vision and Pattern Recognition, ICIEV-IVPR 2018",
publisher = "Institute of Electrical and Electronics Engineers Inc.",
pages = "410--415",
booktitle = "2018 Joint 7th International Conference on Informatics, Electronics and Vision and 2nd International Conference on Imaging, Vision and Pattern Recognition, ICIEV-IVPR 2018",

}

TY - GEN

T1 - Deep neural networks with mixture of experts layers for complex event recognition from images

AU - Li, Mingyao

AU - Kamata, Seiichiro

PY - 2019/2/12

Y1 - 2019/2/12

N2 - With the need for the real-world applications, event recognition from static images has become more and more popular in these years. Although there remain good achievements, recognizing events from images with a complex background like WIDER dataset is still very hard to get good results. In this paper, we show this gap is probably caused by the large discrepancy of data. Most of the existing methods choose to use various modifications on pre-trained CNN network model to solve the problem. Although we follow this thought, after a review of existing methods, we choose two other ways to solve this problem. Firstly, we reveal that a deep one-channel model with end-to-end structure is more suitable to this problem than other multi-channel or multi-task models, which leads we to propose a model under this rule by modifying on one single pre-trained ResNet channel. Secondly, we propose a Mixture of Experts (MoE) neural network layer to overcome the large discrepancy of data. To increase the performance and enhance the specialization of the MoE layer, we also involve a simple neural network transfer method, Elastic Weight Consolidation, to transfer knowledge from SocEID dataset. The result shows that we enhance the accuracy of the WIDER dataset from the state-of-the-art by 9.4% with lower computational time and memory consumption. And some experiments are also listed there to proof the validation of our method.

AB - With the need for the real-world applications, event recognition from static images has become more and more popular in these years. Although there remain good achievements, recognizing events from images with a complex background like WIDER dataset is still very hard to get good results. In this paper, we show this gap is probably caused by the large discrepancy of data. Most of the existing methods choose to use various modifications on pre-trained CNN network model to solve the problem. Although we follow this thought, after a review of existing methods, we choose two other ways to solve this problem. Firstly, we reveal that a deep one-channel model with end-to-end structure is more suitable to this problem than other multi-channel or multi-task models, which leads we to propose a model under this rule by modifying on one single pre-trained ResNet channel. Secondly, we propose a Mixture of Experts (MoE) neural network layer to overcome the large discrepancy of data. To increase the performance and enhance the specialization of the MoE layer, we also involve a simple neural network transfer method, Elastic Weight Consolidation, to transfer knowledge from SocEID dataset. The result shows that we enhance the accuracy of the WIDER dataset from the state-of-the-art by 9.4% with lower computational time and memory consumption. And some experiments are also listed there to proof the validation of our method.

KW - CNN

KW - Complex Event Recognition

KW - Deep Learning

KW - Elastic Weight Consolidation

KW - Event Recognition

KW - Mixture of Experts

UR - http://www.scopus.com/inward/record.url?scp=85063209687&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85063209687&partnerID=8YFLogxK

U2 - 10.1109/ICIEV.2018.8641027

DO - 10.1109/ICIEV.2018.8641027

M3 - Conference contribution

T3 - 2018 Joint 7th International Conference on Informatics, Electronics and Vision and 2nd International Conference on Imaging, Vision and Pattern Recognition, ICIEV-IVPR 2018

SP - 410

EP - 415

BT - 2018 Joint 7th International Conference on Informatics, Electronics and Vision and 2nd International Conference on Imaging, Vision and Pattern Recognition, ICIEV-IVPR 2018

PB - Institute of Electrical and Electronics Engineers Inc.

ER -