A Metric Learning Method for Improving Neural Network Based Kernel Learning for SVM

Peifeng Liang, Xueqin Yao, Takayuki Furuzuki

研究成果: Conference contribution

抄録

A gated linear network is able to mimic the functionality of a pre-trained neural network with a compound activation function R(x) = x ∗ S(x). An SVM can then be formulated to further implicitly optimize the gated linear network, in which a quasi-linear kernel is composed by using the gate signal S(x) generated from the pre-trained neural network. In this way, we realize a neural network based kernel learning. In this paper, a distance metric learning is applied to improving the kernel learning. In the pre-training of neural network, the loss function of distance metric learning is used as a regularization term. With the loss function of distance metric learning, the samples from within-class become closer and that from between-class become farther, which can improve the quasi-linear kernel. Accordingly, the classifier optimized by SVM with quasi-linear kernel will have better performance. The proposed classification method is applied to different real-world datasets and simulation results confirm the effectiveness of the proposed method.

元の言語English
ホスト出版物のタイトルProceedings - 2018 IEEE International Conference on Systems, Man, and Cybernetics, SMC 2018
出版者Institute of Electrical and Electronics Engineers Inc.
ページ1641-1646
ページ数6
ISBN(電子版)9781538666500
DOI
出版物ステータスPublished - 2019 1 16
イベント2018 IEEE International Conference on Systems, Man, and Cybernetics, SMC 2018 - Miyazaki, Japan
継続期間: 2018 10 72018 10 10

出版物シリーズ

名前Proceedings - 2018 IEEE International Conference on Systems, Man, and Cybernetics, SMC 2018

Conference

Conference2018 IEEE International Conference on Systems, Man, and Cybernetics, SMC 2018
Japan
Miyazaki
期間18/10/718/10/10

Fingerprint

Distance Education
Learning
Neural networks
Linear networks
Classifiers
Chemical activation
Kernel
Learning methods

ASJC Scopus subject areas

  • Information Systems
  • Information Systems and Management
  • Health Informatics
  • Artificial Intelligence
  • Computer Networks and Communications
  • Human-Computer Interaction

これを引用

Liang, P., Yao, X., & Furuzuki, T. (2019). A Metric Learning Method for Improving Neural Network Based Kernel Learning for SVM. : Proceedings - 2018 IEEE International Conference on Systems, Man, and Cybernetics, SMC 2018 (pp. 1641-1646). [8616280] (Proceedings - 2018 IEEE International Conference on Systems, Man, and Cybernetics, SMC 2018). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/SMC.2018.00284

A Metric Learning Method for Improving Neural Network Based Kernel Learning for SVM. / Liang, Peifeng; Yao, Xueqin; Furuzuki, Takayuki.

Proceedings - 2018 IEEE International Conference on Systems, Man, and Cybernetics, SMC 2018. Institute of Electrical and Electronics Engineers Inc., 2019. p. 1641-1646 8616280 (Proceedings - 2018 IEEE International Conference on Systems, Man, and Cybernetics, SMC 2018).

研究成果: Conference contribution

Liang, P, Yao, X & Furuzuki, T 2019, A Metric Learning Method for Improving Neural Network Based Kernel Learning for SVM. : Proceedings - 2018 IEEE International Conference on Systems, Man, and Cybernetics, SMC 2018., 8616280, Proceedings - 2018 IEEE International Conference on Systems, Man, and Cybernetics, SMC 2018, Institute of Electrical and Electronics Engineers Inc., pp. 1641-1646, 2018 IEEE International Conference on Systems, Man, and Cybernetics, SMC 2018, Miyazaki, Japan, 18/10/7. https://doi.org/10.1109/SMC.2018.00284
Liang P, Yao X, Furuzuki T. A Metric Learning Method for Improving Neural Network Based Kernel Learning for SVM. : Proceedings - 2018 IEEE International Conference on Systems, Man, and Cybernetics, SMC 2018. Institute of Electrical and Electronics Engineers Inc. 2019. p. 1641-1646. 8616280. (Proceedings - 2018 IEEE International Conference on Systems, Man, and Cybernetics, SMC 2018). https://doi.org/10.1109/SMC.2018.00284
Liang, Peifeng ; Yao, Xueqin ; Furuzuki, Takayuki. / A Metric Learning Method for Improving Neural Network Based Kernel Learning for SVM. Proceedings - 2018 IEEE International Conference on Systems, Man, and Cybernetics, SMC 2018. Institute of Electrical and Electronics Engineers Inc., 2019. pp. 1641-1646 (Proceedings - 2018 IEEE International Conference on Systems, Man, and Cybernetics, SMC 2018).
@inproceedings{73c9f9bda5a54a23b354179cf134d8f6,
title = "A Metric Learning Method for Improving Neural Network Based Kernel Learning for SVM",
abstract = "A gated linear network is able to mimic the functionality of a pre-trained neural network with a compound activation function R(x) = x ∗ S(x). An SVM can then be formulated to further implicitly optimize the gated linear network, in which a quasi-linear kernel is composed by using the gate signal S(x) generated from the pre-trained neural network. In this way, we realize a neural network based kernel learning. In this paper, a distance metric learning is applied to improving the kernel learning. In the pre-training of neural network, the loss function of distance metric learning is used as a regularization term. With the loss function of distance metric learning, the samples from within-class become closer and that from between-class become farther, which can improve the quasi-linear kernel. Accordingly, the classifier optimized by SVM with quasi-linear kernel will have better performance. The proposed classification method is applied to different real-world datasets and simulation results confirm the effectiveness of the proposed method.",
author = "Peifeng Liang and Xueqin Yao and Takayuki Furuzuki",
year = "2019",
month = "1",
day = "16",
doi = "10.1109/SMC.2018.00284",
language = "English",
series = "Proceedings - 2018 IEEE International Conference on Systems, Man, and Cybernetics, SMC 2018",
publisher = "Institute of Electrical and Electronics Engineers Inc.",
pages = "1641--1646",
booktitle = "Proceedings - 2018 IEEE International Conference on Systems, Man, and Cybernetics, SMC 2018",

}

TY - GEN

T1 - A Metric Learning Method for Improving Neural Network Based Kernel Learning for SVM

AU - Liang, Peifeng

AU - Yao, Xueqin

AU - Furuzuki, Takayuki

PY - 2019/1/16

Y1 - 2019/1/16

N2 - A gated linear network is able to mimic the functionality of a pre-trained neural network with a compound activation function R(x) = x ∗ S(x). An SVM can then be formulated to further implicitly optimize the gated linear network, in which a quasi-linear kernel is composed by using the gate signal S(x) generated from the pre-trained neural network. In this way, we realize a neural network based kernel learning. In this paper, a distance metric learning is applied to improving the kernel learning. In the pre-training of neural network, the loss function of distance metric learning is used as a regularization term. With the loss function of distance metric learning, the samples from within-class become closer and that from between-class become farther, which can improve the quasi-linear kernel. Accordingly, the classifier optimized by SVM with quasi-linear kernel will have better performance. The proposed classification method is applied to different real-world datasets and simulation results confirm the effectiveness of the proposed method.

AB - A gated linear network is able to mimic the functionality of a pre-trained neural network with a compound activation function R(x) = x ∗ S(x). An SVM can then be formulated to further implicitly optimize the gated linear network, in which a quasi-linear kernel is composed by using the gate signal S(x) generated from the pre-trained neural network. In this way, we realize a neural network based kernel learning. In this paper, a distance metric learning is applied to improving the kernel learning. In the pre-training of neural network, the loss function of distance metric learning is used as a regularization term. With the loss function of distance metric learning, the samples from within-class become closer and that from between-class become farther, which can improve the quasi-linear kernel. Accordingly, the classifier optimized by SVM with quasi-linear kernel will have better performance. The proposed classification method is applied to different real-world datasets and simulation results confirm the effectiveness of the proposed method.

UR - http://www.scopus.com/inward/record.url?scp=85062233504&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85062233504&partnerID=8YFLogxK

U2 - 10.1109/SMC.2018.00284

DO - 10.1109/SMC.2018.00284

M3 - Conference contribution

T3 - Proceedings - 2018 IEEE International Conference on Systems, Man, and Cybernetics, SMC 2018

SP - 1641

EP - 1646

BT - Proceedings - 2018 IEEE International Conference on Systems, Man, and Cybernetics, SMC 2018

PB - Institute of Electrical and Electronics Engineers Inc.

ER -