A deep neural network based quasi-linear kernel for support vector machines

Weite Li, Bo Zhou, Benhui Chen, Takayuki Furuzuki

Research output: Contribution to journalArticle

8 Citations (Scopus)

Abstract

This paper proposes a deep quasi-linear kernel for support vector machines (SVMs). The deep quasi-linear kernel can be constructed by using a pre-trained deep neural network. To realize this goal, a multilayer gated bilinear classifier is first designed to mimic the functionality of the pre-trained deep neural network, by generating the gate control signals using the deep neural network. Then, a deep quasi-linear kernel is derived by applying an SVM formulation to the multilayer gated bilinear classifier. In this way, we are able to further implicitly optimize the parameters of the multilayer gated bilinear classifier, which are a set of duplicate but independent parameters of the pre-trained deep neural network, by using an SVM optimization. Experimental results on different data sets show that SVMs with the proposed deep quasi-linear kernel have an ability to take advantage of the pre-trained deep neural networks and outperform SVMs with RBF kernels.

Original languageEnglish
Pages (from-to)2558-2565
Number of pages8
JournalIEICE Transactions on Fundamentals of Electronics, Communications and Computer Sciences
VolumeE99A
Issue number12
DOIs
Publication statusPublished - 2016 Dec 1

Fingerprint

Support vector machines
Support Vector Machine
Neural Networks
kernel
Multilayer
Multilayers
Classifiers
Classifier
Signal Control
Optimise
Deep neural networks
Optimization
Formulation
Experimental Results

Keywords

  • Data-dependent kernel
  • Deep neural network
  • Multilayer gated bilinear classifier
  • Support vector machine

ASJC Scopus subject areas

  • Signal Processing
  • Computer Graphics and Computer-Aided Design
  • Applied Mathematics
  • Electrical and Electronic Engineering

Cite this

A deep neural network based quasi-linear kernel for support vector machines. / Li, Weite; Zhou, Bo; Chen, Benhui; Furuzuki, Takayuki.

In: IEICE Transactions on Fundamentals of Electronics, Communications and Computer Sciences, Vol. E99A, No. 12, 01.12.2016, p. 2558-2565.

Research output: Contribution to journalArticle

@article{62974dc4f23144c19b83b586fdb9b1eb,
title = "A deep neural network based quasi-linear kernel for support vector machines",
abstract = "This paper proposes a deep quasi-linear kernel for support vector machines (SVMs). The deep quasi-linear kernel can be constructed by using a pre-trained deep neural network. To realize this goal, a multilayer gated bilinear classifier is first designed to mimic the functionality of the pre-trained deep neural network, by generating the gate control signals using the deep neural network. Then, a deep quasi-linear kernel is derived by applying an SVM formulation to the multilayer gated bilinear classifier. In this way, we are able to further implicitly optimize the parameters of the multilayer gated bilinear classifier, which are a set of duplicate but independent parameters of the pre-trained deep neural network, by using an SVM optimization. Experimental results on different data sets show that SVMs with the proposed deep quasi-linear kernel have an ability to take advantage of the pre-trained deep neural networks and outperform SVMs with RBF kernels.",
keywords = "Data-dependent kernel, Deep neural network, Multilayer gated bilinear classifier, Support vector machine",
author = "Weite Li and Bo Zhou and Benhui Chen and Takayuki Furuzuki",
year = "2016",
month = "12",
day = "1",
doi = "10.1587/transfun.E99.A.2558",
language = "English",
volume = "E99A",
pages = "2558--2565",
journal = "IEICE Transactions on Fundamentals of Electronics, Communications and Computer Sciences",
issn = "0916-8508",
publisher = "Maruzen Co., Ltd/Maruzen Kabushikikaisha",
number = "12",

}

TY - JOUR

T1 - A deep neural network based quasi-linear kernel for support vector machines

AU - Li, Weite

AU - Zhou, Bo

AU - Chen, Benhui

AU - Furuzuki, Takayuki

PY - 2016/12/1

Y1 - 2016/12/1

N2 - This paper proposes a deep quasi-linear kernel for support vector machines (SVMs). The deep quasi-linear kernel can be constructed by using a pre-trained deep neural network. To realize this goal, a multilayer gated bilinear classifier is first designed to mimic the functionality of the pre-trained deep neural network, by generating the gate control signals using the deep neural network. Then, a deep quasi-linear kernel is derived by applying an SVM formulation to the multilayer gated bilinear classifier. In this way, we are able to further implicitly optimize the parameters of the multilayer gated bilinear classifier, which are a set of duplicate but independent parameters of the pre-trained deep neural network, by using an SVM optimization. Experimental results on different data sets show that SVMs with the proposed deep quasi-linear kernel have an ability to take advantage of the pre-trained deep neural networks and outperform SVMs with RBF kernels.

AB - This paper proposes a deep quasi-linear kernel for support vector machines (SVMs). The deep quasi-linear kernel can be constructed by using a pre-trained deep neural network. To realize this goal, a multilayer gated bilinear classifier is first designed to mimic the functionality of the pre-trained deep neural network, by generating the gate control signals using the deep neural network. Then, a deep quasi-linear kernel is derived by applying an SVM formulation to the multilayer gated bilinear classifier. In this way, we are able to further implicitly optimize the parameters of the multilayer gated bilinear classifier, which are a set of duplicate but independent parameters of the pre-trained deep neural network, by using an SVM optimization. Experimental results on different data sets show that SVMs with the proposed deep quasi-linear kernel have an ability to take advantage of the pre-trained deep neural networks and outperform SVMs with RBF kernels.

KW - Data-dependent kernel

KW - Deep neural network

KW - Multilayer gated bilinear classifier

KW - Support vector machine

UR - http://www.scopus.com/inward/record.url?scp=84999133363&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84999133363&partnerID=8YFLogxK

U2 - 10.1587/transfun.E99.A.2558

DO - 10.1587/transfun.E99.A.2558

M3 - Article

AN - SCOPUS:84999133363

VL - E99A

SP - 2558

EP - 2565

JO - IEICE Transactions on Fundamentals of Electronics, Communications and Computer Sciences

JF - IEICE Transactions on Fundamentals of Electronics, Communications and Computer Sciences

SN - 0916-8508

IS - 12

ER -