An autoencoder-based piecewise linear model for nonlinear classification using quasilinear support vector machines

Weite Li, Peifeng Liang, Takayuki Furuzuki

Research output: Contribution to journalArticle

Abstract

In this paper, we propose to implement a piecewise linear model to solve nonlinear classification problems. In order to realize a switch between linear models, a data-dependent gating mechanism achieved by an autoencoder is designed to assign gate signals automatically. We ensure that a diversity of gate signals is available so that it is possible for our model to switch between a large number of linear classifiers. Besides, we also introduce a sparsity level to add a manual control on the flexibility of the proposed model by using a winner-take-all strategy. Therefore, our model can maintain a balance between underfitting and overfitting problems. Then, given a learned gating mechanism, the proposed model is shown to be equivalent to a kernel machine by deriving a quasilinear kernel function with the gating mechanism included. Therefore, a quasilinear support vector machine can be applied to solve the nonlinear classification problems. Experimental results demonstrate that our proposed piecewise linear model performs better than or is at least competitive with its state-of-the-art counterparts.

Original languageEnglish
Pages (from-to)1236-1243
Number of pages8
JournalIEEJ Transactions on Electrical and Electronic Engineering
Volume14
Issue number8
DOIs
Publication statusPublished - 2019 Aug 1

Fingerprint

Support vector machines
Switches
Manual control
Classifiers

Keywords

  • autoencoder
  • kernel machine
  • nonlinear classification
  • support vector machine

ASJC Scopus subject areas

  • Electrical and Electronic Engineering

Cite this

An autoencoder-based piecewise linear model for nonlinear classification using quasilinear support vector machines. / Li, Weite; Liang, Peifeng; Furuzuki, Takayuki.

In: IEEJ Transactions on Electrical and Electronic Engineering, Vol. 14, No. 8, 01.08.2019, p. 1236-1243.

Research output: Contribution to journalArticle

@article{4d71ecd8657440599b1251ebbe5acd66,
title = "An autoencoder-based piecewise linear model for nonlinear classification using quasilinear support vector machines",
abstract = "In this paper, we propose to implement a piecewise linear model to solve nonlinear classification problems. In order to realize a switch between linear models, a data-dependent gating mechanism achieved by an autoencoder is designed to assign gate signals automatically. We ensure that a diversity of gate signals is available so that it is possible for our model to switch between a large number of linear classifiers. Besides, we also introduce a sparsity level to add a manual control on the flexibility of the proposed model by using a winner-take-all strategy. Therefore, our model can maintain a balance between underfitting and overfitting problems. Then, given a learned gating mechanism, the proposed model is shown to be equivalent to a kernel machine by deriving a quasilinear kernel function with the gating mechanism included. Therefore, a quasilinear support vector machine can be applied to solve the nonlinear classification problems. Experimental results demonstrate that our proposed piecewise linear model performs better than or is at least competitive with its state-of-the-art counterparts.",
keywords = "autoencoder, kernel machine, nonlinear classification, support vector machine",
author = "Weite Li and Peifeng Liang and Takayuki Furuzuki",
year = "2019",
month = "8",
day = "1",
doi = "10.1002/tee.22923",
language = "English",
volume = "14",
pages = "1236--1243",
journal = "IEEJ Transactions on Electrical and Electronic Engineering",
issn = "1931-4973",
publisher = "John Wiley and Sons Inc.",
number = "8",

}

TY - JOUR

T1 - An autoencoder-based piecewise linear model for nonlinear classification using quasilinear support vector machines

AU - Li, Weite

AU - Liang, Peifeng

AU - Furuzuki, Takayuki

PY - 2019/8/1

Y1 - 2019/8/1

N2 - In this paper, we propose to implement a piecewise linear model to solve nonlinear classification problems. In order to realize a switch between linear models, a data-dependent gating mechanism achieved by an autoencoder is designed to assign gate signals automatically. We ensure that a diversity of gate signals is available so that it is possible for our model to switch between a large number of linear classifiers. Besides, we also introduce a sparsity level to add a manual control on the flexibility of the proposed model by using a winner-take-all strategy. Therefore, our model can maintain a balance between underfitting and overfitting problems. Then, given a learned gating mechanism, the proposed model is shown to be equivalent to a kernel machine by deriving a quasilinear kernel function with the gating mechanism included. Therefore, a quasilinear support vector machine can be applied to solve the nonlinear classification problems. Experimental results demonstrate that our proposed piecewise linear model performs better than or is at least competitive with its state-of-the-art counterparts.

AB - In this paper, we propose to implement a piecewise linear model to solve nonlinear classification problems. In order to realize a switch between linear models, a data-dependent gating mechanism achieved by an autoencoder is designed to assign gate signals automatically. We ensure that a diversity of gate signals is available so that it is possible for our model to switch between a large number of linear classifiers. Besides, we also introduce a sparsity level to add a manual control on the flexibility of the proposed model by using a winner-take-all strategy. Therefore, our model can maintain a balance between underfitting and overfitting problems. Then, given a learned gating mechanism, the proposed model is shown to be equivalent to a kernel machine by deriving a quasilinear kernel function with the gating mechanism included. Therefore, a quasilinear support vector machine can be applied to solve the nonlinear classification problems. Experimental results demonstrate that our proposed piecewise linear model performs better than or is at least competitive with its state-of-the-art counterparts.

KW - autoencoder

KW - kernel machine

KW - nonlinear classification

KW - support vector machine

UR - http://www.scopus.com/inward/record.url?scp=85069532127&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85069532127&partnerID=8YFLogxK

U2 - 10.1002/tee.22923

DO - 10.1002/tee.22923

M3 - Article

VL - 14

SP - 1236

EP - 1243

JO - IEEJ Transactions on Electrical and Electronic Engineering

JF - IEEJ Transactions on Electrical and Electronic Engineering

SN - 1931-4973

IS - 8

ER -