Condensed vector machines

Learning fast machine for large data

Dung Duc Nguyen, Kazunori Matsumoto, Yasuhiro Takishima, Kazuo Hashimoto

Research output: Contribution to journalArticle

11 Citations (Scopus)

Abstract

Scalability is one of the main challenges for kernel-based methods and support vector machines (SVMs). The quadratic demand in memory for storing kernel matrices makes it impossible for training on million-size data. Sophisticated decomposition algorithms have been proposed to efficiently train SVMs using only important examples, which ideally are the final support vectors (SVs). However, the ability of the decomposition method is limited to large-scale applications where the number of SVs is still too large for a computer's capacity. From another perspective, the large number of SVs slows down SVMs in the testing phase, making it impractical for many applications. In this paper, we introduce the integration of a vector combination scheme to simplify the SVM solution into an incremental working set selection for SVM training. The main objective of the integration is to maintain a minimal number of final SVs, bringing a minimum resource demand and faster training time. Consequently, the learning machines are more compact and run faster thanks to the small number of vectors included in their solution. Experimental results on large benchmark datasets shows that the proposed condensed SVMs achieve both training and testing efficiency while maintaining a generalization ability equivalent to that of normal SVMs.

Original languageEnglish
Article number5605254
Pages (from-to)1903-1914
Number of pages12
JournalIEEE Transactions on Neural Networks
Volume21
Issue number12
DOIs
Publication statusPublished - 2010 Dec
Externally publishedYes

Fingerprint

Support vector machines
Learning systems
Decomposition
Testing
Scalability
Data storage equipment

Keywords

  • Decomposition algorithm
  • Kernel method
  • optimization
  • reduced set method
  • support vector machine

ASJC Scopus subject areas

  • Artificial Intelligence
  • Computer Networks and Communications
  • Computer Science Applications
  • Software

Cite this

Condensed vector machines : Learning fast machine for large data. / Nguyen, Dung Duc; Matsumoto, Kazunori; Takishima, Yasuhiro; Hashimoto, Kazuo.

In: IEEE Transactions on Neural Networks, Vol. 21, No. 12, 5605254, 12.2010, p. 1903-1914.

Research output: Contribution to journalArticle

Nguyen, Dung Duc ; Matsumoto, Kazunori ; Takishima, Yasuhiro ; Hashimoto, Kazuo. / Condensed vector machines : Learning fast machine for large data. In: IEEE Transactions on Neural Networks. 2010 ; Vol. 21, No. 12. pp. 1903-1914.
@article{5793d4f1a76740938b08e34e6f312496,
title = "Condensed vector machines: Learning fast machine for large data",
abstract = "Scalability is one of the main challenges for kernel-based methods and support vector machines (SVMs). The quadratic demand in memory for storing kernel matrices makes it impossible for training on million-size data. Sophisticated decomposition algorithms have been proposed to efficiently train SVMs using only important examples, which ideally are the final support vectors (SVs). However, the ability of the decomposition method is limited to large-scale applications where the number of SVs is still too large for a computer's capacity. From another perspective, the large number of SVs slows down SVMs in the testing phase, making it impractical for many applications. In this paper, we introduce the integration of a vector combination scheme to simplify the SVM solution into an incremental working set selection for SVM training. The main objective of the integration is to maintain a minimal number of final SVs, bringing a minimum resource demand and faster training time. Consequently, the learning machines are more compact and run faster thanks to the small number of vectors included in their solution. Experimental results on large benchmark datasets shows that the proposed condensed SVMs achieve both training and testing efficiency while maintaining a generalization ability equivalent to that of normal SVMs.",
keywords = "Decomposition algorithm, Kernel method, optimization, reduced set method, support vector machine",
author = "Nguyen, {Dung Duc} and Kazunori Matsumoto and Yasuhiro Takishima and Kazuo Hashimoto",
year = "2010",
month = "12",
doi = "10.1109/TNN.2010.2079947",
language = "English",
volume = "21",
pages = "1903--1914",
journal = "IEEE Transactions on Neural Networks and Learning Systems",
issn = "2162-237X",
publisher = "IEEE Computational Intelligence Society",
number = "12",

}

TY - JOUR

T1 - Condensed vector machines

T2 - Learning fast machine for large data

AU - Nguyen, Dung Duc

AU - Matsumoto, Kazunori

AU - Takishima, Yasuhiro

AU - Hashimoto, Kazuo

PY - 2010/12

Y1 - 2010/12

N2 - Scalability is one of the main challenges for kernel-based methods and support vector machines (SVMs). The quadratic demand in memory for storing kernel matrices makes it impossible for training on million-size data. Sophisticated decomposition algorithms have been proposed to efficiently train SVMs using only important examples, which ideally are the final support vectors (SVs). However, the ability of the decomposition method is limited to large-scale applications where the number of SVs is still too large for a computer's capacity. From another perspective, the large number of SVs slows down SVMs in the testing phase, making it impractical for many applications. In this paper, we introduce the integration of a vector combination scheme to simplify the SVM solution into an incremental working set selection for SVM training. The main objective of the integration is to maintain a minimal number of final SVs, bringing a minimum resource demand and faster training time. Consequently, the learning machines are more compact and run faster thanks to the small number of vectors included in their solution. Experimental results on large benchmark datasets shows that the proposed condensed SVMs achieve both training and testing efficiency while maintaining a generalization ability equivalent to that of normal SVMs.

AB - Scalability is one of the main challenges for kernel-based methods and support vector machines (SVMs). The quadratic demand in memory for storing kernel matrices makes it impossible for training on million-size data. Sophisticated decomposition algorithms have been proposed to efficiently train SVMs using only important examples, which ideally are the final support vectors (SVs). However, the ability of the decomposition method is limited to large-scale applications where the number of SVs is still too large for a computer's capacity. From another perspective, the large number of SVs slows down SVMs in the testing phase, making it impractical for many applications. In this paper, we introduce the integration of a vector combination scheme to simplify the SVM solution into an incremental working set selection for SVM training. The main objective of the integration is to maintain a minimal number of final SVs, bringing a minimum resource demand and faster training time. Consequently, the learning machines are more compact and run faster thanks to the small number of vectors included in their solution. Experimental results on large benchmark datasets shows that the proposed condensed SVMs achieve both training and testing efficiency while maintaining a generalization ability equivalent to that of normal SVMs.

KW - Decomposition algorithm

KW - Kernel method

KW - optimization

KW - reduced set method

KW - support vector machine

UR - http://www.scopus.com/inward/record.url?scp=78650072699&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=78650072699&partnerID=8YFLogxK

U2 - 10.1109/TNN.2010.2079947

DO - 10.1109/TNN.2010.2079947

M3 - Article

VL - 21

SP - 1903

EP - 1914

JO - IEEE Transactions on Neural Networks and Learning Systems

JF - IEEE Transactions on Neural Networks and Learning Systems

SN - 2162-237X

IS - 12

M1 - 5605254

ER -