Condensed vector machines: Learning fast machine for large data

Dung Duc Nguyen, Kazunori Matsumoto, Yasuhiro Takishima, Kazuo Hashimoto

Research output: Contribution to journalArticle

13 Citations (Scopus)

Abstract

Scalability is one of the main challenges for kernel-based methods and support vector machines (SVMs). The quadratic demand in memory for storing kernel matrices makes it impossible for training on million-size data. Sophisticated decomposition algorithms have been proposed to efficiently train SVMs using only important examples, which ideally are the final support vectors (SVs). However, the ability of the decomposition method is limited to large-scale applications where the number of SVs is still too large for a computer's capacity. From another perspective, the large number of SVs slows down SVMs in the testing phase, making it impractical for many applications. In this paper, we introduce the integration of a vector combination scheme to simplify the SVM solution into an incremental working set selection for SVM training. The main objective of the integration is to maintain a minimal number of final SVs, bringing a minimum resource demand and faster training time. Consequently, the learning machines are more compact and run faster thanks to the small number of vectors included in their solution. Experimental results on large benchmark datasets shows that the proposed condensed SVMs achieve both training and testing efficiency while maintaining a generalization ability equivalent to that of normal SVMs.

Original languageEnglish
Article number5605254
Pages (from-to)1903-1914
Number of pages12
JournalIEEE Transactions on Neural Networks
Volume21
Issue number12
DOIs
Publication statusPublished - 2010 Dec 1

Keywords

  • Decomposition algorithm
  • Kernel method
  • optimization
  • reduced set method
  • support vector machine

ASJC Scopus subject areas

  • Software
  • Computer Science Applications
  • Computer Networks and Communications
  • Artificial Intelligence

Fingerprint Dive into the research topics of 'Condensed vector machines: Learning fast machine for large data'. Together they form a unique fingerprint.

  • Cite this