Two-stage incremental working set selection for fast support vector training on large datasets

Duc Dung Nguyen, Kazunori Matsumoto, Yasuhiro Takishima, Kazuo Hashimoto, Masahiro Terabe

Research output: Chapter in Book/Report/Conference proceedingConference contribution

3 Citations (Scopus)

Abstract

We propose iSVM - an incremental algorithm that achieves high speed in training support vector machines (SVMs) on large datasets. In the common decomposition framework, iSVM starts with a minimum working set (WS), and then iteratively selects one training example to update the WS in each optimization loop. iSVM employs a two-stage strategy in processing the training data. In the first stage, the most prominent vector among randomly sampled data is added to the WS. This stage results in an approximate SVM solution. The second stage uses temporal solutions to scan through the whole training data once again to find the remaining support vectors (SVs). We show that iSVM is especially efficient for training SVMs on applications where data size is much larger than number of SVs. On the KDD-CUP 1999 network intrusion detection dataset with nearly five millions training examples, iSVM takes less than one hour to train an SVM with 94% testing accuracy, compared to seven hours with LibSVM - one of the state-of-the-art SVM implementations. We also provide analysis and experimental comparisons between iSVM and the related algorithms.

Original languageEnglish
Title of host publicationRIVF 2008 - 2008 IEEE International Conference on Research, Innovation and Vision for the Future in Computing and Communication Technologies
Pages221-226
Number of pages6
DOIs
Publication statusPublished - 2008 Sep 23
Externally publishedYes
EventRIVF 2008 - 2008 IEEE International Conference on Research, Innovation and Vision for the Future in Computing and Communication Technologies - Ho Chi Minh City, Viet Nam
Duration: 2008 Jul 132008 Jul 17

Publication series

NameRIVF 2008 - 2008 IEEE International Conference on Research, Innovation and Vision for the Future in Computing and Communication Technologies

Conference

ConferenceRIVF 2008 - 2008 IEEE International Conference on Research, Innovation and Vision for the Future in Computing and Communication Technologies
CountryViet Nam
CityHo Chi Minh City
Period08/7/1308/7/17

Keywords

  • Decomposition method
  • Optimization
  • Sequential minimal optimization
  • Support vector machine

ASJC Scopus subject areas

  • Artificial Intelligence
  • Computer Networks and Communications
  • Computer Vision and Pattern Recognition

Fingerprint Dive into the research topics of 'Two-stage incremental working set selection for fast support vector training on large datasets'. Together they form a unique fingerprint.

  • Cite this

    Nguyen, D. D., Matsumoto, K., Takishima, Y., Hashimoto, K., & Terabe, M. (2008). Two-stage incremental working set selection for fast support vector training on large datasets. In RIVF 2008 - 2008 IEEE International Conference on Research, Innovation and Vision for the Future in Computing and Communication Technologies (pp. 221-226). [4586359] (RIVF 2008 - 2008 IEEE International Conference on Research, Innovation and Vision for the Future in Computing and Communication Technologies). https://doi.org/10.1109/RIVF.2008.4586359