Relevant, Irredundant Feature Selection and Noisy Example Elimination

George V. Lashkia*, Laurence Anthony

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

29 Citations (Scopus)


In many real-world situations, the method for computing the desired output from a set of inputs is unknown. One strategy for solving these types of problems is to learn the input-output functionality from examples in a training set. However, in many situations it is difficult to know what information is relevant to the task at hand. Subsequently, researchers have investigated ways to deal with the so-called problem of consistency of attributes, i.e., attributes that can distinguish examples from different classes. In this paper, we first prove that the notion of relevance of attributes is directly related to the consistency of attributes, and show how relevant, irredundant attributes can be selected. We then compare different relevant attribute selection algorithms, and show the superiority of algorithms that select irredundant attributes over those that select relevant attributes. We also show that searching for an "optimal" subset of attributes, which is considered to be the main purpose of attribute selection, is not the best way to improve the accuracy of classifiers. Employing sets of relevant, irredundant attributes improves classification accuracy in many more cases. Finally, we propose a new method for selecting relevant examples, which is based on filtering the so-called pattern frequency domain. By identifying examples that are nontypical in the determination of relevant, irredundant attributes, irrelevant examples can be eliminated prior to the learning process. Empirical results using artificial and real databases show the effectiveness of the proposed method in selecting relevant examples leading to improved performance even on greatly reduced training sets.

Original languageEnglish
Pages (from-to)888-897
Number of pages10
JournalIEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics
Issue number2
Publication statusPublished - 2004 Apr 1
Externally publishedYes


  • Example selection
  • Feature selection
  • Relevance
  • Supervised learning
  • Test

ASJC Scopus subject areas

  • Control and Systems Engineering
  • Software
  • Information Systems
  • Human-Computer Interaction
  • Computer Science Applications
  • Electrical and Electronic Engineering


Dive into the research topics of 'Relevant, Irredundant Feature Selection and Noisy Example Elimination'. Together they form a unique fingerprint.

Cite this