Complexity measures and concept learning

Andreas D. Pape, Kenneth J. Kurtz, Hiroki Sayama

Research output: Contribution to journalArticle

1 Citation (Scopus)

Abstract

The nature of concept learning is a core question in cognitive science. Theories must account for the relative difficulty of acquiring different concepts by supervised learners. For a canonical set of six category types, two distinct orderings of classification difficulty have been found. One ordering, which we call paradigm-specific, occurs when adult human learners classify objects with easily distinguishable characteristics such as size, shape, and shading. The general order occurs in all other known cases: when adult humans classify objects with characteristics that are not readily distinguished (e.g., brightness, saturation, hue); for children and monkeys; and when categorization difficulty is extrapolated from errors in identification learning. The paradigm-specific order was found to be predictable mathematically by measuring the logical complexity of tasks, i.e., how concisely the solution can be represented by logical rules.However, logical complexity explains only the paradigm-specific order but not the general order. Here we propose a new difficulty measurement, information complexity, that calculates the amount of uncertainty remaining when a subset of the dimensions are specified. This measurement is based on Shannon entropy. We show that, when the metric extracts minimal uncertainties, this new measurement predicts the paradigm-specific order for the canonical six category types, and when the metric extracts average uncertainties, this new measurement predicts the general order. Moreover, for learning category types beyond the canonical six, we find that the minimal-uncertainty formulation correctly predicts the paradigm-specific order as well or better than existing metrics (Boolean complexity and GIST) in most cases.

Original languageEnglish
Pages (from-to)66-75
Number of pages10
JournalJournal of Mathematical Psychology
Volume64-65
DOIs
Publication statusPublished - 2015 Feb 1
Externally publishedYes

Fingerprint

Concept Learning
Complexity Measure
Uncertainty
Learning
Paradigm
Cognitive Science
Metric
Predict
Entropy
Haplorhini
Classify
Luminance
Shading
Shannon Entropy
Brightness
Categorization
Saturation
Distinct
Calculate
Subset

Keywords

  • Complexity
  • Concepts
  • Induction
  • Learning

ASJC Scopus subject areas

  • Psychology(all)
  • Applied Mathematics

Cite this

Complexity measures and concept learning. / Pape, Andreas D.; Kurtz, Kenneth J.; Sayama, Hiroki.

In: Journal of Mathematical Psychology, Vol. 64-65, 01.02.2015, p. 66-75.

Research output: Contribution to journalArticle

Pape, Andreas D. ; Kurtz, Kenneth J. ; Sayama, Hiroki. / Complexity measures and concept learning. In: Journal of Mathematical Psychology. 2015 ; Vol. 64-65. pp. 66-75.
@article{6f45d549768e40bea51eb2c248f2a93e,
title = "Complexity measures and concept learning",
abstract = "The nature of concept learning is a core question in cognitive science. Theories must account for the relative difficulty of acquiring different concepts by supervised learners. For a canonical set of six category types, two distinct orderings of classification difficulty have been found. One ordering, which we call paradigm-specific, occurs when adult human learners classify objects with easily distinguishable characteristics such as size, shape, and shading. The general order occurs in all other known cases: when adult humans classify objects with characteristics that are not readily distinguished (e.g., brightness, saturation, hue); for children and monkeys; and when categorization difficulty is extrapolated from errors in identification learning. The paradigm-specific order was found to be predictable mathematically by measuring the logical complexity of tasks, i.e., how concisely the solution can be represented by logical rules.However, logical complexity explains only the paradigm-specific order but not the general order. Here we propose a new difficulty measurement, information complexity, that calculates the amount of uncertainty remaining when a subset of the dimensions are specified. This measurement is based on Shannon entropy. We show that, when the metric extracts minimal uncertainties, this new measurement predicts the paradigm-specific order for the canonical six category types, and when the metric extracts average uncertainties, this new measurement predicts the general order. Moreover, for learning category types beyond the canonical six, we find that the minimal-uncertainty formulation correctly predicts the paradigm-specific order as well or better than existing metrics (Boolean complexity and GIST) in most cases.",
keywords = "Complexity, Concepts, Induction, Learning",
author = "Pape, {Andreas D.} and Kurtz, {Kenneth J.} and Hiroki Sayama",
year = "2015",
month = "2",
day = "1",
doi = "10.1016/j.jmp.2015.01.001",
language = "English",
volume = "64-65",
pages = "66--75",
journal = "Journal of Mathematical Psychology",
issn = "0022-2496",
publisher = "Academic Press Inc.",

}

TY - JOUR

T1 - Complexity measures and concept learning

AU - Pape, Andreas D.

AU - Kurtz, Kenneth J.

AU - Sayama, Hiroki

PY - 2015/2/1

Y1 - 2015/2/1

N2 - The nature of concept learning is a core question in cognitive science. Theories must account for the relative difficulty of acquiring different concepts by supervised learners. For a canonical set of six category types, two distinct orderings of classification difficulty have been found. One ordering, which we call paradigm-specific, occurs when adult human learners classify objects with easily distinguishable characteristics such as size, shape, and shading. The general order occurs in all other known cases: when adult humans classify objects with characteristics that are not readily distinguished (e.g., brightness, saturation, hue); for children and monkeys; and when categorization difficulty is extrapolated from errors in identification learning. The paradigm-specific order was found to be predictable mathematically by measuring the logical complexity of tasks, i.e., how concisely the solution can be represented by logical rules.However, logical complexity explains only the paradigm-specific order but not the general order. Here we propose a new difficulty measurement, information complexity, that calculates the amount of uncertainty remaining when a subset of the dimensions are specified. This measurement is based on Shannon entropy. We show that, when the metric extracts minimal uncertainties, this new measurement predicts the paradigm-specific order for the canonical six category types, and when the metric extracts average uncertainties, this new measurement predicts the general order. Moreover, for learning category types beyond the canonical six, we find that the minimal-uncertainty formulation correctly predicts the paradigm-specific order as well or better than existing metrics (Boolean complexity and GIST) in most cases.

AB - The nature of concept learning is a core question in cognitive science. Theories must account for the relative difficulty of acquiring different concepts by supervised learners. For a canonical set of six category types, two distinct orderings of classification difficulty have been found. One ordering, which we call paradigm-specific, occurs when adult human learners classify objects with easily distinguishable characteristics such as size, shape, and shading. The general order occurs in all other known cases: when adult humans classify objects with characteristics that are not readily distinguished (e.g., brightness, saturation, hue); for children and monkeys; and when categorization difficulty is extrapolated from errors in identification learning. The paradigm-specific order was found to be predictable mathematically by measuring the logical complexity of tasks, i.e., how concisely the solution can be represented by logical rules.However, logical complexity explains only the paradigm-specific order but not the general order. Here we propose a new difficulty measurement, information complexity, that calculates the amount of uncertainty remaining when a subset of the dimensions are specified. This measurement is based on Shannon entropy. We show that, when the metric extracts minimal uncertainties, this new measurement predicts the paradigm-specific order for the canonical six category types, and when the metric extracts average uncertainties, this new measurement predicts the general order. Moreover, for learning category types beyond the canonical six, we find that the minimal-uncertainty formulation correctly predicts the paradigm-specific order as well or better than existing metrics (Boolean complexity and GIST) in most cases.

KW - Complexity

KW - Concepts

KW - Induction

KW - Learning

UR - http://www.scopus.com/inward/record.url?scp=84923040469&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84923040469&partnerID=8YFLogxK

U2 - 10.1016/j.jmp.2015.01.001

DO - 10.1016/j.jmp.2015.01.001

M3 - Article

VL - 64-65

SP - 66

EP - 75

JO - Journal of Mathematical Psychology

JF - Journal of Mathematical Psychology

SN - 0022-2496

ER -