An information theoretic perspective of the sparse coding

Hideitsu Hino, Noboru Murata

    Research output: Chapter in Book/Report/Conference proceedingConference contribution

    2 Citations (Scopus)

    Abstract

    The sparse coding method is formulated as an information theoretic optimization problem. The rate distortion theory leads to an objective functional which can be interpreted as an information theoretic formulation of the sparse coding. Viewing as an entropy minimization problem, the rate distortion theory and consequently the sparse coding are extended to discriminative variants. As a concrete example of this information theoretic sparse coding, a discriminative non-linear sparse coding algorithm with neural networks is proposed. Experimental results of gender classification by face images show that the discriminative sparse coding is more robust to noise, compared to the conventional method which directly uses images as inputs to a linear support vector machine.

    Original languageEnglish
    Title of host publicationLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
    Pages84-93
    Number of pages10
    Volume5551 LNCS
    EditionPART 1
    DOIs
    Publication statusPublished - 2009
    Event6th International Symposium on Neural Networks, ISNN 2009 - Wuhan
    Duration: 2009 May 262009 May 29

    Publication series

    NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
    NumberPART 1
    Volume5551 LNCS
    ISSN (Print)03029743
    ISSN (Electronic)16113349

    Other

    Other6th International Symposium on Neural Networks, ISNN 2009
    CityWuhan
    Period09/5/2609/5/29

    Fingerprint

    Sparse Coding
    Support vector machines
    Entropy
    Rate-distortion
    Neural networks
    Minimization Problem
    Support Vector Machine
    Face
    Neural Networks
    Optimization Problem
    Formulation
    Experimental Results

    Keywords

    • Gender Classification
    • Neural Network
    • Rate Distortion Theory
    • Sparse Coding

    ASJC Scopus subject areas

    • Computer Science(all)
    • Theoretical Computer Science

    Cite this

    Hino, H., & Murata, N. (2009). An information theoretic perspective of the sparse coding. In Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) (PART 1 ed., Vol. 5551 LNCS, pp. 84-93). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 5551 LNCS, No. PART 1). https://doi.org/10.1007/978-3-642-01507-6_11

    An information theoretic perspective of the sparse coding. / Hino, Hideitsu; Murata, Noboru.

    Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics). Vol. 5551 LNCS PART 1. ed. 2009. p. 84-93 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 5551 LNCS, No. PART 1).

    Research output: Chapter in Book/Report/Conference proceedingConference contribution

    Hino, H & Murata, N 2009, An information theoretic perspective of the sparse coding. in Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics). PART 1 edn, vol. 5551 LNCS, Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), no. PART 1, vol. 5551 LNCS, pp. 84-93, 6th International Symposium on Neural Networks, ISNN 2009, Wuhan, 09/5/26. https://doi.org/10.1007/978-3-642-01507-6_11
    Hino H, Murata N. An information theoretic perspective of the sparse coding. In Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics). PART 1 ed. Vol. 5551 LNCS. 2009. p. 84-93. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); PART 1). https://doi.org/10.1007/978-3-642-01507-6_11
    Hino, Hideitsu ; Murata, Noboru. / An information theoretic perspective of the sparse coding. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics). Vol. 5551 LNCS PART 1. ed. 2009. pp. 84-93 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); PART 1).
    @inproceedings{df71e1e3e77741689dd2bf2e9716924f,
    title = "An information theoretic perspective of the sparse coding",
    abstract = "The sparse coding method is formulated as an information theoretic optimization problem. The rate distortion theory leads to an objective functional which can be interpreted as an information theoretic formulation of the sparse coding. Viewing as an entropy minimization problem, the rate distortion theory and consequently the sparse coding are extended to discriminative variants. As a concrete example of this information theoretic sparse coding, a discriminative non-linear sparse coding algorithm with neural networks is proposed. Experimental results of gender classification by face images show that the discriminative sparse coding is more robust to noise, compared to the conventional method which directly uses images as inputs to a linear support vector machine.",
    keywords = "Gender Classification, Neural Network, Rate Distortion Theory, Sparse Coding",
    author = "Hideitsu Hino and Noboru Murata",
    year = "2009",
    doi = "10.1007/978-3-642-01507-6_11",
    language = "English",
    isbn = "3642015069",
    volume = "5551 LNCS",
    series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",
    number = "PART 1",
    pages = "84--93",
    booktitle = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",
    edition = "PART 1",

    }

    TY - GEN

    T1 - An information theoretic perspective of the sparse coding

    AU - Hino, Hideitsu

    AU - Murata, Noboru

    PY - 2009

    Y1 - 2009

    N2 - The sparse coding method is formulated as an information theoretic optimization problem. The rate distortion theory leads to an objective functional which can be interpreted as an information theoretic formulation of the sparse coding. Viewing as an entropy minimization problem, the rate distortion theory and consequently the sparse coding are extended to discriminative variants. As a concrete example of this information theoretic sparse coding, a discriminative non-linear sparse coding algorithm with neural networks is proposed. Experimental results of gender classification by face images show that the discriminative sparse coding is more robust to noise, compared to the conventional method which directly uses images as inputs to a linear support vector machine.

    AB - The sparse coding method is formulated as an information theoretic optimization problem. The rate distortion theory leads to an objective functional which can be interpreted as an information theoretic formulation of the sparse coding. Viewing as an entropy minimization problem, the rate distortion theory and consequently the sparse coding are extended to discriminative variants. As a concrete example of this information theoretic sparse coding, a discriminative non-linear sparse coding algorithm with neural networks is proposed. Experimental results of gender classification by face images show that the discriminative sparse coding is more robust to noise, compared to the conventional method which directly uses images as inputs to a linear support vector machine.

    KW - Gender Classification

    KW - Neural Network

    KW - Rate Distortion Theory

    KW - Sparse Coding

    UR - http://www.scopus.com/inward/record.url?scp=69949085278&partnerID=8YFLogxK

    UR - http://www.scopus.com/inward/citedby.url?scp=69949085278&partnerID=8YFLogxK

    U2 - 10.1007/978-3-642-01507-6_11

    DO - 10.1007/978-3-642-01507-6_11

    M3 - Conference contribution

    AN - SCOPUS:69949085278

    SN - 3642015069

    SN - 9783642015069

    VL - 5551 LNCS

    T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

    SP - 84

    EP - 93

    BT - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

    ER -