Leveraging features from background and salient regions for automatic image annotation

Supheakmungkol Sarin, Michael Fahrmair, Matthias Wagner, Wataru Kameyama

    Research output: Contribution to journalArticle

    Abstract

    In this era of information explosion, automating the annotation process of digital images is a crucial step towards efficient and effective management of this increasingly high volume of content. However, this still is a highly challenging task for the research community. One of the main bottlenecks is the lack of integrity and diversity of features. We propose to solve this problem by utilizing 43 image features that cover the holistic content of the image from global to subject, background and scene. In our approach, salient regions and the background are separated without prior knowledge. Each of them together with the whole image are treated independently for feature extraction. Extensive experiments were designed to show the efficiency and the effectiveness of our approach. We chose two publicly available datasets manually annotated with diverse nature of images for our experiments, namely, the Corel5K and ESP Game datasets. We confirm the superior performance of our approach over the use of a single whole image using sign test with p-value < 0.05. Furthermore, our combined feature set gives satisfactory performance compared to recently proposed approaches especially in terms of generalization even with just a simple combination. We also obtain a better performance with the same feature set versus the grid-based approach. More importantly, when using our features with the state-of-the-art technique, our results show higher performance in a variety of standard metrics.

    Original languageEnglish
    Pages (from-to)250-266
    Number of pages17
    JournalJournal of Information Processing
    Volume20
    Issue number1
    DOIs
    Publication statusPublished - 2012

    Fingerprint

    Explosions
    Feature extraction
    Experiments

    Keywords

    • Automatic image annotation
    • Background
    • Holistic features extraction
    • K nearest neighbours
    • Salient regions

    ASJC Scopus subject areas

    • Computer Science(all)

    Cite this

    Leveraging features from background and salient regions for automatic image annotation. / Sarin, Supheakmungkol; Fahrmair, Michael; Wagner, Matthias; Kameyama, Wataru.

    In: Journal of Information Processing, Vol. 20, No. 1, 2012, p. 250-266.

    Research output: Contribution to journalArticle

    Sarin, Supheakmungkol ; Fahrmair, Michael ; Wagner, Matthias ; Kameyama, Wataru. / Leveraging features from background and salient regions for automatic image annotation. In: Journal of Information Processing. 2012 ; Vol. 20, No. 1. pp. 250-266.
    @article{21627763889f4d81922d4577ab3b6d12,
    title = "Leveraging features from background and salient regions for automatic image annotation",
    abstract = "In this era of information explosion, automating the annotation process of digital images is a crucial step towards efficient and effective management of this increasingly high volume of content. However, this still is a highly challenging task for the research community. One of the main bottlenecks is the lack of integrity and diversity of features. We propose to solve this problem by utilizing 43 image features that cover the holistic content of the image from global to subject, background and scene. In our approach, salient regions and the background are separated without prior knowledge. Each of them together with the whole image are treated independently for feature extraction. Extensive experiments were designed to show the efficiency and the effectiveness of our approach. We chose two publicly available datasets manually annotated with diverse nature of images for our experiments, namely, the Corel5K and ESP Game datasets. We confirm the superior performance of our approach over the use of a single whole image using sign test with p-value < 0.05. Furthermore, our combined feature set gives satisfactory performance compared to recently proposed approaches especially in terms of generalization even with just a simple combination. We also obtain a better performance with the same feature set versus the grid-based approach. More importantly, when using our features with the state-of-the-art technique, our results show higher performance in a variety of standard metrics.",
    keywords = "Automatic image annotation, Background, Holistic features extraction, K nearest neighbours, Salient regions",
    author = "Supheakmungkol Sarin and Michael Fahrmair and Matthias Wagner and Wataru Kameyama",
    year = "2012",
    doi = "10.2197/ipsjjip.vol.20.250",
    language = "English",
    volume = "20",
    pages = "250--266",
    journal = "Journal of Information Processing",
    issn = "0387-5806",
    publisher = "Information Processing Society of Japan",
    number = "1",

    }

    TY - JOUR

    T1 - Leveraging features from background and salient regions for automatic image annotation

    AU - Sarin, Supheakmungkol

    AU - Fahrmair, Michael

    AU - Wagner, Matthias

    AU - Kameyama, Wataru

    PY - 2012

    Y1 - 2012

    N2 - In this era of information explosion, automating the annotation process of digital images is a crucial step towards efficient and effective management of this increasingly high volume of content. However, this still is a highly challenging task for the research community. One of the main bottlenecks is the lack of integrity and diversity of features. We propose to solve this problem by utilizing 43 image features that cover the holistic content of the image from global to subject, background and scene. In our approach, salient regions and the background are separated without prior knowledge. Each of them together with the whole image are treated independently for feature extraction. Extensive experiments were designed to show the efficiency and the effectiveness of our approach. We chose two publicly available datasets manually annotated with diverse nature of images for our experiments, namely, the Corel5K and ESP Game datasets. We confirm the superior performance of our approach over the use of a single whole image using sign test with p-value < 0.05. Furthermore, our combined feature set gives satisfactory performance compared to recently proposed approaches especially in terms of generalization even with just a simple combination. We also obtain a better performance with the same feature set versus the grid-based approach. More importantly, when using our features with the state-of-the-art technique, our results show higher performance in a variety of standard metrics.

    AB - In this era of information explosion, automating the annotation process of digital images is a crucial step towards efficient and effective management of this increasingly high volume of content. However, this still is a highly challenging task for the research community. One of the main bottlenecks is the lack of integrity and diversity of features. We propose to solve this problem by utilizing 43 image features that cover the holistic content of the image from global to subject, background and scene. In our approach, salient regions and the background are separated without prior knowledge. Each of them together with the whole image are treated independently for feature extraction. Extensive experiments were designed to show the efficiency and the effectiveness of our approach. We chose two publicly available datasets manually annotated with diverse nature of images for our experiments, namely, the Corel5K and ESP Game datasets. We confirm the superior performance of our approach over the use of a single whole image using sign test with p-value < 0.05. Furthermore, our combined feature set gives satisfactory performance compared to recently proposed approaches especially in terms of generalization even with just a simple combination. We also obtain a better performance with the same feature set versus the grid-based approach. More importantly, when using our features with the state-of-the-art technique, our results show higher performance in a variety of standard metrics.

    KW - Automatic image annotation

    KW - Background

    KW - Holistic features extraction

    KW - K nearest neighbours

    KW - Salient regions

    UR - http://www.scopus.com/inward/record.url?scp=84871207775&partnerID=8YFLogxK

    UR - http://www.scopus.com/inward/citedby.url?scp=84871207775&partnerID=8YFLogxK

    U2 - 10.2197/ipsjjip.vol.20.250

    DO - 10.2197/ipsjjip.vol.20.250

    M3 - Article

    AN - SCOPUS:84871207775

    VL - 20

    SP - 250

    EP - 266

    JO - Journal of Information Processing

    JF - Journal of Information Processing

    SN - 0387-5806

    IS - 1

    ER -