Social image tags as a source of word embeddings

A Task-oriented Evaluation

    Research output: Chapter in Book/Report/Conference proceedingConference contribution

    1 Citation (Scopus)

    Abstract

    Distributional hypothesis has been playing a central role in statistical NLP. Recently, however, its limitation in incorporating perceptual and empirical knowledge is noted, eliciting a field of perceptually grounded computational semantics. Typical sources of features in such a research are image datasets, where images are accompanied by linguistic tags and/or descriptions. Mainstream approaches employ machine learning techniques to integrate/combine visual features with linguistic features. In contrast to or supplementing these approaches, this study assesses the effectiveness of social image tags in generating word embeddings, and argues that these generated representations exhibit somewhat different and favorable behaviors from corpus-originated representations. More specifically, we generated word embeddings by using image tags obtained from a large social image dataset YFCC100M, which collects Flickr images and the associated tags. We evaluated the efficacy of generated word embeddings with standard semantic similarity/relatedness tasks, which showed that comparable performances with corpus-originated word embeddings were attained. These results further suggest that the generated embeddings could be effective in discriminating synonyms and antonyms, which has been an issue in distributional hypothesis-based approaches. In summary, social image tags can be utilized as yet another source of visually enforced features, provided the amount of available tags is large enough.

    Original languageEnglish
    Title of host publicationLREC 2018 - 11th International Conference on Language Resources and Evaluation
    EditorsHitoshi Isahara, Bente Maegaard, Stelios Piperidis, Christopher Cieri, Thierry Declerck, Koiti Hasida, Helene Mazo, Khalid Choukri, Sara Goggi, Joseph Mariani, Asuncion Moreno, Nicoletta Calzolari, Jan Odijk, Takenobu Tokunaga
    PublisherEuropean Language Resources Association (ELRA)
    Pages969-973
    Number of pages5
    ISBN (Electronic)9791095546009
    Publication statusPublished - 2019 Jan 1
    Event11th International Conference on Language Resources and Evaluation, LREC 2018 - Miyazaki, Japan
    Duration: 2018 May 72018 May 12

    Other

    Other11th International Conference on Language Resources and Evaluation, LREC 2018
    CountryJapan
    CityMiyazaki
    Period18/5/718/5/12

    Fingerprint

    evaluation
    semantics
    linguistics
    Tag
    Evaluation
    learning
    performance

    Keywords

    • Antonyms
    • Image tags
    • Semantic similarity
    • Social media
    • Synonyms
    • Word embeddings

    ASJC Scopus subject areas

    • Linguistics and Language
    • Education
    • Library and Information Sciences
    • Language and Linguistics

    Cite this

    Hasegawa, M., Kobayashi, T., & Hayashi, Y. (2019). Social image tags as a source of word embeddings: A Task-oriented Evaluation. In H. Isahara, B. Maegaard, S. Piperidis, C. Cieri, T. Declerck, K. Hasida, H. Mazo, K. Choukri, S. Goggi, J. Mariani, A. Moreno, N. Calzolari, J. Odijk, ... T. Tokunaga (Eds.), LREC 2018 - 11th International Conference on Language Resources and Evaluation (pp. 969-973). European Language Resources Association (ELRA).

    Social image tags as a source of word embeddings : A Task-oriented Evaluation. / Hasegawa, Mika; Kobayashi, Tetsunori; Hayashi, Yoshihiko.

    LREC 2018 - 11th International Conference on Language Resources and Evaluation. ed. / Hitoshi Isahara; Bente Maegaard; Stelios Piperidis; Christopher Cieri; Thierry Declerck; Koiti Hasida; Helene Mazo; Khalid Choukri; Sara Goggi; Joseph Mariani; Asuncion Moreno; Nicoletta Calzolari; Jan Odijk; Takenobu Tokunaga. European Language Resources Association (ELRA), 2019. p. 969-973.

    Research output: Chapter in Book/Report/Conference proceedingConference contribution

    Hasegawa, M, Kobayashi, T & Hayashi, Y 2019, Social image tags as a source of word embeddings: A Task-oriented Evaluation. in H Isahara, B Maegaard, S Piperidis, C Cieri, T Declerck, K Hasida, H Mazo, K Choukri, S Goggi, J Mariani, A Moreno, N Calzolari, J Odijk & T Tokunaga (eds), LREC 2018 - 11th International Conference on Language Resources and Evaluation. European Language Resources Association (ELRA), pp. 969-973, 11th International Conference on Language Resources and Evaluation, LREC 2018, Miyazaki, Japan, 18/5/7.
    Hasegawa M, Kobayashi T, Hayashi Y. Social image tags as a source of word embeddings: A Task-oriented Evaluation. In Isahara H, Maegaard B, Piperidis S, Cieri C, Declerck T, Hasida K, Mazo H, Choukri K, Goggi S, Mariani J, Moreno A, Calzolari N, Odijk J, Tokunaga T, editors, LREC 2018 - 11th International Conference on Language Resources and Evaluation. European Language Resources Association (ELRA). 2019. p. 969-973
    Hasegawa, Mika ; Kobayashi, Tetsunori ; Hayashi, Yoshihiko. / Social image tags as a source of word embeddings : A Task-oriented Evaluation. LREC 2018 - 11th International Conference on Language Resources and Evaluation. editor / Hitoshi Isahara ; Bente Maegaard ; Stelios Piperidis ; Christopher Cieri ; Thierry Declerck ; Koiti Hasida ; Helene Mazo ; Khalid Choukri ; Sara Goggi ; Joseph Mariani ; Asuncion Moreno ; Nicoletta Calzolari ; Jan Odijk ; Takenobu Tokunaga. European Language Resources Association (ELRA), 2019. pp. 969-973
    @inproceedings{c1757160dc36481b94321436f8bde704,
    title = "Social image tags as a source of word embeddings: A Task-oriented Evaluation",
    abstract = "Distributional hypothesis has been playing a central role in statistical NLP. Recently, however, its limitation in incorporating perceptual and empirical knowledge is noted, eliciting a field of perceptually grounded computational semantics. Typical sources of features in such a research are image datasets, where images are accompanied by linguistic tags and/or descriptions. Mainstream approaches employ machine learning techniques to integrate/combine visual features with linguistic features. In contrast to or supplementing these approaches, this study assesses the effectiveness of social image tags in generating word embeddings, and argues that these generated representations exhibit somewhat different and favorable behaviors from corpus-originated representations. More specifically, we generated word embeddings by using image tags obtained from a large social image dataset YFCC100M, which collects Flickr images and the associated tags. We evaluated the efficacy of generated word embeddings with standard semantic similarity/relatedness tasks, which showed that comparable performances with corpus-originated word embeddings were attained. These results further suggest that the generated embeddings could be effective in discriminating synonyms and antonyms, which has been an issue in distributional hypothesis-based approaches. In summary, social image tags can be utilized as yet another source of visually enforced features, provided the amount of available tags is large enough.",
    keywords = "Antonyms, Image tags, Semantic similarity, Social media, Synonyms, Word embeddings",
    author = "Mika Hasegawa and Tetsunori Kobayashi and Yoshihiko Hayashi",
    year = "2019",
    month = "1",
    day = "1",
    language = "English",
    pages = "969--973",
    editor = "Hitoshi Isahara and Bente Maegaard and Stelios Piperidis and Christopher Cieri and Thierry Declerck and Koiti Hasida and Helene Mazo and Khalid Choukri and Sara Goggi and Joseph Mariani and Asuncion Moreno and Nicoletta Calzolari and Jan Odijk and Takenobu Tokunaga",
    booktitle = "LREC 2018 - 11th International Conference on Language Resources and Evaluation",
    publisher = "European Language Resources Association (ELRA)",

    }

    TY - GEN

    T1 - Social image tags as a source of word embeddings

    T2 - A Task-oriented Evaluation

    AU - Hasegawa, Mika

    AU - Kobayashi, Tetsunori

    AU - Hayashi, Yoshihiko

    PY - 2019/1/1

    Y1 - 2019/1/1

    N2 - Distributional hypothesis has been playing a central role in statistical NLP. Recently, however, its limitation in incorporating perceptual and empirical knowledge is noted, eliciting a field of perceptually grounded computational semantics. Typical sources of features in such a research are image datasets, where images are accompanied by linguistic tags and/or descriptions. Mainstream approaches employ machine learning techniques to integrate/combine visual features with linguistic features. In contrast to or supplementing these approaches, this study assesses the effectiveness of social image tags in generating word embeddings, and argues that these generated representations exhibit somewhat different and favorable behaviors from corpus-originated representations. More specifically, we generated word embeddings by using image tags obtained from a large social image dataset YFCC100M, which collects Flickr images and the associated tags. We evaluated the efficacy of generated word embeddings with standard semantic similarity/relatedness tasks, which showed that comparable performances with corpus-originated word embeddings were attained. These results further suggest that the generated embeddings could be effective in discriminating synonyms and antonyms, which has been an issue in distributional hypothesis-based approaches. In summary, social image tags can be utilized as yet another source of visually enforced features, provided the amount of available tags is large enough.

    AB - Distributional hypothesis has been playing a central role in statistical NLP. Recently, however, its limitation in incorporating perceptual and empirical knowledge is noted, eliciting a field of perceptually grounded computational semantics. Typical sources of features in such a research are image datasets, where images are accompanied by linguistic tags and/or descriptions. Mainstream approaches employ machine learning techniques to integrate/combine visual features with linguistic features. In contrast to or supplementing these approaches, this study assesses the effectiveness of social image tags in generating word embeddings, and argues that these generated representations exhibit somewhat different and favorable behaviors from corpus-originated representations. More specifically, we generated word embeddings by using image tags obtained from a large social image dataset YFCC100M, which collects Flickr images and the associated tags. We evaluated the efficacy of generated word embeddings with standard semantic similarity/relatedness tasks, which showed that comparable performances with corpus-originated word embeddings were attained. These results further suggest that the generated embeddings could be effective in discriminating synonyms and antonyms, which has been an issue in distributional hypothesis-based approaches. In summary, social image tags can be utilized as yet another source of visually enforced features, provided the amount of available tags is large enough.

    KW - Antonyms

    KW - Image tags

    KW - Semantic similarity

    KW - Social media

    KW - Synonyms

    KW - Word embeddings

    UR - http://www.scopus.com/inward/record.url?scp=85059881983&partnerID=8YFLogxK

    UR - http://www.scopus.com/inward/citedby.url?scp=85059881983&partnerID=8YFLogxK

    M3 - Conference contribution

    SP - 969

    EP - 973

    BT - LREC 2018 - 11th International Conference on Language Resources and Evaluation

    A2 - Isahara, Hitoshi

    A2 - Maegaard, Bente

    A2 - Piperidis, Stelios

    A2 - Cieri, Christopher

    A2 - Declerck, Thierry

    A2 - Hasida, Koiti

    A2 - Mazo, Helene

    A2 - Choukri, Khalid

    A2 - Goggi, Sara

    A2 - Mariani, Joseph

    A2 - Moreno, Asuncion

    A2 - Calzolari, Nicoletta

    A2 - Odijk, Jan

    A2 - Tokunaga, Takenobu

    PB - European Language Resources Association (ELRA)

    ER -