Classifying community QA questions that contain an image

Kenta Tamaki, Riku Togashi, Sosuke Kato, Sumio Fujita, Hideyuki Maeda, Tetsuya Sakai

Research output: Chapter in Book/Report/Conference proceedingConference contribution

1 Citation (Scopus)

Abstract

We consider the problem of automatically assigning a category to a given question posted to a Community Question Answering (CQA) site, where the question contains not only text but also an image. For example, CQA users may post a photograph of a dress and ask the community "Is this appropriate for a wedding?" where the appropriate category for this question might be "Manners, Ceremonial occasions." We tackle this problem using Convolutional Neural Networks with a DualNet architecture for combining the image and text representations. Our experiments with real data from Yahoo Chiebukuro and crowdsourced gold-standard categories show that the DualNet approach outperforms a text-only baseline (p = .0000), a sum-and-product baseline (p = .0000), Multimodal Compact Bilinear pooling (p = .0000), and a combination of sum-and-product and MCB (p = .0000), where the p-values are based on a randomised Tukey Honestly Significant Difference test with B = 5000 trials.

Original languageEnglish
Title of host publicationICTIR 2018 - Proceedings of the 2018 ACM SIGIR International Conference on the Theory of Information Retrieval
PublisherAssociation for Computing Machinery, Inc
Pages219-222
Number of pages4
ISBN (Electronic)9781450356565
DOIs
Publication statusPublished - 2018 Sep 10
Event8th ACM SIGIR International Conference on the Theory of Information Retrieval, ICTIR 2018 - Tianjin, China
Duration: 2018 Sep 142018 Sep 17

Publication series

NameICTIR 2018 - Proceedings of the 2018 ACM SIGIR International Conference on the Theory of Information Retrieval

Conference

Conference8th ACM SIGIR International Conference on the Theory of Information Retrieval, ICTIR 2018
CountryChina
CityTianjin
Period18/9/1418/9/17

Fingerprint

Gold
Neural networks
Experiments

Keywords

  • community question answering
  • convolutional neural networks
  • question categorisation

ASJC Scopus subject areas

  • Information Systems
  • Computer Science (miscellaneous)

Cite this

Tamaki, K., Togashi, R., Kato, S., Fujita, S., Maeda, H., & Sakai, T. (2018). Classifying community QA questions that contain an image. In ICTIR 2018 - Proceedings of the 2018 ACM SIGIR International Conference on the Theory of Information Retrieval (pp. 219-222). (ICTIR 2018 - Proceedings of the 2018 ACM SIGIR International Conference on the Theory of Information Retrieval). Association for Computing Machinery, Inc. https://doi.org/10.1145/3234944.3234948

Classifying community QA questions that contain an image. / Tamaki, Kenta; Togashi, Riku; Kato, Sosuke; Fujita, Sumio; Maeda, Hideyuki; Sakai, Tetsuya.

ICTIR 2018 - Proceedings of the 2018 ACM SIGIR International Conference on the Theory of Information Retrieval. Association for Computing Machinery, Inc, 2018. p. 219-222 (ICTIR 2018 - Proceedings of the 2018 ACM SIGIR International Conference on the Theory of Information Retrieval).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Tamaki, K, Togashi, R, Kato, S, Fujita, S, Maeda, H & Sakai, T 2018, Classifying community QA questions that contain an image. in ICTIR 2018 - Proceedings of the 2018 ACM SIGIR International Conference on the Theory of Information Retrieval. ICTIR 2018 - Proceedings of the 2018 ACM SIGIR International Conference on the Theory of Information Retrieval, Association for Computing Machinery, Inc, pp. 219-222, 8th ACM SIGIR International Conference on the Theory of Information Retrieval, ICTIR 2018, Tianjin, China, 18/9/14. https://doi.org/10.1145/3234944.3234948
Tamaki K, Togashi R, Kato S, Fujita S, Maeda H, Sakai T. Classifying community QA questions that contain an image. In ICTIR 2018 - Proceedings of the 2018 ACM SIGIR International Conference on the Theory of Information Retrieval. Association for Computing Machinery, Inc. 2018. p. 219-222. (ICTIR 2018 - Proceedings of the 2018 ACM SIGIR International Conference on the Theory of Information Retrieval). https://doi.org/10.1145/3234944.3234948
Tamaki, Kenta ; Togashi, Riku ; Kato, Sosuke ; Fujita, Sumio ; Maeda, Hideyuki ; Sakai, Tetsuya. / Classifying community QA questions that contain an image. ICTIR 2018 - Proceedings of the 2018 ACM SIGIR International Conference on the Theory of Information Retrieval. Association for Computing Machinery, Inc, 2018. pp. 219-222 (ICTIR 2018 - Proceedings of the 2018 ACM SIGIR International Conference on the Theory of Information Retrieval).
@inproceedings{b46e1fb801c94fba88f5605f97d097ab,
title = "Classifying community QA questions that contain an image",
abstract = "We consider the problem of automatically assigning a category to a given question posted to a Community Question Answering (CQA) site, where the question contains not only text but also an image. For example, CQA users may post a photograph of a dress and ask the community {"}Is this appropriate for a wedding?{"} where the appropriate category for this question might be {"}Manners, Ceremonial occasions.{"} We tackle this problem using Convolutional Neural Networks with a DualNet architecture for combining the image and text representations. Our experiments with real data from Yahoo Chiebukuro and crowdsourced gold-standard categories show that the DualNet approach outperforms a text-only baseline (p = .0000), a sum-and-product baseline (p = .0000), Multimodal Compact Bilinear pooling (p = .0000), and a combination of sum-and-product and MCB (p = .0000), where the p-values are based on a randomised Tukey Honestly Significant Difference test with B = 5000 trials.",
keywords = "community question answering, convolutional neural networks, question categorisation",
author = "Kenta Tamaki and Riku Togashi and Sosuke Kato and Sumio Fujita and Hideyuki Maeda and Tetsuya Sakai",
year = "2018",
month = "9",
day = "10",
doi = "10.1145/3234944.3234948",
language = "English",
series = "ICTIR 2018 - Proceedings of the 2018 ACM SIGIR International Conference on the Theory of Information Retrieval",
publisher = "Association for Computing Machinery, Inc",
pages = "219--222",
booktitle = "ICTIR 2018 - Proceedings of the 2018 ACM SIGIR International Conference on the Theory of Information Retrieval",

}

TY - GEN

T1 - Classifying community QA questions that contain an image

AU - Tamaki, Kenta

AU - Togashi, Riku

AU - Kato, Sosuke

AU - Fujita, Sumio

AU - Maeda, Hideyuki

AU - Sakai, Tetsuya

PY - 2018/9/10

Y1 - 2018/9/10

N2 - We consider the problem of automatically assigning a category to a given question posted to a Community Question Answering (CQA) site, where the question contains not only text but also an image. For example, CQA users may post a photograph of a dress and ask the community "Is this appropriate for a wedding?" where the appropriate category for this question might be "Manners, Ceremonial occasions." We tackle this problem using Convolutional Neural Networks with a DualNet architecture for combining the image and text representations. Our experiments with real data from Yahoo Chiebukuro and crowdsourced gold-standard categories show that the DualNet approach outperforms a text-only baseline (p = .0000), a sum-and-product baseline (p = .0000), Multimodal Compact Bilinear pooling (p = .0000), and a combination of sum-and-product and MCB (p = .0000), where the p-values are based on a randomised Tukey Honestly Significant Difference test with B = 5000 trials.

AB - We consider the problem of automatically assigning a category to a given question posted to a Community Question Answering (CQA) site, where the question contains not only text but also an image. For example, CQA users may post a photograph of a dress and ask the community "Is this appropriate for a wedding?" where the appropriate category for this question might be "Manners, Ceremonial occasions." We tackle this problem using Convolutional Neural Networks with a DualNet architecture for combining the image and text representations. Our experiments with real data from Yahoo Chiebukuro and crowdsourced gold-standard categories show that the DualNet approach outperforms a text-only baseline (p = .0000), a sum-and-product baseline (p = .0000), Multimodal Compact Bilinear pooling (p = .0000), and a combination of sum-and-product and MCB (p = .0000), where the p-values are based on a randomised Tukey Honestly Significant Difference test with B = 5000 trials.

KW - community question answering

KW - convolutional neural networks

KW - question categorisation

UR - http://www.scopus.com/inward/record.url?scp=85063515836&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85063515836&partnerID=8YFLogxK

U2 - 10.1145/3234944.3234948

DO - 10.1145/3234944.3234948

M3 - Conference contribution

AN - SCOPUS:85063515836

T3 - ICTIR 2018 - Proceedings of the 2018 ACM SIGIR International Conference on the Theory of Information Retrieval

SP - 219

EP - 222

BT - ICTIR 2018 - Proceedings of the 2018 ACM SIGIR International Conference on the Theory of Information Retrieval

PB - Association for Computing Machinery, Inc

ER -