Evaluating evaluation measures for ordinal classification and ordinal quantification

Tetsuya Sakai*

*この研究の対応する著者

研究成果: Conference contribution

1 被引用数 (Scopus)

抄録

Ordinal Classification (OC) is an important classification task where the classes are ordinal. For example, an OC task for sentiment analysis could have the following classes: highly positive, positive, neutral, negative, highly negative. Clearly, evaluation measures for an OC task should penalise misclassifications by considering the ordinal nature of the classes (e.g., highly positive misclassified as positive vs. misclassifed as highly negative). Ordinal Quantification (OQ) is a related task where the gold data is a distribution over ordinal classes, and the system is required to estimate this distribution. Evaluation measures for an OQ task should also take the ordinal nature of the classes into account. However, for both OC and OQ, there are only a small number of known evaluation measures that meet this basic requirement. In the present study, we utilise data from the SemEval and NTCIR communities to clarify the properties of nine evaluation measures in the context of OC tasks, and six measures in the context of OQ tasks.

本文言語English
ホスト出版物のタイトルACL-IJCNLP 2021 - 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing, Proceedings of the Conference
出版社Association for Computational Linguistics (ACL)
ページ2759-2769
ページ数11
ISBN(電子版)9781954085527
出版ステータスPublished - 2021
イベントJoint Conference of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing, ACL-IJCNLP 2021 - Virtual, Online
継続期間: 2021 8月 12021 8月 6

出版物シリーズ

名前ACL-IJCNLP 2021 - 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing, Proceedings of the Conference

Conference

ConferenceJoint Conference of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing, ACL-IJCNLP 2021
CityVirtual, Online
Period21/8/121/8/6

ASJC Scopus subject areas

  • ソフトウェア
  • 計算理論と計算数学
  • 言語学および言語
  • 言語および言語学

フィンガープリント

「Evaluating evaluation measures for ordinal classification and ordinal quantification」の研究トピックを掘り下げます。これらがまとまってユニークなフィンガープリントを構成します。

引用スタイル