Automatic scoring method for open answer task in the SJ-CAT speaking test considering utterance difficulty level

Hao Lu, Takeshi Yamada, Shingo Imai, Takahiro Shinozaki, Ryuichi Nisimura, Kenkichi Ishizuka, Shoji Makino, Nobuhiko Kitawaki

研究成果: Conference contribution

抄録

In this paper, we propose an automatic scoring method for the open answer task of the Japanese speaking test SJ-CAT. The proposed method first extracts a set of features from an input answer utterance and then estimates a vocabulary richness score by human raters, which ranges from 0 to 4, by employing SVR (support vector regression). We devised a novel set of features, namely text statistics weighted by word reliability, to assess the abundance of vocabulary and expression, and degree of word relevance based on the hierarchical distance in a thesaurus to evaluate the suitability of vocabulary. We confirmed experimentally that the proposed method provides good estimates of the human richness score, with a correlation coefficient of 0.92 and an RMSE (root mean square error) of 0.56. We also showed that the proposed method is relatively robust to differences among examinees and among questions used for training and testing.

本文言語English
ホスト出版物のタイトル2014 Asia-Pacific Signal and Information Processing Association Annual Summit and Conference, APSIPA 2014
出版社Institute of Electrical and Electronics Engineers Inc.
ISBN(電子版)9786163618238
DOI
出版ステータスPublished - 2014 2 12
外部発表はい
イベント2014 Asia-Pacific Signal and Information Processing Association Annual Summit and Conference, APSIPA 2014 - Chiang Mai, Thailand
継続期間: 2014 12 92014 12 12

出版物シリーズ

名前2014 Asia-Pacific Signal and Information Processing Association Annual Summit and Conference, APSIPA 2014

Other

Other2014 Asia-Pacific Signal and Information Processing Association Annual Summit and Conference, APSIPA 2014
国/地域Thailand
CityChiang Mai
Period14/12/914/12/12

ASJC Scopus subject areas

  • 信号処理
  • 情報システム

フィンガープリント

「Automatic scoring method for open answer task in the SJ-CAT speaking test considering utterance difficulty level」の研究トピックを掘り下げます。これらがまとまってユニークなフィンガープリントを構成します。

引用スタイル