Automatic scoring method considering quality and content of speech for scat Japanese speaking test

Naoko Okubo, Yuto Yamahata, Takeshi Yamada, Shingo Imai, Kenkichi Ishizuka, Takahiro Shinozaki, Ryuichi Nisimura, Shoji Makino, Nobuhiko Kitawaki

Research output: Chapter in Book/Report/Conference proceedingConference contribution

1 Citation (Scopus)

Abstract

We are now developing a Japanese speaking test called SCAT, which is part of J-CAT (Japanese Computerized Adaptive Test), a free online proficiency test for Japanese language learners. In this paper, we focus on the sentence-reading-aloud task and the sentence generation task in SCAT, and propose an automatic scoring method for estimating the overall score of answer speech, which is holistically determined by language teachers according to a rating standard. In that process, teachers carefully consider different factors but do not rate the scores of them. We therefore analyze how each factor contributes to the overall score. The factors are divided into two categories: the quality of speech and the content of speech. The former includes pronunciation and intonation, and the latter representation and vocabulary. We then propose an automatic scoring method based on the analysis. Experimental results confirm that the proposed method gives relatively accurate estimates of the overall score.

Original languageEnglish
Title of host publicationProceedings of the 2012 International Conference on Speech Database and Assessments, Oriental COCOSDA 2012
Pages72-77
Number of pages6
DOIs
Publication statusPublished - 2012 Dec 1
Externally publishedYes
Event2012 15th International Conference on Speech Database and Assessments, Oriental COCOSDA 2012 - Macau
Duration: 2012 Dec 92012 Dec 12

Other

Other2012 15th International Conference on Speech Database and Assessments, Oriental COCOSDA 2012
CityMacau
Period12/12/912/12/12

Fingerprint

Research Design
Language Tests
Vocabulary
Statistical Factor Analysis
Reading
Language

Keywords

  • automatic scoring
  • content of speech
  • J-CAT
  • quality of speech
  • SCAT Japanese speaking test

ASJC Scopus subject areas

  • Software
  • Speech and Hearing

Cite this

Okubo, N., Yamahata, Y., Yamada, T., Imai, S., Ishizuka, K., Shinozaki, T., ... Kitawaki, N. (2012). Automatic scoring method considering quality and content of speech for scat Japanese speaking test. In Proceedings of the 2012 International Conference on Speech Database and Assessments, Oriental COCOSDA 2012 (pp. 72-77). [6422460] https://doi.org/10.1109/ICSDA.2012.6422460

Automatic scoring method considering quality and content of speech for scat Japanese speaking test. / Okubo, Naoko; Yamahata, Yuto; Yamada, Takeshi; Imai, Shingo; Ishizuka, Kenkichi; Shinozaki, Takahiro; Nisimura, Ryuichi; Makino, Shoji; Kitawaki, Nobuhiko.

Proceedings of the 2012 International Conference on Speech Database and Assessments, Oriental COCOSDA 2012. 2012. p. 72-77 6422460.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Okubo, N, Yamahata, Y, Yamada, T, Imai, S, Ishizuka, K, Shinozaki, T, Nisimura, R, Makino, S & Kitawaki, N 2012, Automatic scoring method considering quality and content of speech for scat Japanese speaking test. in Proceedings of the 2012 International Conference on Speech Database and Assessments, Oriental COCOSDA 2012., 6422460, pp. 72-77, 2012 15th International Conference on Speech Database and Assessments, Oriental COCOSDA 2012, Macau, 12/12/9. https://doi.org/10.1109/ICSDA.2012.6422460
Okubo N, Yamahata Y, Yamada T, Imai S, Ishizuka K, Shinozaki T et al. Automatic scoring method considering quality and content of speech for scat Japanese speaking test. In Proceedings of the 2012 International Conference on Speech Database and Assessments, Oriental COCOSDA 2012. 2012. p. 72-77. 6422460 https://doi.org/10.1109/ICSDA.2012.6422460
Okubo, Naoko ; Yamahata, Yuto ; Yamada, Takeshi ; Imai, Shingo ; Ishizuka, Kenkichi ; Shinozaki, Takahiro ; Nisimura, Ryuichi ; Makino, Shoji ; Kitawaki, Nobuhiko. / Automatic scoring method considering quality and content of speech for scat Japanese speaking test. Proceedings of the 2012 International Conference on Speech Database and Assessments, Oriental COCOSDA 2012. 2012. pp. 72-77
@inproceedings{58a0cde3254c463e854b1886bbea5331,
title = "Automatic scoring method considering quality and content of speech for scat Japanese speaking test",
abstract = "We are now developing a Japanese speaking test called SCAT, which is part of J-CAT (Japanese Computerized Adaptive Test), a free online proficiency test for Japanese language learners. In this paper, we focus on the sentence-reading-aloud task and the sentence generation task in SCAT, and propose an automatic scoring method for estimating the overall score of answer speech, which is holistically determined by language teachers according to a rating standard. In that process, teachers carefully consider different factors but do not rate the scores of them. We therefore analyze how each factor contributes to the overall score. The factors are divided into two categories: the quality of speech and the content of speech. The former includes pronunciation and intonation, and the latter representation and vocabulary. We then propose an automatic scoring method based on the analysis. Experimental results confirm that the proposed method gives relatively accurate estimates of the overall score.",
keywords = "automatic scoring, content of speech, J-CAT, quality of speech, SCAT Japanese speaking test",
author = "Naoko Okubo and Yuto Yamahata and Takeshi Yamada and Shingo Imai and Kenkichi Ishizuka and Takahiro Shinozaki and Ryuichi Nisimura and Shoji Makino and Nobuhiko Kitawaki",
year = "2012",
month = "12",
day = "1",
doi = "10.1109/ICSDA.2012.6422460",
language = "English",
isbn = "9781467328104",
pages = "72--77",
booktitle = "Proceedings of the 2012 International Conference on Speech Database and Assessments, Oriental COCOSDA 2012",

}

TY - GEN

T1 - Automatic scoring method considering quality and content of speech for scat Japanese speaking test

AU - Okubo, Naoko

AU - Yamahata, Yuto

AU - Yamada, Takeshi

AU - Imai, Shingo

AU - Ishizuka, Kenkichi

AU - Shinozaki, Takahiro

AU - Nisimura, Ryuichi

AU - Makino, Shoji

AU - Kitawaki, Nobuhiko

PY - 2012/12/1

Y1 - 2012/12/1

N2 - We are now developing a Japanese speaking test called SCAT, which is part of J-CAT (Japanese Computerized Adaptive Test), a free online proficiency test for Japanese language learners. In this paper, we focus on the sentence-reading-aloud task and the sentence generation task in SCAT, and propose an automatic scoring method for estimating the overall score of answer speech, which is holistically determined by language teachers according to a rating standard. In that process, teachers carefully consider different factors but do not rate the scores of them. We therefore analyze how each factor contributes to the overall score. The factors are divided into two categories: the quality of speech and the content of speech. The former includes pronunciation and intonation, and the latter representation and vocabulary. We then propose an automatic scoring method based on the analysis. Experimental results confirm that the proposed method gives relatively accurate estimates of the overall score.

AB - We are now developing a Japanese speaking test called SCAT, which is part of J-CAT (Japanese Computerized Adaptive Test), a free online proficiency test for Japanese language learners. In this paper, we focus on the sentence-reading-aloud task and the sentence generation task in SCAT, and propose an automatic scoring method for estimating the overall score of answer speech, which is holistically determined by language teachers according to a rating standard. In that process, teachers carefully consider different factors but do not rate the scores of them. We therefore analyze how each factor contributes to the overall score. The factors are divided into two categories: the quality of speech and the content of speech. The former includes pronunciation and intonation, and the latter representation and vocabulary. We then propose an automatic scoring method based on the analysis. Experimental results confirm that the proposed method gives relatively accurate estimates of the overall score.

KW - automatic scoring

KW - content of speech

KW - J-CAT

KW - quality of speech

KW - SCAT Japanese speaking test

UR - http://www.scopus.com/inward/record.url?scp=84874256962&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84874256962&partnerID=8YFLogxK

U2 - 10.1109/ICSDA.2012.6422460

DO - 10.1109/ICSDA.2012.6422460

M3 - Conference contribution

AN - SCOPUS:84874256962

SN - 9781467328104

SP - 72

EP - 77

BT - Proceedings of the 2012 International Conference on Speech Database and Assessments, Oriental COCOSDA 2012

ER -