Open answer scoring for S-CAT automated speaking test system using support vector regression

Yutaka Ono, Misuzu Otake, Takahiro Shinozaki, Ryuichi Nisimura, Takeshi Yamada, Kenkichi Ishizuka, Yasuo Horiuchi, Shingo Kuroiwa, Shingo Imai

Research output: Chapter in Book/Report/Conference proceedingConference contribution

1 Citation (Scopus)

Abstract

We are developing S-CAT computer test system that will be the first automated adaptive speaking test for Japanese. The speaking ability of examinees is scored using speech processing techniques without human raters. By using computers for the scoring, it is possible to largely reduce the scoring cost and provide a convenient means for language learners to evaluate their learning status. While the S-CAT test has several categories of question items, open answer question is technically the most challenging one since examinees freely talk about a given topic or argue something for a given material. For this problem, we proposed to use support vector regression (SVR) with various features. Some of the features rely on speech recognition hypothesis and others do not. SVR is more robust than multiple regression and the best result was obtained when 390 dimensional features that combine everything were used. The correlation coefficients between human rated and SVR estimated scores were 0.878, 0.847, 0.853, and 0.872 for fluency, accuracy, content, and richness measures, respectively.

Original languageEnglish
Title of host publication2012 Conference Handbook - Asia-Pacific Signal and Information Processing Association Annual Summit and Conference, APSIPA ASC 2012
Publication statusPublished - 2012 Dec 1
Externally publishedYes
Event2012 4th Asia-Pacific Signal and Information Processing Association Annual Summit and Conference, APSIPA ASC 2012 - Hollywood, CA
Duration: 2012 Dec 32012 Dec 6

Other

Other2012 4th Asia-Pacific Signal and Information Processing Association Annual Summit and Conference, APSIPA ASC 2012
CityHollywood, CA
Period12/12/312/12/6

Fingerprint

Speech processing
Speech recognition
Costs

ASJC Scopus subject areas

  • Information Systems

Cite this

Ono, Y., Otake, M., Shinozaki, T., Nisimura, R., Yamada, T., Ishizuka, K., ... Imai, S. (2012). Open answer scoring for S-CAT automated speaking test system using support vector regression. In 2012 Conference Handbook - Asia-Pacific Signal and Information Processing Association Annual Summit and Conference, APSIPA ASC 2012 [6411787]

Open answer scoring for S-CAT automated speaking test system using support vector regression. / Ono, Yutaka; Otake, Misuzu; Shinozaki, Takahiro; Nisimura, Ryuichi; Yamada, Takeshi; Ishizuka, Kenkichi; Horiuchi, Yasuo; Kuroiwa, Shingo; Imai, Shingo.

2012 Conference Handbook - Asia-Pacific Signal and Information Processing Association Annual Summit and Conference, APSIPA ASC 2012. 2012. 6411787.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Ono, Y, Otake, M, Shinozaki, T, Nisimura, R, Yamada, T, Ishizuka, K, Horiuchi, Y, Kuroiwa, S & Imai, S 2012, Open answer scoring for S-CAT automated speaking test system using support vector regression. in 2012 Conference Handbook - Asia-Pacific Signal and Information Processing Association Annual Summit and Conference, APSIPA ASC 2012., 6411787, 2012 4th Asia-Pacific Signal and Information Processing Association Annual Summit and Conference, APSIPA ASC 2012, Hollywood, CA, 12/12/3.
Ono Y, Otake M, Shinozaki T, Nisimura R, Yamada T, Ishizuka K et al. Open answer scoring for S-CAT automated speaking test system using support vector regression. In 2012 Conference Handbook - Asia-Pacific Signal and Information Processing Association Annual Summit and Conference, APSIPA ASC 2012. 2012. 6411787
Ono, Yutaka ; Otake, Misuzu ; Shinozaki, Takahiro ; Nisimura, Ryuichi ; Yamada, Takeshi ; Ishizuka, Kenkichi ; Horiuchi, Yasuo ; Kuroiwa, Shingo ; Imai, Shingo. / Open answer scoring for S-CAT automated speaking test system using support vector regression. 2012 Conference Handbook - Asia-Pacific Signal and Information Processing Association Annual Summit and Conference, APSIPA ASC 2012. 2012.
@inproceedings{380f7e1c1fd142e985d1469269b2abd4,
title = "Open answer scoring for S-CAT automated speaking test system using support vector regression",
abstract = "We are developing S-CAT computer test system that will be the first automated adaptive speaking test for Japanese. The speaking ability of examinees is scored using speech processing techniques without human raters. By using computers for the scoring, it is possible to largely reduce the scoring cost and provide a convenient means for language learners to evaluate their learning status. While the S-CAT test has several categories of question items, open answer question is technically the most challenging one since examinees freely talk about a given topic or argue something for a given material. For this problem, we proposed to use support vector regression (SVR) with various features. Some of the features rely on speech recognition hypothesis and others do not. SVR is more robust than multiple regression and the best result was obtained when 390 dimensional features that combine everything were used. The correlation coefficients between human rated and SVR estimated scores were 0.878, 0.847, 0.853, and 0.872 for fluency, accuracy, content, and richness measures, respectively.",
author = "Yutaka Ono and Misuzu Otake and Takahiro Shinozaki and Ryuichi Nisimura and Takeshi Yamada and Kenkichi Ishizuka and Yasuo Horiuchi and Shingo Kuroiwa and Shingo Imai",
year = "2012",
month = "12",
day = "1",
language = "English",
isbn = "9780615700502",
booktitle = "2012 Conference Handbook - Asia-Pacific Signal and Information Processing Association Annual Summit and Conference, APSIPA ASC 2012",

}

TY - GEN

T1 - Open answer scoring for S-CAT automated speaking test system using support vector regression

AU - Ono, Yutaka

AU - Otake, Misuzu

AU - Shinozaki, Takahiro

AU - Nisimura, Ryuichi

AU - Yamada, Takeshi

AU - Ishizuka, Kenkichi

AU - Horiuchi, Yasuo

AU - Kuroiwa, Shingo

AU - Imai, Shingo

PY - 2012/12/1

Y1 - 2012/12/1

N2 - We are developing S-CAT computer test system that will be the first automated adaptive speaking test for Japanese. The speaking ability of examinees is scored using speech processing techniques without human raters. By using computers for the scoring, it is possible to largely reduce the scoring cost and provide a convenient means for language learners to evaluate their learning status. While the S-CAT test has several categories of question items, open answer question is technically the most challenging one since examinees freely talk about a given topic or argue something for a given material. For this problem, we proposed to use support vector regression (SVR) with various features. Some of the features rely on speech recognition hypothesis and others do not. SVR is more robust than multiple regression and the best result was obtained when 390 dimensional features that combine everything were used. The correlation coefficients between human rated and SVR estimated scores were 0.878, 0.847, 0.853, and 0.872 for fluency, accuracy, content, and richness measures, respectively.

AB - We are developing S-CAT computer test system that will be the first automated adaptive speaking test for Japanese. The speaking ability of examinees is scored using speech processing techniques without human raters. By using computers for the scoring, it is possible to largely reduce the scoring cost and provide a convenient means for language learners to evaluate their learning status. While the S-CAT test has several categories of question items, open answer question is technically the most challenging one since examinees freely talk about a given topic or argue something for a given material. For this problem, we proposed to use support vector regression (SVR) with various features. Some of the features rely on speech recognition hypothesis and others do not. SVR is more robust than multiple regression and the best result was obtained when 390 dimensional features that combine everything were used. The correlation coefficients between human rated and SVR estimated scores were 0.878, 0.847, 0.853, and 0.872 for fluency, accuracy, content, and richness measures, respectively.

UR - http://www.scopus.com/inward/record.url?scp=84874448433&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84874448433&partnerID=8YFLogxK

M3 - Conference contribution

AN - SCOPUS:84874448433

SN - 9780615700502

BT - 2012 Conference Handbook - Asia-Pacific Signal and Information Processing Association Annual Summit and Conference, APSIPA ASC 2012

ER -