Item difficulty parameter estimation using the idea of the graded response model and computerized adaptive testing

Koken Ozaki, Hideki Toyoda

Research output: Contribution to journalArticle

4 Citations (Scopus)

Abstract

In test operations using IRT (item response theory), items are included in a test before being used to rate subjects and the response data is used to estimate their item parameters. However, this method of test operation may lead to item content leakage and an adequate test operation can become difficult. To address this problem, Ozaki and Toyoda (2005, 2006) developed item difficulty parameter estimation methods that use paired comparison data from the perspective of the difficulty of items as judged by raters familiar with the field. In the present paper, an improved method of item difficulty parameter estimation is developed. In this new method, an item for which the difficulty parameter is to be estimated is compared with multiple items simultaneously, from the perspective of their difficulty. This is not a one-to-one comparison but a one-to-many comparison. In the comparisons, raters are informed that items selected from an item pool are ordered according to difficulty. The order will provide insight to improve the accuracy of judgment.

Original languageEnglish
Pages (from-to)1-12
Number of pages12
JournalJapanese Psychological Research
Volume51
Issue number1
DOIs
Publication statusPublished - 2009 Mar

Fingerprint

Computer Simulation
Matched-Pair Analysis

Keywords

  • Computerized adaptive testing
  • Difficulty parameter estimation
  • Equating
  • Graded response model
  • Test operation

ASJC Scopus subject areas

  • Psychology(all)

Cite this

Item difficulty parameter estimation using the idea of the graded response model and computerized adaptive testing. / Ozaki, Koken; Toyoda, Hideki.

In: Japanese Psychological Research, Vol. 51, No. 1, 03.2009, p. 1-12.

Research output: Contribution to journalArticle

@article{120867cd64c246dbace01d199e6a04ea,
title = "Item difficulty parameter estimation using the idea of the graded response model and computerized adaptive testing",
abstract = "In test operations using IRT (item response theory), items are included in a test before being used to rate subjects and the response data is used to estimate their item parameters. However, this method of test operation may lead to item content leakage and an adequate test operation can become difficult. To address this problem, Ozaki and Toyoda (2005, 2006) developed item difficulty parameter estimation methods that use paired comparison data from the perspective of the difficulty of items as judged by raters familiar with the field. In the present paper, an improved method of item difficulty parameter estimation is developed. In this new method, an item for which the difficulty parameter is to be estimated is compared with multiple items simultaneously, from the perspective of their difficulty. This is not a one-to-one comparison but a one-to-many comparison. In the comparisons, raters are informed that items selected from an item pool are ordered according to difficulty. The order will provide insight to improve the accuracy of judgment.",
keywords = "Computerized adaptive testing, Difficulty parameter estimation, Equating, Graded response model, Test operation",
author = "Koken Ozaki and Hideki Toyoda",
year = "2009",
month = "3",
doi = "10.1111/j.1468-5884.2009.00383.x",
language = "English",
volume = "51",
pages = "1--12",
journal = "Japanese Psychological Research",
issn = "0021-5368",
publisher = "Wiley-Blackwell",
number = "1",

}

TY - JOUR

T1 - Item difficulty parameter estimation using the idea of the graded response model and computerized adaptive testing

AU - Ozaki, Koken

AU - Toyoda, Hideki

PY - 2009/3

Y1 - 2009/3

N2 - In test operations using IRT (item response theory), items are included in a test before being used to rate subjects and the response data is used to estimate their item parameters. However, this method of test operation may lead to item content leakage and an adequate test operation can become difficult. To address this problem, Ozaki and Toyoda (2005, 2006) developed item difficulty parameter estimation methods that use paired comparison data from the perspective of the difficulty of items as judged by raters familiar with the field. In the present paper, an improved method of item difficulty parameter estimation is developed. In this new method, an item for which the difficulty parameter is to be estimated is compared with multiple items simultaneously, from the perspective of their difficulty. This is not a one-to-one comparison but a one-to-many comparison. In the comparisons, raters are informed that items selected from an item pool are ordered according to difficulty. The order will provide insight to improve the accuracy of judgment.

AB - In test operations using IRT (item response theory), items are included in a test before being used to rate subjects and the response data is used to estimate their item parameters. However, this method of test operation may lead to item content leakage and an adequate test operation can become difficult. To address this problem, Ozaki and Toyoda (2005, 2006) developed item difficulty parameter estimation methods that use paired comparison data from the perspective of the difficulty of items as judged by raters familiar with the field. In the present paper, an improved method of item difficulty parameter estimation is developed. In this new method, an item for which the difficulty parameter is to be estimated is compared with multiple items simultaneously, from the perspective of their difficulty. This is not a one-to-one comparison but a one-to-many comparison. In the comparisons, raters are informed that items selected from an item pool are ordered according to difficulty. The order will provide insight to improve the accuracy of judgment.

KW - Computerized adaptive testing

KW - Difficulty parameter estimation

KW - Equating

KW - Graded response model

KW - Test operation

UR - http://www.scopus.com/inward/record.url?scp=62549104196&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=62549104196&partnerID=8YFLogxK

U2 - 10.1111/j.1468-5884.2009.00383.x

DO - 10.1111/j.1468-5884.2009.00383.x

M3 - Article

VL - 51

SP - 1

EP - 12

JO - Japanese Psychological Research

JF - Japanese Psychological Research

SN - 0021-5368

IS - 1

ER -