An Item Response Theory for peer assessment

Maomi Ueno, Keizo Nagaoka

    Research output: Chapter in Book/Report/Conference proceedingConference contribution

    Abstract

    This paper proposes an Item Response Theory for peer assessment. The proposed model is a modified Graded Item Response model which incorporates the assessor's assessment criterion parameters. Its advantages are as follows: 1. The learners' abilities are evaluated on the same scale even if the assessors have different assessment criteria. 2. The learners' abilities are evaluated by taking into consideration each assessor's reliability, which produces reliable evaluation results. 3. Parameters from missing data can be easily estimated. 4. From 1-3, the parameter estimation accuracy is improved. 5. Characteristics of the tasks and assessors can be analyzed from the estimated parameters. Moreover, the two indices, "strictness of the assessment criterion" and "consistency of the assessor," are proposed based on the model. The proposed model was applied to actual data, which demonstrated its effectiveness.

    Original languageEnglish
    Title of host publicationProceedings of the 7th IASTED International Conference on Web-Based Education, WBE 2008
    Pages229-234
    Number of pages6
    Publication statusPublished - 2008
    Event7th IASTED International Conference on Web-Based Education, WBE 2008 - Innsbruck
    Duration: 2008 Mar 172008 Mar 19

    Other

    Other7th IASTED International Conference on Web-Based Education, WBE 2008
    CityInnsbruck
    Period08/3/1708/3/19

    Fingerprint

    assessment criteria
    ability
    Parameter estimation
    evaluation

    Keywords

    • Collaborative learning
    • Cscl
    • E-learning
    • Peer assessment
    • Portfolio

    ASJC Scopus subject areas

    • Computer Networks and Communications
    • Human-Computer Interaction
    • Education

    Cite this

    Ueno, M., & Nagaoka, K. (2008). An Item Response Theory for peer assessment. In Proceedings of the 7th IASTED International Conference on Web-Based Education, WBE 2008 (pp. 229-234)

    An Item Response Theory for peer assessment. / Ueno, Maomi; Nagaoka, Keizo.

    Proceedings of the 7th IASTED International Conference on Web-Based Education, WBE 2008. 2008. p. 229-234.

    Research output: Chapter in Book/Report/Conference proceedingConference contribution

    Ueno, M & Nagaoka, K 2008, An Item Response Theory for peer assessment. in Proceedings of the 7th IASTED International Conference on Web-Based Education, WBE 2008. pp. 229-234, 7th IASTED International Conference on Web-Based Education, WBE 2008, Innsbruck, 08/3/17.
    Ueno M, Nagaoka K. An Item Response Theory for peer assessment. In Proceedings of the 7th IASTED International Conference on Web-Based Education, WBE 2008. 2008. p. 229-234
    Ueno, Maomi ; Nagaoka, Keizo. / An Item Response Theory for peer assessment. Proceedings of the 7th IASTED International Conference on Web-Based Education, WBE 2008. 2008. pp. 229-234
    @inproceedings{fb8c2e501cc8441eaf08221a048f7699,
    title = "An Item Response Theory for peer assessment",
    abstract = "This paper proposes an Item Response Theory for peer assessment. The proposed model is a modified Graded Item Response model which incorporates the assessor's assessment criterion parameters. Its advantages are as follows: 1. The learners' abilities are evaluated on the same scale even if the assessors have different assessment criteria. 2. The learners' abilities are evaluated by taking into consideration each assessor's reliability, which produces reliable evaluation results. 3. Parameters from missing data can be easily estimated. 4. From 1-3, the parameter estimation accuracy is improved. 5. Characteristics of the tasks and assessors can be analyzed from the estimated parameters. Moreover, the two indices, {"}strictness of the assessment criterion{"} and {"}consistency of the assessor,{"} are proposed based on the model. The proposed model was applied to actual data, which demonstrated its effectiveness.",
    keywords = "Collaborative learning, Cscl, E-learning, Peer assessment, Portfolio",
    author = "Maomi Ueno and Keizo Nagaoka",
    year = "2008",
    language = "English",
    isbn = "9780889867239",
    pages = "229--234",
    booktitle = "Proceedings of the 7th IASTED International Conference on Web-Based Education, WBE 2008",

    }

    TY - GEN

    T1 - An Item Response Theory for peer assessment

    AU - Ueno, Maomi

    AU - Nagaoka, Keizo

    PY - 2008

    Y1 - 2008

    N2 - This paper proposes an Item Response Theory for peer assessment. The proposed model is a modified Graded Item Response model which incorporates the assessor's assessment criterion parameters. Its advantages are as follows: 1. The learners' abilities are evaluated on the same scale even if the assessors have different assessment criteria. 2. The learners' abilities are evaluated by taking into consideration each assessor's reliability, which produces reliable evaluation results. 3. Parameters from missing data can be easily estimated. 4. From 1-3, the parameter estimation accuracy is improved. 5. Characteristics of the tasks and assessors can be analyzed from the estimated parameters. Moreover, the two indices, "strictness of the assessment criterion" and "consistency of the assessor," are proposed based on the model. The proposed model was applied to actual data, which demonstrated its effectiveness.

    AB - This paper proposes an Item Response Theory for peer assessment. The proposed model is a modified Graded Item Response model which incorporates the assessor's assessment criterion parameters. Its advantages are as follows: 1. The learners' abilities are evaluated on the same scale even if the assessors have different assessment criteria. 2. The learners' abilities are evaluated by taking into consideration each assessor's reliability, which produces reliable evaluation results. 3. Parameters from missing data can be easily estimated. 4. From 1-3, the parameter estimation accuracy is improved. 5. Characteristics of the tasks and assessors can be analyzed from the estimated parameters. Moreover, the two indices, "strictness of the assessment criterion" and "consistency of the assessor," are proposed based on the model. The proposed model was applied to actual data, which demonstrated its effectiveness.

    KW - Collaborative learning

    KW - Cscl

    KW - E-learning

    KW - Peer assessment

    KW - Portfolio

    UR - http://www.scopus.com/inward/record.url?scp=62949159415&partnerID=8YFLogxK

    UR - http://www.scopus.com/inward/citedby.url?scp=62949159415&partnerID=8YFLogxK

    M3 - Conference contribution

    SN - 9780889867239

    SP - 229

    EP - 234

    BT - Proceedings of the 7th IASTED International Conference on Web-Based Education, WBE 2008

    ER -