RankwithTA: A robust and accurate peer grading mechanism for MOOCs

Hui Fang, Yufeng Wang, Qun Jin, Jianhua Ma

    Research output: Chapter in Book/Report/Conference proceedingConference contribution

    3 Citations (Scopus)

    Abstract

    Massive Online Open Courses (MOOCs) have the potential to revolutionize higher education with their wide outreach and accessibility. One of key challenges in MOOCs is the student evaluation: The large number of students makes it infeasible for instructors or teaching assistants (TAs) to grade all assignments. Peer grading-having students assess each other-is a promising approach to tackling the problem of evaluation at scale. The user evaluations are then used directly, or aggregated into a consensus value. However, lacking an incentive scheme, users have no motive in making effort in completing the evaluations, providing inaccurate answers instead. To address the above issues, we propose and implement a peer grading scheme, RankwithTA. Specifically, considering that the quality of a student determines both her performance in the assignment and her grading ability, RankwithTA makes the grade each student received depend on both the quality of the solution they submitted, and on the quality of their review and grading work to incentivize students' correct grading, Furthermore, the ground truth is incorporated, which utilizes external calibration by having some students graded by instructors or TAs to provide a basis for accuracy. The simulation results illustrate that RankwithTA performs better than the existing schemes.

    Original languageEnglish
    Title of host publicationProceedings of 2017 IEEE International Conference on Teaching, Assessment and Learning for Engineering, TALE 2017
    PublisherInstitute of Electrical and Electronics Engineers Inc.
    Pages497-502
    Number of pages6
    Volume2018-January
    ISBN (Electronic)9781538609002
    DOIs
    Publication statusPublished - 2018 Jan 8
    Event2017 IEEE International Conference on Teaching, Assessment and Learning for Engineering, TALE 2017 - Tai Po, Hong Kong
    Duration: 2017 Dec 122017 Dec 14

    Other

    Other2017 IEEE International Conference on Teaching, Assessment and Learning for Engineering, TALE 2017
    CountryHong Kong
    CityTai Po
    Period17/12/1217/12/14

      Fingerprint

    Keywords

    • Accuracy
    • Incentive
    • Massive Online Open Courses (MOOCs)
    • Peer grading

    ASJC Scopus subject areas

    • Engineering (miscellaneous)
    • Education

    Cite this

    Fang, H., Wang, Y., Jin, Q., & Ma, J. (2018). RankwithTA: A robust and accurate peer grading mechanism for MOOCs. In Proceedings of 2017 IEEE International Conference on Teaching, Assessment and Learning for Engineering, TALE 2017 (Vol. 2018-January, pp. 497-502). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/TALE.2017.8252331