RankwithTA: A robust and accurate peer grading mechanism for MOOCs

Hui Fang, Yufeng Wang, Qun Jin, Jianhua Ma

Research output: Chapter in Book/Report/Conference proceedingConference contribution

3 Citations (Scopus)

Abstract

Massive Online Open Courses (MOOCs) have the potential to revolutionize higher education with their wide outreach and accessibility. One of key challenges in MOOCs is the student evaluation: The large number of students makes it infeasible for instructors or teaching assistants (TAs) to grade all assignments. Peer grading-having students assess each other-is a promising approach to tackling the problem of evaluation at scale. The user evaluations are then used directly, or aggregated into a consensus value. However, lacking an incentive scheme, users have no motive in making effort in completing the evaluations, providing inaccurate answers instead. To address the above issues, we propose and implement a peer grading scheme, RankwithTA. Specifically, considering that the quality of a student determines both her performance in the assignment and her grading ability, RankwithTA makes the grade each student received depend on both the quality of the solution they submitted, and on the quality of their review and grading work to incentivize students' correct grading, Furthermore, the ground truth is incorporated, which utilizes external calibration by having some students graded by instructors or TAs to provide a basis for accuracy. The simulation results illustrate that RankwithTA performs better than the existing schemes.

Original languageEnglish
Title of host publicationProceedings of 2017 IEEE International Conference on Teaching, Assessment and Learning for Engineering, TALE 2017
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages497-502
Number of pages6
ISBN (Electronic)9781538609002
DOIs
Publication statusPublished - 2017 Jul 1
Event2017 IEEE International Conference on Teaching, Assessment and Learning for Engineering, TALE 2017 - Tai Po, Hong Kong
Duration: 2017 Dec 122017 Dec 14

Publication series

NameProceedings of 2017 IEEE International Conference on Teaching, Assessment and Learning for Engineering, TALE 2017
Volume2018-January

Other

Other2017 IEEE International Conference on Teaching, Assessment and Learning for Engineering, TALE 2017
CountryHong Kong
CityTai Po
Period17/12/1217/12/14

Keywords

  • Accuracy
  • Incentive
  • Massive Online Open Courses (MOOCs)
  • Peer grading

ASJC Scopus subject areas

  • Engineering (miscellaneous)
  • Education

Fingerprint Dive into the research topics of 'RankwithTA: A robust and accurate peer grading mechanism for MOOCs'. Together they form a unique fingerprint.

  • Cite this

    Fang, H., Wang, Y., Jin, Q., & Ma, J. (2017). RankwithTA: A robust and accurate peer grading mechanism for MOOCs. In Proceedings of 2017 IEEE International Conference on Teaching, Assessment and Learning for Engineering, TALE 2017 (pp. 497-502). (Proceedings of 2017 IEEE International Conference on Teaching, Assessment and Learning for Engineering, TALE 2017; Vol. 2018-January). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/TALE.2017.8252331