TSP

Truthful Grading-Based Strategyproof Peer Selection for MOOCs

Yufeng Wang, Hui Fang, Chonghu Cheng, Qun Jin

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Considering that the challenges in using peer grading to select the best-k students in MOOCs (massive open online courses) are twofold: first is strategyproof, i.e., students should not benefit by untruthfully reporting valuations; second, instead of exerting (costly) efforts to evaluate, students may randomly provide (or just guess) the evaluations on other peers. This paper proposes a truthful grading-based strategyproof peer selection scheme for MOOCs, TSP. Specifically, all students are partitioned into same-size clusters, and each student only evaluates students in other clusters. Moreover, peer prediction mechanism was utilized to motivate each student to truthfully report their gradings through comparing their reports with other peer students who conduct same evaluation tasks. The theoretical analysis and simulation results show that TSP can both stimulate students to truthfully reporting their gradings, and meanwhile select best k-students in strategyproof way.

Original languageEnglish
Title of host publicationProceedings of 2018 IEEE International Conference on Teaching, Assessment, and Learning for Engineering, TALE 2018
EditorsMark J.W. Lee, Sasha Nikolic, Gary K.W. Wong, Jun Shen, Montserrat Ros, Leon C. U. Lei, Neelakantam Venkatarayalu
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages679-684
Number of pages6
ISBN (Electronic)9781538665220
DOIs
Publication statusPublished - 2019 Jan 16
Event2018 IEEE International Conference on Teaching, Assessment, and Learning for Engineering, TALE 2018 - Wollongong, Australia
Duration: 2018 Dec 42018 Dec 7

Publication series

NameProceedings of 2018 IEEE International Conference on Teaching, Assessment, and Learning for Engineering, TALE 2018

Conference

Conference2018 IEEE International Conference on Teaching, Assessment, and Learning for Engineering, TALE 2018
CountryAustralia
CityWollongong
Period18/12/418/12/7

Fingerprint

grading
Students
student
evaluation
simulation

Keywords

  • massive online open course (MOOC)
  • peer selection
  • strategyproof
  • truthful grading

ASJC Scopus subject areas

  • Education
  • Artificial Intelligence
  • Computer Networks and Communications
  • Engineering (miscellaneous)

Cite this

Wang, Y., Fang, H., Cheng, C., & Jin, Q. (2019). TSP: Truthful Grading-Based Strategyproof Peer Selection for MOOCs. In M. J. W. Lee, S. Nikolic, G. K. W. Wong, J. Shen, M. Ros, L. C. U. Lei, & N. Venkatarayalu (Eds.), Proceedings of 2018 IEEE International Conference on Teaching, Assessment, and Learning for Engineering, TALE 2018 (pp. 679-684). [8615340] (Proceedings of 2018 IEEE International Conference on Teaching, Assessment, and Learning for Engineering, TALE 2018). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/TALE.2018.8615340

TSP : Truthful Grading-Based Strategyproof Peer Selection for MOOCs. / Wang, Yufeng; Fang, Hui; Cheng, Chonghu; Jin, Qun.

Proceedings of 2018 IEEE International Conference on Teaching, Assessment, and Learning for Engineering, TALE 2018. ed. / Mark J.W. Lee; Sasha Nikolic; Gary K.W. Wong; Jun Shen; Montserrat Ros; Leon C. U. Lei; Neelakantam Venkatarayalu. Institute of Electrical and Electronics Engineers Inc., 2019. p. 679-684 8615340 (Proceedings of 2018 IEEE International Conference on Teaching, Assessment, and Learning for Engineering, TALE 2018).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Wang, Y, Fang, H, Cheng, C & Jin, Q 2019, TSP: Truthful Grading-Based Strategyproof Peer Selection for MOOCs. in MJW Lee, S Nikolic, GKW Wong, J Shen, M Ros, LCU Lei & N Venkatarayalu (eds), Proceedings of 2018 IEEE International Conference on Teaching, Assessment, and Learning for Engineering, TALE 2018., 8615340, Proceedings of 2018 IEEE International Conference on Teaching, Assessment, and Learning for Engineering, TALE 2018, Institute of Electrical and Electronics Engineers Inc., pp. 679-684, 2018 IEEE International Conference on Teaching, Assessment, and Learning for Engineering, TALE 2018, Wollongong, Australia, 18/12/4. https://doi.org/10.1109/TALE.2018.8615340
Wang Y, Fang H, Cheng C, Jin Q. TSP: Truthful Grading-Based Strategyproof Peer Selection for MOOCs. In Lee MJW, Nikolic S, Wong GKW, Shen J, Ros M, Lei LCU, Venkatarayalu N, editors, Proceedings of 2018 IEEE International Conference on Teaching, Assessment, and Learning for Engineering, TALE 2018. Institute of Electrical and Electronics Engineers Inc. 2019. p. 679-684. 8615340. (Proceedings of 2018 IEEE International Conference on Teaching, Assessment, and Learning for Engineering, TALE 2018). https://doi.org/10.1109/TALE.2018.8615340
Wang, Yufeng ; Fang, Hui ; Cheng, Chonghu ; Jin, Qun. / TSP : Truthful Grading-Based Strategyproof Peer Selection for MOOCs. Proceedings of 2018 IEEE International Conference on Teaching, Assessment, and Learning for Engineering, TALE 2018. editor / Mark J.W. Lee ; Sasha Nikolic ; Gary K.W. Wong ; Jun Shen ; Montserrat Ros ; Leon C. U. Lei ; Neelakantam Venkatarayalu. Institute of Electrical and Electronics Engineers Inc., 2019. pp. 679-684 (Proceedings of 2018 IEEE International Conference on Teaching, Assessment, and Learning for Engineering, TALE 2018).
@inproceedings{b0c9d04514dc4428a5744a61d11314f5,
title = "TSP: Truthful Grading-Based Strategyproof Peer Selection for MOOCs",
abstract = "Considering that the challenges in using peer grading to select the best-k students in MOOCs (massive open online courses) are twofold: first is strategyproof, i.e., students should not benefit by untruthfully reporting valuations; second, instead of exerting (costly) efforts to evaluate, students may randomly provide (or just guess) the evaluations on other peers. This paper proposes a truthful grading-based strategyproof peer selection scheme for MOOCs, TSP. Specifically, all students are partitioned into same-size clusters, and each student only evaluates students in other clusters. Moreover, peer prediction mechanism was utilized to motivate each student to truthfully report their gradings through comparing their reports with other peer students who conduct same evaluation tasks. The theoretical analysis and simulation results show that TSP can both stimulate students to truthfully reporting their gradings, and meanwhile select best k-students in strategyproof way.",
keywords = "massive online open course (MOOC), peer selection, strategyproof, truthful grading",
author = "Yufeng Wang and Hui Fang and Chonghu Cheng and Qun Jin",
year = "2019",
month = "1",
day = "16",
doi = "10.1109/TALE.2018.8615340",
language = "English",
series = "Proceedings of 2018 IEEE International Conference on Teaching, Assessment, and Learning for Engineering, TALE 2018",
publisher = "Institute of Electrical and Electronics Engineers Inc.",
pages = "679--684",
editor = "Lee, {Mark J.W.} and Sasha Nikolic and Wong, {Gary K.W.} and Jun Shen and Montserrat Ros and Lei, {Leon C. U.} and Neelakantam Venkatarayalu",
booktitle = "Proceedings of 2018 IEEE International Conference on Teaching, Assessment, and Learning for Engineering, TALE 2018",

}

TY - GEN

T1 - TSP

T2 - Truthful Grading-Based Strategyproof Peer Selection for MOOCs

AU - Wang, Yufeng

AU - Fang, Hui

AU - Cheng, Chonghu

AU - Jin, Qun

PY - 2019/1/16

Y1 - 2019/1/16

N2 - Considering that the challenges in using peer grading to select the best-k students in MOOCs (massive open online courses) are twofold: first is strategyproof, i.e., students should not benefit by untruthfully reporting valuations; second, instead of exerting (costly) efforts to evaluate, students may randomly provide (or just guess) the evaluations on other peers. This paper proposes a truthful grading-based strategyproof peer selection scheme for MOOCs, TSP. Specifically, all students are partitioned into same-size clusters, and each student only evaluates students in other clusters. Moreover, peer prediction mechanism was utilized to motivate each student to truthfully report their gradings through comparing their reports with other peer students who conduct same evaluation tasks. The theoretical analysis and simulation results show that TSP can both stimulate students to truthfully reporting their gradings, and meanwhile select best k-students in strategyproof way.

AB - Considering that the challenges in using peer grading to select the best-k students in MOOCs (massive open online courses) are twofold: first is strategyproof, i.e., students should not benefit by untruthfully reporting valuations; second, instead of exerting (costly) efforts to evaluate, students may randomly provide (or just guess) the evaluations on other peers. This paper proposes a truthful grading-based strategyproof peer selection scheme for MOOCs, TSP. Specifically, all students are partitioned into same-size clusters, and each student only evaluates students in other clusters. Moreover, peer prediction mechanism was utilized to motivate each student to truthfully report their gradings through comparing their reports with other peer students who conduct same evaluation tasks. The theoretical analysis and simulation results show that TSP can both stimulate students to truthfully reporting their gradings, and meanwhile select best k-students in strategyproof way.

KW - massive online open course (MOOC)

KW - peer selection

KW - strategyproof

KW - truthful grading

UR - http://www.scopus.com/inward/record.url?scp=85062085811&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85062085811&partnerID=8YFLogxK

U2 - 10.1109/TALE.2018.8615340

DO - 10.1109/TALE.2018.8615340

M3 - Conference contribution

T3 - Proceedings of 2018 IEEE International Conference on Teaching, Assessment, and Learning for Engineering, TALE 2018

SP - 679

EP - 684

BT - Proceedings of 2018 IEEE International Conference on Teaching, Assessment, and Learning for Engineering, TALE 2018

A2 - Lee, Mark J.W.

A2 - Nikolic, Sasha

A2 - Wong, Gary K.W.

A2 - Shen, Jun

A2 - Ros, Montserrat

A2 - Lei, Leon C. U.

A2 - Venkatarayalu, Neelakantam

PB - Institute of Electrical and Electronics Engineers Inc.

ER -