SSPA: an effective semi-supervised peer assessment method for large scale MOOCs

Yufeng Wang, Hui Fang, Qun Jin, Jianhua Ma

Research output: Contribution to journalArticle

Abstract

Peer assessment has become a primary solution to the challenge of evaluating a large number of students in Massive Open Online Courses (MOOCs). In peer assessment, all students need to evaluate a subset of other students’ assignments, and then these peer grades are aggregated to predict a final score for each student. Unfortunately, due to the lack of grading experience or the heterogeneous grading abilities, students may introduce unintentional deviations in the evaluation. This paper proposes and implements a semi-supervised peer assessment method (SSPA) that incorporates a small number of teacher’s gradings as ground truth, and uses them to externally calibrate the procedure of aggregating peer grades. Specifically, each student’s grading ability is directly (if students have common peer assessments with teacher) or indirectly (if students have no common peer assessments with teacher) measured with the grading similarity between the student and teacher. Then, SSPA utilizes the weighted aggregation of peer grades to infer the final score of each student. Based on both real dataset and synthetic datasets, the experimental results illustrate that SSPA performs better than the existing methods.

Original languageEnglish
JournalInteractive Learning Environments
DOIs
Publication statusAccepted/In press - 2019 Jan 1
Externally publishedYes

Fingerprint

Students
grading
student
teacher
ability
aggregation
Agglomeration
lack
evaluation
experience

Keywords

  • accuracy
  • grading ability
  • Massive open online courses (MOOCs)
  • peer grading
  • semi-supervised learning

ASJC Scopus subject areas

  • Education
  • Computer Science Applications

Cite this

SSPA : an effective semi-supervised peer assessment method for large scale MOOCs. / Wang, Yufeng; Fang, Hui; Jin, Qun; Ma, Jianhua.

In: Interactive Learning Environments, 01.01.2019.

Research output: Contribution to journalArticle

@article{f4a11f3f798c413881d16a91619d98d2,
title = "SSPA: an effective semi-supervised peer assessment method for large scale MOOCs",
abstract = "Peer assessment has become a primary solution to the challenge of evaluating a large number of students in Massive Open Online Courses (MOOCs). In peer assessment, all students need to evaluate a subset of other students’ assignments, and then these peer grades are aggregated to predict a final score for each student. Unfortunately, due to the lack of grading experience or the heterogeneous grading abilities, students may introduce unintentional deviations in the evaluation. This paper proposes and implements a semi-supervised peer assessment method (SSPA) that incorporates a small number of teacher’s gradings as ground truth, and uses them to externally calibrate the procedure of aggregating peer grades. Specifically, each student’s grading ability is directly (if students have common peer assessments with teacher) or indirectly (if students have no common peer assessments with teacher) measured with the grading similarity between the student and teacher. Then, SSPA utilizes the weighted aggregation of peer grades to infer the final score of each student. Based on both real dataset and synthetic datasets, the experimental results illustrate that SSPA performs better than the existing methods.",
keywords = "accuracy, grading ability, Massive open online courses (MOOCs), peer grading, semi-supervised learning",
author = "Yufeng Wang and Hui Fang and Qun Jin and Jianhua Ma",
year = "2019",
month = "1",
day = "1",
doi = "10.1080/10494820.2019.1648299",
language = "English",
journal = "Interactive Learning Environments",
issn = "1049-4820",
publisher = "Routledge",

}

TY - JOUR

T1 - SSPA

T2 - an effective semi-supervised peer assessment method for large scale MOOCs

AU - Wang, Yufeng

AU - Fang, Hui

AU - Jin, Qun

AU - Ma, Jianhua

PY - 2019/1/1

Y1 - 2019/1/1

N2 - Peer assessment has become a primary solution to the challenge of evaluating a large number of students in Massive Open Online Courses (MOOCs). In peer assessment, all students need to evaluate a subset of other students’ assignments, and then these peer grades are aggregated to predict a final score for each student. Unfortunately, due to the lack of grading experience or the heterogeneous grading abilities, students may introduce unintentional deviations in the evaluation. This paper proposes and implements a semi-supervised peer assessment method (SSPA) that incorporates a small number of teacher’s gradings as ground truth, and uses them to externally calibrate the procedure of aggregating peer grades. Specifically, each student’s grading ability is directly (if students have common peer assessments with teacher) or indirectly (if students have no common peer assessments with teacher) measured with the grading similarity between the student and teacher. Then, SSPA utilizes the weighted aggregation of peer grades to infer the final score of each student. Based on both real dataset and synthetic datasets, the experimental results illustrate that SSPA performs better than the existing methods.

AB - Peer assessment has become a primary solution to the challenge of evaluating a large number of students in Massive Open Online Courses (MOOCs). In peer assessment, all students need to evaluate a subset of other students’ assignments, and then these peer grades are aggregated to predict a final score for each student. Unfortunately, due to the lack of grading experience or the heterogeneous grading abilities, students may introduce unintentional deviations in the evaluation. This paper proposes and implements a semi-supervised peer assessment method (SSPA) that incorporates a small number of teacher’s gradings as ground truth, and uses them to externally calibrate the procedure of aggregating peer grades. Specifically, each student’s grading ability is directly (if students have common peer assessments with teacher) or indirectly (if students have no common peer assessments with teacher) measured with the grading similarity between the student and teacher. Then, SSPA utilizes the weighted aggregation of peer grades to infer the final score of each student. Based on both real dataset and synthetic datasets, the experimental results illustrate that SSPA performs better than the existing methods.

KW - accuracy

KW - grading ability

KW - Massive open online courses (MOOCs)

KW - peer grading

KW - semi-supervised learning

UR - http://www.scopus.com/inward/record.url?scp=85070318727&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85070318727&partnerID=8YFLogxK

U2 - 10.1080/10494820.2019.1648299

DO - 10.1080/10494820.2019.1648299

M3 - Article

AN - SCOPUS:85070318727

JO - Interactive Learning Environments

JF - Interactive Learning Environments

SN - 1049-4820

ER -