A course evaluation model using paired comparisons to integrate importance of rating criteria and students' ratings of teaching: Comparison of students' and teachers' evaluations

Kazuya Ikehara, Hideki Toyoda

Research output: Contribution to journalArticle

1 Citation (Scopus)

Abstract

The use of course evaluations by students has increased over the last few decades. However, because some teachers have made negative comments about this procedure, classes should be evaluated from both the students' and the teacher's point of view. In addition, rating scales make it difficult to deal with data reflecting the strength of the students' responses. The purpose of the present study was to propose a model for course evaluation that extends the Group Analytic Hierarchy Process by structural equation modeling (Group AHP by SEM, or GAS; Toyoda, Yonemura, & Saito, 2004, in Japanese), so as to remove effects of the strength of the students' ratings and make comparisons between classes possible. The proposed model can be used to estimate an overall evaluation that considers differences in importance of criteria between students and teachers. In the present study, university students from 23 classes and their teachers completed questionnaires, after which the proposed model was applied to the data. The results suggested a difference in the importance of criteria between the students and the teachers, but little difference was found in the overall evaluations.

Original languageEnglish
Pages (from-to)48-59
Number of pages12
JournalJapanese Journal of Educational Psychology
Volume60
Issue number1
DOIs
Publication statusPublished - 2012

Fingerprint

Matched-Pair Analysis
Teaching
rating
Students
teacher
evaluation
student
rating scale
Group
questionnaire
university

Keywords

  • Course evaluations by students
  • Group analytic hierarchy process (group AHP)
  • Markov chain monte carlo method
  • Strength of students ratings

ASJC Scopus subject areas

  • Developmental and Educational Psychology
  • Education

Cite this

@article{e97680db5ebf49bb8740fa1eff757e4a,
title = "A course evaluation model using paired comparisons to integrate importance of rating criteria and students' ratings of teaching: Comparison of students' and teachers' evaluations",
abstract = "The use of course evaluations by students has increased over the last few decades. However, because some teachers have made negative comments about this procedure, classes should be evaluated from both the students' and the teacher's point of view. In addition, rating scales make it difficult to deal with data reflecting the strength of the students' responses. The purpose of the present study was to propose a model for course evaluation that extends the Group Analytic Hierarchy Process by structural equation modeling (Group AHP by SEM, or GAS; Toyoda, Yonemura, & Saito, 2004, in Japanese), so as to remove effects of the strength of the students' ratings and make comparisons between classes possible. The proposed model can be used to estimate an overall evaluation that considers differences in importance of criteria between students and teachers. In the present study, university students from 23 classes and their teachers completed questionnaires, after which the proposed model was applied to the data. The results suggested a difference in the importance of criteria between the students and the teachers, but little difference was found in the overall evaluations.",
keywords = "Course evaluations by students, Group analytic hierarchy process (group AHP), Markov chain monte carlo method, Strength of students ratings",
author = "Kazuya Ikehara and Hideki Toyoda",
year = "2012",
doi = "10.5926/jjep.60.48",
language = "English",
volume = "60",
pages = "48--59",
journal = "Japanese Journal of Educational Psychology",
issn = "0021-5015",
publisher = "Japanese Association of Educational Psychology",
number = "1",

}

TY - JOUR

T1 - A course evaluation model using paired comparisons to integrate importance of rating criteria and students' ratings of teaching

T2 - Comparison of students' and teachers' evaluations

AU - Ikehara, Kazuya

AU - Toyoda, Hideki

PY - 2012

Y1 - 2012

N2 - The use of course evaluations by students has increased over the last few decades. However, because some teachers have made negative comments about this procedure, classes should be evaluated from both the students' and the teacher's point of view. In addition, rating scales make it difficult to deal with data reflecting the strength of the students' responses. The purpose of the present study was to propose a model for course evaluation that extends the Group Analytic Hierarchy Process by structural equation modeling (Group AHP by SEM, or GAS; Toyoda, Yonemura, & Saito, 2004, in Japanese), so as to remove effects of the strength of the students' ratings and make comparisons between classes possible. The proposed model can be used to estimate an overall evaluation that considers differences in importance of criteria between students and teachers. In the present study, university students from 23 classes and their teachers completed questionnaires, after which the proposed model was applied to the data. The results suggested a difference in the importance of criteria between the students and the teachers, but little difference was found in the overall evaluations.

AB - The use of course evaluations by students has increased over the last few decades. However, because some teachers have made negative comments about this procedure, classes should be evaluated from both the students' and the teacher's point of view. In addition, rating scales make it difficult to deal with data reflecting the strength of the students' responses. The purpose of the present study was to propose a model for course evaluation that extends the Group Analytic Hierarchy Process by structural equation modeling (Group AHP by SEM, or GAS; Toyoda, Yonemura, & Saito, 2004, in Japanese), so as to remove effects of the strength of the students' ratings and make comparisons between classes possible. The proposed model can be used to estimate an overall evaluation that considers differences in importance of criteria between students and teachers. In the present study, university students from 23 classes and their teachers completed questionnaires, after which the proposed model was applied to the data. The results suggested a difference in the importance of criteria between the students and the teachers, but little difference was found in the overall evaluations.

KW - Course evaluations by students

KW - Group analytic hierarchy process (group AHP)

KW - Markov chain monte carlo method

KW - Strength of students ratings

UR - http://www.scopus.com/inward/record.url?scp=84873902872&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84873902872&partnerID=8YFLogxK

U2 - 10.5926/jjep.60.48

DO - 10.5926/jjep.60.48

M3 - Article

AN - SCOPUS:84873902872

VL - 60

SP - 48

EP - 59

JO - Japanese Journal of Educational Psychology

JF - Japanese Journal of Educational Psychology

SN - 0021-5015

IS - 1

ER -