TY - JOUR
T1 - A comprehensive review of Rasch measurement in language assessment
T2 - Recommendations and guidelines for research
AU - Aryadoust, Vahid
AU - Ng, Li Ying
AU - Sayama, Hiroki
N1 - Publisher Copyright:
© The Author(s) 2020.
PY - 2021/1
Y1 - 2021/1
N2 - Over the past decades, the application of Rasch measurement in language assessment has gradually increased. In the present study, we coded 215 papers using Rasch measurement published in 21 applied linguistics journals for multiple features. We found that seven Rasch models and 23 software packages were adopted in these papers, with many-facet Rasch measurement (n = 100) and Facets (n = 113) being the most frequently used Rasch model and software, respectively. Significant differences were detected between the number of papers that applied Rasch measurement to different language skills and components, with writing (n = 63) and grammar (n = 12) being the most and least frequently investigated, respectively. In addition, significant differences were found between the number of papers reporting person separation (n = 73, not reported: n = 142) and item separation (n = 59, not reported: n = 156) and those that did not. An alarming finding was how few papers reported unidimensionality check (n = 57 vs 158) and local independence (n = 19 vs 196). Finally, a multilayer network analysis revealed that research involving Rasch measurement has created two major discrete communities of practice (clusters), which can be characterized by features such as language skills, the Rasch models used, and the reporting of item reliability/separation vs person reliability/separation. Cluster 1 was accordingly labelled the production and performance cluster, whereas cluster 2 was labelled the perception and language elements cluster. Guidelines and recommendations for analyzing unidimensionality, local independence, data-to-model fit, and reliability in Rasch model analysis are proposed.
AB - Over the past decades, the application of Rasch measurement in language assessment has gradually increased. In the present study, we coded 215 papers using Rasch measurement published in 21 applied linguistics journals for multiple features. We found that seven Rasch models and 23 software packages were adopted in these papers, with many-facet Rasch measurement (n = 100) and Facets (n = 113) being the most frequently used Rasch model and software, respectively. Significant differences were detected between the number of papers that applied Rasch measurement to different language skills and components, with writing (n = 63) and grammar (n = 12) being the most and least frequently investigated, respectively. In addition, significant differences were found between the number of papers reporting person separation (n = 73, not reported: n = 142) and item separation (n = 59, not reported: n = 156) and those that did not. An alarming finding was how few papers reported unidimensionality check (n = 57 vs 158) and local independence (n = 19 vs 196). Finally, a multilayer network analysis revealed that research involving Rasch measurement has created two major discrete communities of practice (clusters), which can be characterized by features such as language skills, the Rasch models used, and the reporting of item reliability/separation vs person reliability/separation. Cluster 1 was accordingly labelled the production and performance cluster, whereas cluster 2 was labelled the perception and language elements cluster. Guidelines and recommendations for analyzing unidimensionality, local independence, data-to-model fit, and reliability in Rasch model analysis are proposed.
KW - Fit
KW - Rasch measurement
KW - language assessment
KW - local independence
KW - modularity maximization method
KW - network analysis
KW - reliability and separation
KW - unidimensionality
UR - http://www.scopus.com/inward/record.url?scp=85087713648&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85087713648&partnerID=8YFLogxK
U2 - 10.1177/0265532220927487
DO - 10.1177/0265532220927487
M3 - Article
AN - SCOPUS:85087713648
VL - 38
SP - 6
EP - 40
JO - Language Testing
JF - Language Testing
SN - 0265-5322
IS - 1
ER -