Univariate generalizability theory in language assessment

Yasuyo Sawaki, Xiaoming Xi

Research output: Chapter in Book/Report/Conference proceedingChapter

2 Citations (Scopus)

Abstract

This chapter provides a brief overview of generalizability theory (G theory), a framework for modeling different sources of score variability contributing to measurement inconsistency. First, this chapter introduces basic concepts of G theory in relation to characteristics of classical test theory (CTT), highlighting a notable strength of G theory in allowing for the analysis of selected and constructed item responses for both norm-referenced and criterion-referenced assessments. Then a step-by-step demonstration is provided to illustrate how a dataset could be analyzed using G theory to address specific research questions concerning measurement consistency. Major considerations for determining rating designs are also discussed so as to ensure that a G theory analysis generates meaningful and interpretable results. As an illustrative example, the process of analyzing a language performance assessment dataset by using a univariate G theory analysis software, GENOVA, is described. Results of the analysis are discussed along with key considerations in designing and conducting language assessment studies using G theory.

Original languageEnglish
Title of host publicationQuantitative Data Analysis for Language Assessment Volume I
Subtitle of host publicationFundamental Techniques
PublisherTaylor and Francis
Pages30-53
Number of pages24
ISBN (Electronic)9781351741231
DOIs
Publication statusPublished - 2019 Jan 1

ASJC Scopus subject areas

  • Social Sciences(all)
  • Psychology(all)
  • Arts and Humanities(all)

Fingerprint

Dive into the research topics of 'Univariate generalizability theory in language assessment'. Together they form a unique fingerprint.

Cite this