Rater Judgments and Word Difficulty: Conceptualizing the Substantive Validity of the VST
Derek N. Canninga, Stuart McLeanb, and Joseph P. Vittac
aSeigakuen University; bMomoyama Gakuin University; bKyushu University
doi: https://doi.org/10.7820/vli.v11.2.canning
Download this article (pdf)
Abstract
The substantive component of construct validity requires a confrontation between empirical test results and content relevance. The Vocabulary Size Test (VST) has been extensively validated in terms of empirical results. Less is known, however, about expert judgments of content relevance. The VST was constructed and validated according to the principle that frequency underlies vocabulary acquisition. This does not mean, however, that the two are equivalent. To better understand the construct of word difficulty as it is measured in VSTs, the results of two Rasch analyses, one on the VST, the other a Multi Facet Rasch Model (MFRM) of a questionnaire distributed to education practitioners were correlated (n = 80, r = 0.67, p < 0.001, 95% CI = [0.53, 0.77]). Semi-structured interviews were then conducted to explore how practitioners understand the concept of word difficulty. Findings indicate that word difficulty is understood to encompass more than frequency, validating recent research into the predictive power of lexical sophistication variables.
Citation
Canning, D. N., McLean, S., and Vitta, J. P. (2022). Rater judgments and word difficulty: Conceptualizing the substantive validity of the VST. Vocabulary Learning and Instruction, 11(2), 30–37. https://doi.org/10.7820/vli.v11.2.canning