An Empirical Examination of the Effect of Guessing on Vocabulary Size Test Scores
Stuart McLean (a), Brandon Kramer (b) and Jeffrey Stewart (c)
(a) Kansai University; (b) Momoyama University; (c) Kyushu Sangyo University
doi: http://dx.doi.org/10.7820/vli.v04.1.mclean.et.al
Download this article (pdf)

Abstract
The Vocabulary Size Test (VST) was created to provide a reliable estimate
of a second language learner’s written receptive vocabulary size, measuring
from the most frequent fourteen 1,000 word families of the spoken
subsection of the British National Corpus.While some have recommended
that users should limit the amount of the test taken to only slightly
above a student’s level, others argue that learners should take every
level of the test. However, this raises concerns that correct responses
on lower frequency levels could largely be attributed to guesses rather
than vocabulary knowledge. In this paper we analyze a data set of
3,373 Japanese university students’ responses to the first eight levels of
the original VST under the 3PL model, in order to determine the
minimum expected score on the test for learners of low ability, examine
the proportion of low-level students’ scores on the lowest frequency level
tested that can be attributed to guessing under the 3PL model, and
conduct a model fit comparison to determine whether the 3PL model
offers a significantly better description of the data than the Rasch model.
The results indicate that a substantial portion of lower level learners’
scores on items testing low-frequency words can be attributed to guessing
and support the position that students should not sit every level of the
test. The authors recommend using the results of the 3PL analysis in
order to determine which sections of the test learners of different
proficiency levels should sit.

Citation
McLean et al. (2015). An empirical examination of the effect of guessing on vocabulary size test scores. Vocabulary Learning and Instruction, 4 (1), 26-35. doi: 10.7820/vli.v04.1.mclean.et.al