News

ISSN Number

2632-6779 (Print)  

2633-6898 (Online)

Abstracting/Indexing/Listing

Scopus

Ulrich’s Periodicals Directory (ProQuest)

MLA International Bibliography

MLA Directory of Periodicals

Directory of Open Access Journals (DOAJ)

QOAM (Quality Open Access Market)

British National Bibliography

WAC Clearinghouse Journal Listings

EBSCO Education

ICI Journals Master List

ERIH PLUS

CNKI Scholar

Gale-Cengage

WorldCat

Crossref

Baidu Scholar

British Library

J-Gate

ROAD

BASE

Publons

Google Scholar

Semantic Scholar

ORE Directory

TIRF

China National Center for Philosophy and Social Sciences Documentation

 

Home Journal Index 2024-2

Validating Pecorari, Shaw and Malmström's (2019) Academic Vocabulary Test – Form 1: Evidence from Rasch-based Analyses and Retrospective Focus-group Interviews

Download Full PDF

Chi Duc Nguyen

Hanh Thi Hoang

VNU University of Languages and International Studies

Vietnam National University – Hanoi, Vietnam

 

Abstract

This study aimed to assess the construct validity of Pecorari, Shaw and Malmström’s (2019) Academic Vocabulary Test – Form 1 (AVT1). To this end, it first employed Rasch-based statistical evidence generated from the test responses of 989 high-school and university students in Vietnam to inspect five major aspects of Messickian construct validity: Content, Substantive, Structural, Generalizability and External (Messick,1995). It then moved on to thematically analyze the data collected from the follow-up focus-group interviews with 50 students randomly selected from the test-takers to detect any emerged patterns in their actual engagement with the test. Results from Rasch-based analyses showed that AVT1 could sufficiently measure the target ability – receptive academic vocabulary breadth – of an overwhelming 889 out of the 989 test-takers (90%) and there were seven statistically distinct groups of item difficulty in the empirical item hierarchy. In general, test items and test-takers performed as predicted by a priori hypotheses and displayed good fit to the Rasch model. Principal Component Analysis indicated that the test items formed a fundamentally unidimensional construct, suggesting that the test only measured one meaningful dimension, presumably the receptive academic vocabulary breadth. The evidence for the invariance in item calibration and person measure as well as the test’s external reliability enabled the generalization of score properties and interpretations across populations, settings, and tasks. Results from the interview data analyses revealed that AVT1 did allow some room for guessing effects, which was predicted by the test developers. These findings altogether suggest that this test can be a useful tool to gauge receptive academic vocabulary breadth.

 

Keywords

Academic Vocabulary Test, breadth of academic vocabulary knowledge, test validation, Messickian construct validity, Rasch-based analysis