Differential Effects of Generative AI Editing on the Language Quality of Essays across Proficiency Levels

Abstract

While numerous studies have explored the affordances of generative artificial intelligence (AI) for writing, few have empirically examined its automated language editing capabilities. To address this gap, this paper examines this feature and its potential to level the playing field in academic writing across different proficiency levels. In particular, we investigate the extent to which ChatGPT enhances the linguistic quality of essays and reduces disparities in language quality among students with varying proficiency levels. A total of 200 essays (50 from each proficiency level, namely A2-0, B1-1, B1-2, and B2-0) were selected from ICNALE, ensuring equal representation across 10 Asian countries. The results indicate that ChatGPT significantly improves language quality in six of the seven assessed indicators, with these improvements occurring consistently across all proficiency levels. Furthermore, the findings demonstrate its ability to minimise disparities in language quality among essays written by students with different proficiency levels. The study concludes with a discussion of the implications of these findings for writing instruction, assessment, and research.

Keywords

Full text

How to cite