Validating the English Language Exam for university entry in Shanghai, China

Abstract

The University Entrance Exam (UEE) plays a vital role in the Chinese education system. Yet despite it being a high-stakes exam for over nine million senior high school students each year, there is little research demonstrating that the UEE is of an appropriate quality. This is partly due to the very nature of the UEE. Little information about it is publicly available and there has been no significant validation research.

This study provides the first comprehensive attempt to validate the University Entrance English Exam. The study is focused on the Shanghai English Language Exam, which is one of the principal tests for university entry in Shanghai. There were two phases in the study. The first was a qualitative content analysis, which compared the past 10 years of UEE papers with English Curriculum Standards, and with Bachman and Palmer’s model of language ability, in both instances with the aim of identifying what the UEE is actually testing. The second phase conducted empirical studies, namely mock exams, classroom observations and interviews, and think-aloud tasks. Six test qualities – reliability, construct validity, authenticity, interactiveness, impact, and practicality – were measured in these studies. The mock exam participants were first-year university students, whereas classroom observations, interviews and think-aloud task participants were senior high school students.

In the first phase the study found that by English Curriculum Standards, the UEE adequately tested language skills and language knowledge but failed to test affect and attitude, learning strategies, and cultural understanding by those same standards. Measured against Bachman and Palmer’s model of language ability, the UEE focused on grammatical and textual knowledge and strategic competence but not functional and sociolinguistic knowledge. In the second phase the study found variable responses, with impact registering highest and authenticity registering lowest.

The UEE has been the subject of much controversy, but this study, while critical of it in a number of aspects, finds that overall its performance is above average and better than many of its critics would concede. Given the controversy, the study has important implications for stakeholders, particularly curriculum developers, examination designers, universities and schools, and teachers and students, allowing them to make far better informed decisions concerning this high-stakes exam than has previously been possible.