Techniques for Monitoring the Comparability of Examination Standards

Comparable examinations have to be at the same standard. But what do people mean by ‘examination standard’ and what kinds of comparability are expected? How is evidence to be gathered about these types of comparability and are all of these approaches valid? This chapter outlines different definitions of examination comparability used in England by academics and the expectations of the media and general public. The purposes to which assessment results are put are discussed, as the alternative conceptions of examination comparability are linked to the uses of the assessment results. Given that there are different approaches, some commentators have proposed that we should select a single definition of examination standards and stick to it, so that the system is clearer and false expectations are not raised about what the examination system can realistically deliver. Whether a particular definition of examination standards can be prioritised above others is considered, as well as the implications of so doing. 1 What does society mean by examination standards? The word standard is used in a multiplicity of ways, leading to a great deal of confusion. As an example, Aldrich (2000) notes that the Department for Education and Employment White Paper Excellence in Schools (1997) has the following subheading in one section, ‘Raising standards: our top priority’. Aldrich points out that there is more than one way in which educational standards can be raised. To take a sporting analogy, high-jump standards can be improved by increasing the height of the bar that people have to jump over, or by raising the number of people who can jump over the bar. Educationally, raising standards can mean expecting more of students or expecting more students to be able to demonstrate performance at a given level. Herein lies the root of a problem. Without being explicit about exactly what is meant by examination standards, many commentators are critical. Whilst some of the definitions of examination standards used are consistent with academic definitions (see later), some are ruled out by assessment specialists as too simplistic or not part of the standard-setting process. Let Techniques for monitoring the comparability of examination standards, edited by: Paul Newton, Jo-Anne Baird, Harvey Goldstein, Helen Patrick and Peter Tymms QCAChap4CB:QCAChap4CB 11/12/2007 17:36 Page 124