The great validity debate

The great validity debateValidity is the most important term in the educational and psychological measurement lexicon.Measurement professionals are generally happy to agree about that.What they are less happy to agree about is what the term ought to mean.North American measurement professionals have negotiated a kind of consensus on this thorny issue, through the definition and description of validity in the Standards for Educational and Psychological Testing (American Educational Research Association, American Psychological Association, and National Council on Measurement in Education [AERA, APA, and NCME], 2014).Yet, the status of this consensus is unclear, given continuing debate amongst scholars, and given the fact that all sorts of different definitions and descriptions can be found on the websites of measurement organisations within the USA and elsewhere, and within the pages of prominent textbooks.In short, there is no widespread professional consensus concerning the best way to use the term.In 1997, Linda Crocker penned an editorial for the North American National Council on Measurement in Education (NCME) publication, Educational Measurement: Issues and Practice, entitled: The Great Validity Debate (Crocker, 1997, p. 4).Her editorial introduced a special issue of the journal devoted to a controversy which she described as having been 'brewing in psychometric circles' since the late 1980s.It concerned the significance of consequences for the concept of validity and pivoted, for many, around the issue of whether validation should be 'regarded as a scientific, empirical enterprise or a sociopolitical process as well.'She suggested that: 'the prevailing argument in this debate will shape the nature of measurement practice and professional preparation for years to come.'Well over a decade later, Newton and Shaw undertook an extensive review of the literature on validity, to provide a foundation for an introductory overview of the concept of validity (Newton & Shaw, 2014).Their research led them to conclude that no position in this debate had yet prevailed.Not only was the controversy over consequences still raging (e.g.Cizek, 2012), new controversies had arisen, including debate over the relationship between validity and truth (e.g.Borsboom & Markus, 2013;Borsboom, Mellenbergh, & van Heerden, 2004; Kane, 2013a Kane, , 2013b)).In an attempt to explore potential for resolving these debates, Newton and Shaw organised a coordinated session at the 2014 NCME Annual Meeting, in Philadelphia, entitled: What is the Best Way to Use the Term 'Validity'?The six focal papers at the heart of this new special issue began life in that session.Lorrie Shepard, a contributor to the original special issue (edited by Linda Crocker in 1997) contributed a 'reflective overview' to the session and agreed to provide a similar contribution in this new special issue.In the spirit of facilitating debate, we decided to introduce an element of peer commentary to the following pages.The six focal papers were prepared simultaneously and then circulated to a group of leading measurement professionals for