Measuring the quality of election administration

Abstract How do we distinguish between a 'genuine', 'free and fair', or 'legitimate' election and an election that is something less? In this article, we offer an answer to this vexing question: the Election Administration Systems Index (EASI). EASI is a practical, transparent, and sustainable tool for measuring the quality of elections in the developing world. The following pages describe the current limitations in measuring election quality, detail the EASI approach, and provide a comparative analysis of the results of its pilot implementation. EASI scores are drawn from a survey of experts on elections in the target country following a recent nationwide election. The analytical framework is comprised of three electoral dimensions: participation, competition, and integrity of the process. We also divide these dimensions temporally according to the electoral cycle: either pre-election, during the election, or post-election. The final product is a set of six primary scores displayed across dimension and time. By aggregating the survey data in this fashion, we provide for a nuanced assessment of an election by each dimension and across the cycle. As our pilot results demonstrate, EASI is a diagnostic tool for identifying electoral strengths and weaknesses and serves well for comparative assessments. Keywords: election administrationelection quality assessmentsgovernanceelectoral observationlegitimacyindicatorsdemocratizationdemocracy promotion Acknowledgements We are indebted to a good number of election scholars, practitioners, and pilot participants whose expertise and suggestions contributed in important ways to the development of EASI. Our colleague, Karen Buerkle, was involved early on in the conceptualization of an elections index. We also thank the two anonymous reviewers of the manuscript for their helpful comments. RTI International, a non-profit research and development institute based in Research Triangle Park, North Carolina, sponsored EASI's development. EASI is an RTI trademark. Notes Collier and Levitsky, 'Democracy with Adjectives'. The institutes are the National Democratic Institute, or NDI, and the International Republican Institute, or IRI. Bjornlund, Beyond Free and Fair, 57–73. See, for example, Rae, The Political Consequences of Electoral Laws; Lijphart, 'The Political Consequences of Electoral Laws'; Powell, Elections as Instruments of Democracy; and Farrell, Electoral Systems. See, for example, Estok, Nevitte, and Cowan, 'The Quick Count and Electoral Observation'; Goodwin-Gill, Free and Fair Elections; International IDEA, 'Code of Conduct for the Ethical and Professional Administration of Elections'; Munck, Measuring Democracy; and Trebilcock and Chitalkar, 'From Normative to Substantive Democracy'. See, for example, Alvarez, Hall, and Hyde, Election Fraud; Alvarez and Hall, 'Building Secure and Transparent Elections through Standard Operating Procedures'; Birch, 'Project on Explaining Electoral Malpractice in New and Semi-Democracies'; and Bjornlund, Beyond Free and Fair. Massicotte, Blais, and Yoshinaka, Establishing the Rules of the Game. International IDEA, International Electoral Standards, 6. Bjornlund, Beyond Free and Fair, 120–1. Mozaffar and Schedler, 'The Comparative Study of Electoral Governance', 17–20. Elklit and Reynolds, 'A Framework for the Systematic Study of Election Quality'. Kelley and Kolev, 'Election Quality and International Observation 1975–2004'. Birch, 'Project on Explaining Electoral Malpractice in New and Semi-Democracies'. Elklit and Reynolds, 'A Framework for the Systematic Study of Election Quality', 154–5. An expert panel drawing on a variety of sources and observer reports codes each question on a 1–4 scale. See also Elklit and Skaaning, 'Coding Manual'. Kelley, 'D-Minus Elections'. Pastor, 'The Role of Electoral Administration in Democratic Transitions'; Lehoucq and Molina, Stuffing the Ballot Box; and Eisenstadt, 'Electoral Justice in Mexico'. See also Mozaffar and Schedler, 'The Comparative Study of Electoral Governance', 19. Geddes, 'What Do We Know About Democratization After Twenty Years?'. See also Mozaffar and Schedler, 'The Comparative Study of Electoral Governance', 20. Kelley and Kolev, 'Election Quality and International Observation 1975–2004', 4; and Hyde and Marinov, 'National Elections Across Democracy and Autocracy'. Mozaffar and Schedler, 'The Comparative Study of Electoral Governance'. Ibid. A new, expansive set of indicators, including election data, is being collected for a novel approach to democracy conceptualization and measurement as part of the internationally supported 'V-Dem: Varieties of Democracy' project. See Coppedge and Gerring, 'Conceptualizing and Measuring Democracy: A New Approach'. Two other indices with election components are the Bertelsmann Transformation Index and Cingranelli and Richards' Human Rights Database, but we have excluded these in order to simplify this analysis. Note that Bertelsmann performed worse than the EIU in statistical analysis. Many survey efforts, notably Afrobarometer and Americas Barometer, ask respondents about their perceptions of election quality, which is distinct from assessing the actual process of conducting elections. Similarly, Polity IV includes consideration of executive recruitment and selection, regulation of participation, and competitiveness of participation, but at an abstract level that does not capture critical details of election administration. Bollen and Paxton, 'Subjective Measures of Liberal Democracy', recommend a panel of more than one or two. Casper and Tufis, 'Correlation Versus Interchangeability'; and Bollen and Paxton, 'Subjective Measures of Liberal Democracy'. The five components of the EIU index are: electoral process and pluralism, functioning of government, political participation, political culture, and civil liberties. The rule of thumb for analysis of multi-concept indicators is that a Cronbach's alpha result above 0.7 is good, but 0.9 and higher signals an unreasonable lack of variation. See discussion of Cronback's alpha in Carmines and Zeller, 'Reliability and Validity Assessment'. The six components of the GII are: civil society, public information, and media; elections; government accountability; administration and civil service; oversight and regulation; and anti-corruption and rule of law. Bollen and Paxton, 'Subjective Measures of Liberal Democracy'. A temporal division is also supported by Elklit and Reynolds, 'A Framework for the Systematic Study of Election Quality', and tacitly by Mozaffar and Scheduler, 'The Comparative Study of Electoral Governance', 7–11. Collier and Levitsky, 'Democracy with Adjectives'; and Dahl, Polyarchy. Diamond et al., Consolidating Third Wave Democracies, 8–9; O'Donnell, 'Horizontal Accountability in New Democracies'; Hartlyn, The Struggle for Democratic Politics in the Dominican Republic, 11; and Munck, Measuring Democracy, 124–5. Munck, Measuring Democracy, 125. In 'The Comparative Study of Electoral Governance', Mozaffar and Schedler's discussion of electoral governance implicitly divides the electoral cycle into periods of rule making, rule application, and rule adjudication. Seven of the 48 questions in the survey are used for informational purposes. An additional five questions address the quality of the rules governing party and candidate registration, media access, and other features dealing with electoral competition in the pre-voting period. As discussed below, these five de jure questions are paired with de facto questions 12 to 16 in Table 1, serving as the basis for an elections rule-of-law analysis. See Munck, Measuring Democracy, 30–5, 48–51. Participants easily access the survey via a web link and respondents are to be paid a small honorarium for their time and effort.