Validity Evidence for the New American Board of Emergency Medicine Certifying Examination

ABSTRACT Introduction Certification of emergency physicians includes a two‐phase process, including both a written qualifying examination and an oral certification examination. In 2026, the American Board of Emergency Medicine will institute a new certifying examination (CE), which will be an in‐person assessment with objective structured clinical examinations (OSCEs), including focused communication‐centered cases and procedural assessments. The objective of this paper is to provide initial validity evidence for the new CE. Methods A unified view of validity was used to guide the collection of evidence for content validity for the new CE. Moreover, an evaluation of possible validity threats was conducted. An iterative and collaborative process including focus groups, a summit, one‐on‐one employer interviews, and public and patient input was used to identify key domains and establish the exam blueprint for the new CE. The exploratory phase for exam development, known as the Becoming Certified Initiative, spanned a two‐year timeline. Evidence was collected at a practice administration of the CE through the evaluation of candidates' and examiners' perceptions of alignment of case content to emergency medicine (EM) practice, as well as understanding and clarity of the cases and their instructions. Results The new CE blueprint comprises 10 components: four clinical cases covering clinical decision‐making and prioritization and six OSCEs covering procedures, ultrasound, patient‐centered communication, reassessment, difficult conversations (delivering unwanted or unexpected information), and managing conflict. Both candidate and examiner volunteers rated the cases as clinically accurate and relevant to EM practice. They also reported that case instructions and prompts within the cases were generally clear and understandable. Conclusion This study provides important early validity evidence for the new CE to inform emergency physicians and the public at large.