Evaluation of differences in postgraduate year one pharmacy residency application rubrics

Background: The American Society of Health-System Pharmacists standards require pharmacy residency programmes to systematically evaluate candidates. This study aimed to assess differences among eight PGY1 pharmacy residency application rubrics used to select candidates for interviews. Methods: In December 2018, fifty applicants to the University of Kentucky HealthCare’s PGY1 Pharmacy Residency Programme were evaluated using eight different rubrics. A subgroup of sixteen applicants had their scores assessed by multiple scorers to analyse inter-rater variability. The main goal was to examine score variability across all applicants. Scores were normalised to a common scale, and statistical analysis included descriptive statistics and ANOVA tests. Results: The evaluation of a candidate pool (n = 520) using eight unique rubrics yielded an overall mean score of 66.98 out of 100, with significant score differences across rubrics (p < 0.001). Inter-rater variability was low, with a maximum difference of six percent in scores. All rubrics assessed work experience, leadership positions, and publications, but not all included letters of recommendation. Letters of recommendation, work experience, and rotation experiences received the highest weight in the rubrics. Conclusion: There was notable variability in scores among eight different PGY1 pharmacy residency application rubrics. While aspects like rotation experiences were highly ranked, others, such as letters of recommendation, varied significantly. Programmes prioritise different elements based on their preferences, leading to differences in applicant evaluations. This allows programmes to find candidates that fit their practice, but applicants should be aware that their portfolios may be assessed differently across programmes.