Evaluating a National Training Program for Health Services and Policy Research: Building Capacity for Learning Health Systems

ABSTRACT Introduction Healthcare systems face persistent challenges, including workforce pressures and increasing patient complexity, and leaders are increasingly seeking approaches that support rapid learning and improvement. Learning Health Systems (LHSs) support continuous knowledge generation and application and depend on professionals with both traditional research competencies and skills in knowledge mobilization, implementation science, and collaborative leadership. The Health Systems Impact Fellowship (HSIF) National Cohort Training Program (NCTP) prepares doctoral students, postdoctoral fellows, and early‐career researchers (ECRs) in Health Services and Policy Research (HSPR) through embedded research training. Methods We evaluated NCTP outcomes using the Kirkpatrick Evaluation Framework (KEF). Using a convergent mixed‐methods survey design, we collected quantitative ratings and qualitative narrative feedback via online surveys mapped to KEF levels (reaction, learning, behavior, and results/impacts). Descriptive statistics summarized ratings across survey rounds, and directed content analysis (guided by the KEF) was used to analyze open‐ended responses. Results Among 195 invited participants, response rates varied by training session. Overall satisfaction (agree/strongly agree) was 60.6% and varied across themes (52.6%–78.6%), reflecting diverse training needs within a multidisciplinary cohort. Respondents reported knowledge gains (over 70% for several items) and improved confidence (62%). Behavior‐ and impact‐level evidence was less frequent but included early examples of skill application and perceived shifts in organizational practices. Qualitative feedback helped clarify what drove variation across themes. Interim findings were used to refine both training delivery and the evaluation approach. Conclusion Fellowship‐based training programs such as the NCTP can support knowledge‐to‐action capabilities relevant to LHSs. Future evaluations should incorporate longer‐term follow‐up and additional data sources to better assess behavior change and organizational impact.