Six years’ experience using specialty national residency index: An early dashboard to document change over time

BACKGROUND AND OBJECTIVES: A decade ago, the Association of Family Medicine Residency Directors developed the Residency Performance Index (RPI) as a novel dashboard of metrics to support residency programs’ quality improvement efforts. Although the RPI has since been discontinued, we sought to identify lessons learned from an analysis of 6 years of data collected while the RPI was in use to inform future quality and accreditation efforts implemented at the national level. METHODS: The RPI collected data from 2012-2017 for nearly 250 distinct family medicine residency programs, identifying strengths and areas for improvement. Eighty-two programs provided data for 3 or more years of measures allowing analysis of improvement trends. RESULTS: For participating programs, aggregate data over 6 years indicated the majority had stable leadership and accreditation. Total family medicine center (FMC) visits by graduates and resident visit demographics were robust. Graduate scope of practice was consistent with nationally publicized trends. Programs hit most aspirational targets more than 40% of the time. However, analysis for those programs with 3 or more years of data revealed that the tool did not result in significant changes for most metrics. Linear regression analysis showed improvements in total patient visits, visits under 10 years of age, and certain procedural competencies for those programs with 3 or more years of data. CONCLUSIONS: The RPI was the first and only nationally utilized family medicine graduate medical education quality improvement tool. Individual programs did not show substantial change in quantifiable metrics over time despite limited evidence of select programmatic improvements. Nationally, aggregated data provided insight into scope of practice and other areas of interest in residency training. Further efforts in provision of residency improvement tools are important to support programs given the increasing complexity and high stakes of family medicine residency education.