The Reliability and Validity of the American Board of Internal Medicine Monthly Evaluation Form

Steven J. Durning*, Lannie J. Cation, Jeffrey L. Jackson

*Corresponding author for this work

Research output: Contribution to journalReview articlepeer-review

19 Scopus citations

Abstract

Purpose. To report indicators of reliability and validity of the American Board of Internal Medicine Evaluation Form (ABIM-MEF) at one institution (Wright-Patterson Medical Center). Method. Completed ABIM-MEFs from 1990-1999 were reviewed. Reliability measures included Cronbach α, interrater reliability, and rating consistency between different types of staff and rotations. Construct validity was investigated by tracking ABIM-MEF scores over time and with factor analysis. Predictive validity was assessed by correlating ABIM-MEF scores with the In-training Examination and ABIM Certifying Examination results. Results. The 71 residents averaged 12 ABIM-MEFs per year. The forms had a Cronbach α of 0.96 and high interrater reliability (intraclass correlation coefficients > 0.80). Ratings did not differ by type of attending or rotation, except that noninternists rated residents lower on procedural skills. ABIM-MEF questions about judgment, knowledge, and clinical skills showed significant improvement from month to month during each academic year as well as year to year. In contrast, questions on professional attitudes, humanism, and procedural skills sections improved between postgraduate year 1 and postgraduate year 2 only. ABIM-MEF questions collapsed into two domains in factor analysis: judgment-knowledge-skills and attitude-humanism. ABIM-MEF questions from judgment and knowledge sections modestly predicted In-training Examination and ABIM Certifying Examination results. In contrast, professional attitude, humanism, and clinical as well as procedural skill questions had little discriminative ability. Conclusions. The ABIM-MEF appears to be reliable and valid. Further, factor analysis results support the ABIM's movement to simplify the monthly evaluation form to the new Accreditation Council for Graduate Medical Education core competencies.

Original languageEnglish
Pages (from-to)1175-1182
Number of pages8
JournalAcademic Medicine
Volume78
Issue number11
DOIs
StatePublished - Nov 2003
Externally publishedYes

Fingerprint

Dive into the research topics of 'The Reliability and Validity of the American Board of Internal Medicine Monthly Evaluation Form'. Together they form a unique fingerprint.

Cite this