The feasibility and construct validity of graduate and supervisor surveys in a pulmonary fellowship training program

Purpose: The purpose of this study was to determine the feasibility and construct validity of our graduate and employer survey and its potential use in programmatic evaluation. Methods: An identical survey instrument was sent to fellowship graduates from our institution and their employers. We estimated feasibility through determining our survey response rate. Construct validity was assessed by comparing graduate self-ratings and employer ratings and through comparing survey ratings with passing board-certification examinations and referrals to the National Practitioner Databank. Results: Eleven years of graduates were included (n = 38 graduates). Response rates were 84% and 82% for graduates and their employers, respectively. Mean supervisor ratings were greater than graduate self-ratings on all questions. The lowest rating consistently across survey years was in graduates' self-perception of their ability to direct a PFT lab. One fellow failed his pulmonary boards and 1 failed his critical care boards on the first attempt. No graduates were referred to the National Practitioner Databank during our study period. Conclusions: This fellowship survey appears to be feasible and have construct validity. The consistency in the relatively low self-rating of graduates in directing a PFT lab has led to important areas for curriculum reform.