A Home Hemodialysis Objective Structured Clinical Examination for Formative Assessment of Nephrology Fellows

KEY POINTS: Because of low patient numbers and uneven geographical distribution, nephrology fellows may not have experience caring for home hemodialysis (HHD) patients. We developed and preliminarily validated a formative HHD objective structured clinical examination. The objective structured clinical examination may be used to assess HHD curriculum and basic HHD clinical skills in individual fellows.

BACKGROUND: The Accreditation Council for Graduate Medical Education requires graduating nephrology fellows demonstrate competence in home hemodialysis (HHD). Because low patient numbers may lead to training gaps, clinical experience may be enhanced using simulation. We designed and preliminarily validated a formative objective structured clinical examination (OSCE) assessing clinical care of an uncomplicated patient initiating HHD, using a unified model of construct validity.

METHODS: The OSCE was developed by a nephrology fellow and five faculty (three practicing HHD). The nine-member test committee (five in HHD practice; one HHD care partner) assessed test item difficulty/relevance and determined passing scores. The final test consisted of 27 items (31 possible points); seven were evidence-based/standard-of-care questions (9.5 possible points). Passing score was 20 of 31 points (65%). Median relevance for all items was “important” or “essential.” Content validity index was 0.84. On preliminary validation by 11 board-certified volunteers (four practicing HHD), overall mean±SD score was 27.5±2 (100% passing), kappa=0.83 (95% confidence interval, 0.67 to 0.99). Validator evidence-based question score was 9.0±0.6.

RESULTS: Thirty-eight fellows (9 programs, 21 first-year; 17 second-year) were tested. Seventy-one percent passed (Cronbach α=0.70). Fellows' mean±SD scores were lower than validators: 21.5±4.0 versus 27.5±2, P < 0.001, as were their scores on evidence-based questions: 7.4±1.4 versus 9.0±0.6, P < 0.001. Eighty-eight percent of evidence-based/standard-of-care questions were answered correctly by validators versus 62% by fellows ( P < 0.001). Forty-two percent of fellows were able to name four potential benefits and two risks associated with HHD; 79% recognized that the primary risk of buttonhole cannulation was infection. Seventy-four percent correctly identified minimum single pool Kt/V for thrice-weekly hemodialysis, and 29% knew the minimum standard weekly Kt/V target. Eighty-eight percent of fellows surveyed (22 of 25) agreed/strongly agreed that the OSCE was useful in self-assessing proficiency.

CONCLUSIONS: The OSCE may be used as a formative assessment of fellow proficiency in prescribing HHD.