Assessing Quality of Surgical Real-World Data from an Automated Electronic Health Record Pipeline

Background: Significant analysis errors can be caused by nonvalidated data quality of electronic health records data. To determine surgical data fitness, a framework of foundational and study-specific data analyses was adapted and assessed using conformance, completeness, and plausibility analyses. Study Design: Electronic health records-derived data from a cohort of 241,695 patients undergoing 412,182 procedures from October 1, 2014 to August 31, 2018 at 3 hospital sites was evaluated. Data quality analyses tested CPT codes, medication administrations, vital signs, provider notes, labs, orders, diagnosis codes, medication lists, and encounters. Results: Foundational checks showed that all encounters had procedures within the inclusion period, all admission dates occurred before discharge dates, and race was missing for 1% of patients. All procedures had associated CPT codes, 69% had recorded blood pressure, pulse, temperature, respiration rate, and oxygen saturation. After curation, all medication matched RxNorm medication naming standards, 84% of procedures had current outpatient medication lists, and 15% of procedures had missing procedure notes. Study-specific checks temporally validated CPT codes, intraoperative medication doses were in conventional units, and of the 13,500 patients who received blood pressure medication intraoperatively, 93% had a systolic blood pressure >140 mmHg. All procedure notes were completed within less than 30 days of the procedure and 93% of patients after total knee arthroplasty had postoperative physical therapy notes. All patients with postoperative troponin-T lab values ≥0.10 ng/mL had more than 1 ECG with relevant diagnoses. Postoperative opioid prescription decreased by 8.8% and nonopioid use increased by 8.8%. Conclusions: High levels of conformance, completeness, and clinical plausability demonstrate higher quality of real-world data fitness and low levels demonstrate less-fit-for-use data.