Evaluation of an Opioid Overdose Composite Risk Score Cutoff in Active Duty Military Service Members

Objective: To evaluate the current cutoff score and a recalibrated adaptation of the Veterans Health Administration (VHA) Risk Index for Serious Prescription Opioid-Induced Respiratory Depression or Overdose (RIOSORD) in active duty service members. Design: Retrospective case-control. Setting: Military Health System. Subjects: Active duty service members dispensed ≥ 1 opioid prescription between January 1, 2018, and December 31, 2019. Methods: Service members with a documented opioid overdose were matched 1:10 to controls. An active duty-specific (AD) RIOSORD was constructed using the VHA RIOSORD components. Analyses examined the risk stratification and predictive characteristics of two RIOSORD versions (VHA and AD). Results: Cases (n = 95) were matched with 950 controls. Only 6 of the original 17 elements were retained in the AD RIOSORD. Long-acting or extended-release opioid prescriptions, antidepressant prescriptions, hospitalization, and emergency department visits were associated with overdose events. The VHA RIOSORD had fair performance (C-statistic 0.77, 95% CI 0.75, 0.79), while the AD RIOSORD did not demonstrate statistically significant performance improvement (C-statistic 0.78, 95% CI, 0.77, 0.80). The DoD selected cut point (VHA RIOSORD > 32) only identified 22 of 95 ORD outcomes (Sensitivity 0.23), while an AD-specific cut point (AD RIOSORD > 16) correctly identified 53 of 95 adverse events (Sensitivity 0.56). Conclusions: Results highlight the need to continually recalibrate predictive models and to consider multiple measures of performance. Although both models had similar overall performance with respect to the C-statistic, an AD-specific index threshold improves sensitivity. The calibrated AD RIOSORD does not represent an end-state, but a bridge to a future model developed on a wider range of patient variables, taking into consideration features that capture both care received, and care that was not received.