Improving explainability of post-separation suicide attempt prediction models for transitioning service members: insights from the Army Study to Assess Risk and Resilience in Servicemembers — Longitudinal Study

Risk of U.S. Army soldier suicide-related behaviors increases substantially after separation from service. As universal prevention programs have been unable to resolve this problem, a previously reported machine learning model was developed using pre-separation predictors to target high-risk transitioning service members (TSMs) for more intensive interventions. This model is currently being used in a demonstration project. The model is limited, though, in two ways. First, the model was developed and trained in a relatively small cross-validation sample (n = 4044) and would likely be improved if a larger sample was available. Second, the model provides no guidance on subtyping high-risk TSMs. This report presents results of an attempt to refine the model to address these limitations by re-estimating the model in a larger sample (n = 5909) and attempting to develop embedded models for differential risk of post-separation stressful life events (SLEs) known to mediate the association of model predictions with post-separation nonfatal suicide attempts (SAs; n = 4957). Analysis used data from the Army STARRS Longitudinal Surveys. The revised model improved prediction of post-separation SAs in the first year (AUC = 0.85) and second-third years (AUC = 0.77) after separation, but embedded models could not predict post-separation SLEs with enough accuracy to support intervention targeting.