Development and validation of machine-learning algorithms predicting retention, overdoses, and all-cause mortality among US military Veterans treated with buprenorphine for opioid use disorder

Abstract: Aim: The aim of this study was to develop and validate machine-learning algorithms predicting retention, overdoses, and all-cause mortality among US military veterans treated with buprenorphine for opioid use disorder (B-MOUD). Methods: This study used a prognostic cohort study design and data from the U.S. Veterans Healthcare Administration on veterans initiating B-MOUD from fiscal years 2006-2020. Candidate predictors (n=114) were measured over the 12 months before B-MOUD initiation. Veterans’ B-MOUD episodes were randomly divided into training (80%; n=45,238) and testing samples (20%; n=11,309). Candidate algorithms (multiple logistic regression, least absolute shrinkage and selection operator regression, random forest (RF), gradient boosting machine (GBM), and deep neural network (DNN)) were used to build and validate a classification model to predict six different binary outcomes important to persons in B-MOUD treatment: 1) B-MOUD retention, 2) any overdose, 3) opioid-related overdose, 4) overdose death, 5) opioid overdose death, and 6) all-cause mortality. To balance the outcome categories, various over- and under-sampling techniques were employed. Model performance was assessed using standard classification statistics [e.g., area under the receiver operating characteristic curve (AUC-ROC)]. Results: Training and testing samples had similar characteristics (93% male, 78% White, mean age was 46.5 years). In the testing sample, the GBM model slightly outperformed other models in predicting B-MOUD retention (AUC-ROC=0.72). Generally, RF models outperformed other models in predicting any overdose (AUC-ROC=0.77) and opioid overdose (AUC-ROC=0.77). RF and GBM outperformed other models for overdose death (AUC-ROC=0.74 for both), and RF and DNN outperformed other models for opioid overdose death (RF AUC-ROC=0.79; DNN AUC-ROC=0.78). RF and GBM also outperformed other models for all-cause mortality (AUC-ROC=0.76 for both). Conclusions: Machine-learning algorithms, particularly RF, GBM, and DNN, can accurately predict B-MOUD retention, overdoses, and all-cause mortality among Veterans initiating B-MOUD with moderate predictive performance.

Read the full article
Report a problem with this article

Related articles

  • More for Researchers

    Identifying opioid relapse during COVID-19 using natural language processing of nationwide Veterans Health Administration electronic medical record data

    Abstract: Novel and automated means of opioid use and relapse risk detection are needed. Unstructured electronic medical record data, including written progress notes, can be mined for clinically relevant information, including the presence of substance use and relapse-critical markers of risk and recovery from opioid use disorder (OUD). In this study, we used natural language processing (NLP) to automate the extraction of opioid relapses, and the timing of these occurrences, from veteran patients' electronic medical record. We then demonstrated the utility of our NLP tool via analysis of pre-/post-COVID-19 opioid relapse trends among veterans with OUD. For this demonstration, we analyzed data from 107,606 veterans OUD enrolled in Veterans Health Administration, comparing a pandemic-exposed cohort (n = 53,803; January 2019-March 2021) to a matched prepandemic cohort (n = 53,803; October 2017-December 2019). The recall of our NLP tool was 75% and our precision was 94%, demonstrating moderate sensitivity and excellent specificity. Using the NLP tool, we found that the odds of opioid relapse postpandemic onset were proportionally higher compared to prepandemic trends, despite patients having fewer mental health encounters from which to derive instances of relapse postpandemic onset. In this research application of the tool, and as hypothesized, we found that opioid relapse risk was elevated postpandemic. The application of NLP Methods: to identify and monitor relapse risk holds promise for future surveillance, risk prevention, and clinical outcome research.