Back to 2024 Display Posters
Analyzing Supervised Machine Learning Models For The Prediction Of Endoleaks Following Endovascular Aortic Aneurysm Repair
Sharon C. Kiang, MD1, Daniel Roh, B.S.
2, Andrew Cabrera, B.S.
2, Ryan Mezan, MD
2, Sina Asaadi, MD
2, Zion Shih, B.S.
1, Jeffrey J. Siracuse, MD
3, Caitlin W. Hicks, MD
4, Roger T. Tomihama, MD
2.
1Loma Linda Veterans Healthcare System, Loma Linda, CA, USA,
2Loma Linda University Medical Center, Loma Linda, CA, USA,
3Boston University, Chobanian and Avedisian School of Medicine, Boston, MA, USA,
4Johns Hopkins University School of Medicine, Baltimore, MD, USA.
OBJECTIVES: The purpose of this study was to evaluate the performance of various supervised machine learning algorithms to predict the occurrence of endoleaks following endovascular abdominal aortic aneurysm repair.
METHODS: This is an IRB-approved retrospective study that analyzed the demographic and clinical data of 72 patients with endoleak and 72 propensity-matched control patients with no endoleak. Seven machine learning models were selected: random forest, gradient boosting, decision tree, support vector machine, gaussian-naïve Bayes, multi-layer perceptron, and logistic regression. Sixty-four clinical and demographic variables were used as the feature matrix, which was standardized between a scale of 0 and 1 for both continuous and categorical variables. The data was split 70:30 amongst the training and testing sets, respectively. The training data was inputted into each of the machine learning models with their respective preset hyperparameters. Then, the testing data was analyzed by each trained model and the predictions of each of the models were compared with the ground truth. The sensitivities, specificities, confusion matrices, and AUC curves were then calculated.
RESULTS: Initial results demonstrated that all seven models performed with AUCs ranging from 0.39 to 0.51. A permutation feature importance (PFI) was performed on the 64 clinical data points, which significantly to improved the AUC of the models. Smoking, ASA, and COPD had the highest average feature importance for all models, with an average AUC improvement of 0.123. After PFI was implemented, the Random Forest Classifier was the best-performing model, with a sensitivity of 48%, a specificity of 74%, and an AUC improvement from 0.39 to 0.65. The Support Vector Machine Classifier was the least accurate model, with a sensitivity of 52%, a specificity of 74%, and an AUC decrease from 0.51 to 0.35. (Fig 1)
CONCLUSIONS: Based on the clinical variables identified through the utilization of a PFI methodology, machine learning algorithms were able to further optimize predictive capacity in detecting endoleaks. Using PFI, we can identify the clinical data points that most significantly impact the performance of each model, leading us to optimize each model to identify the model that will best predict the occurrence of endoleaks after EVAR.
Back to 2024 Display Posters
