A Retrospective Machine Learning Analysis to Predict 3-Month Nonunion of Unstable Distal Clavicle Fracture Patients Treated with Open Reduction and Internal Fixation

Introduction

Distal clavicle fractures constitute a prevalent type of upper limb fracture, representing 28% of all adult clavicle fractures.¹ Among these, 10% to 52% are classified as displaced and unstable, attributed to the multidirectional forces exerted by the robust trapezius, latissimus dorsi, and pectoral muscles.² Recent research indicates that the nonunion rate for unstable distal clavicle fractures (UDCFs) subjected to nonoperative treatment can reach up to 30%.^3,4 Consequently, Open reduction and internal fixation (ORIF), incorporating anatomic locking plate and hook plate techniques, is widely recommended in China. Nevertheless, clinicians frequently encounter adverse outcomes, including internal fixation failure, re-fracture, 3-month nonunion (ie, delayed union) and eventual bone nonunion.^5–8 Nonunion poses significant treatment challenges and incurs considerable financial burdens, with indirect costs like productivity losses being pivotal in escalating the total expenses for non-union patients. Hence, any strategy aimed at reducing healing duration, facilitating quicker return to work, and minimizing financial implications should be advocated for patients with fractures and non-unions. Predicting high-risk factors following UDCFs could yield compelling evidence to guide reasonable intraoperative and postoperative interventions, thereby abbreviating the disease course and alleviating discomfort.

Some studies reveal that various risk factors are associated with the adverse outcomes of 3-month nonunion and nonunion in fracture patients. These include physiological factors (eg, advanced age, smoking), comorbidities (eg, metabolic diseases, use of non-steroidal anti-inflammatory drugs, diabetes, nutritional deficiencies), and patient-specific factors (eg, fracture pattern and location, displacement, infection, degree of bone loss, and severity of soft tissue injury).^9,10 Distal clavicle fractures, particularly unstable types, are prone to complications such as nonunion, significantly impacting patient recovery and healthcare resources. Despite advancements in surgical techniques, the nonunion rates remain high, posing a challenge for effective management. Traditional functional assessments of the shoulder joint, such as the Constant-Murley and Neer scores, predominantly evaluate recovery from a functional standpoint without providing reliable predictions for fracture healing outcomes.

Recent research has demonstrated the growing application of ML models in the medical sector.^11–16 Logistic regression is a mature modeling technique in public health, boasts high computational efficiency and accurate predictions.^17,18 Random forest is a robust machine learning algorithm, combines multiple-decision trees to effectively mitigate overfitting and handle both discrete and continuous data without normalization.¹⁹ Extreme gradient boosting is a classic ensemble boosting framework, excels in high training efficiency, good prediction outcomes, numerous controllable parameters, and user-friendly implementation.²⁰ Multi-layer perceptron is a neural network model capable of capturing complex patterns and relationships in data through its layered architecture, offering high flexibility and predictive power.²¹ Category boosting enhances the performance of categorical data predictions by combining multiple weak learners, focusing on difficult-to-classify instances for improved accuracy.

In comparison to traditional regression models, ML has been validated as an effective method for prognosis prediction in modeling due to its capacity to intricately analyze complex non-linear interrelations among variables,^22,23 and to enhance prediction accuracy through its superior algorithms, particularly when analyzing large datasets with numerous variables.^24,25 Crucially, the widespread adoption of Electronic Patient Record systems and the extensive use of structured patient data have made sophisticated algorithmic modeling and bedside application feasible.²⁶

Recent studies emphasize the potential of predictive modeling in enhancing the prognosis of fracture healing by identifying risk factors that are not immediately modifiable but crucial for postoperative management. Recognizing the significant variability in nonunion rates across different demographics and institutions, our study aims to develop a machine learning model that incorporates both modifiable and non-modifiable predictors. This approach seeks to enable tailored postoperative strategies to mitigate the risk of 3-month nonunion and enhance patient outcomes across diverse clinical settings. By focusing on a high-risk population within a specific medical setting, this study provides a foundation for validating our model in broader contexts, thereby addressing the pressing need for precise and adaptable predictive tools in orthopedic surgery.

Patients and Methods

Study Population

This retrospective study was performed in the Nanjing Luhe People’s Hospital, China, where electronic health records from the orthopedic unit for patients diagnosed with UDCFs were collected and analyzed, spanning from January 2015 to May 2023. Before data collection, we estimated the minimum sample size required using the 10 EPV (events per variable) rule. We expected 4 variables to enter our final and assume that in our database incidence rate of the outcome is 25% based on previous study. Therefore, we need at least (4*10)/0.25=160 patients according to the EPV rule. The primary outcome was the union of UDCFs within the first three months post-operation, as determined by clinical signs of healing and corroborated by shoulder X-rays images. The exclusion criteria were patients with incomplete clinical or pre-treatment data, those receiving conservative treatment, individuals under 18, and patients with concurrent neurovascular injuries.

Predictors and Outcomes

In this study, the candidate variables for our model included demographic factors such as age, sex, smoking status, drinking status, level of education, and labor intensity, medical histories like diabetes mellitus, hypertension, anemia, and hypoproteinemia, baseline characteristics including height, weight, diastolic and systolic blood pressures, presence of comminuted or multiple fractures, and results of blood biochemical tests, as well as surgical details such as the method of operation, types of implants (including the lateral locking plate and Hook plate), CCD, type of anesthesia, duration of surgery, volume of blood loss, length of incision, and use of osteogenic stimuli, and postoperative outcomes, including 3-month fracture union status and complications.

We sought to predict a binary outcome, either the achievement of union within 3 months or a 3-month nonunion, and evaluated the status of fracture union during the initial 3 months postoperatively by analyzing shoulder X-rays images, focusing on the disappearance of the fracture line and the formation of continuous callus as indicators of union. CCD was defined as the measure between the coracoid process and the inferior aspect of the clavicle,²⁷ determined from shoulder X-ray images taken within 7 days after surgery. In addition, educational attainment was categorized into two levels: high, for those with college education, and low, for those without.

Data Pre-Processing

The dataset was randomly split into a training set (70% of the cases) and a testing set (30% of the cases) to ensure robust internal validation of the predictive models. Two methods were mainly used to deal with missing data in both training and test sets. Variables with a missing data rate exceeding 30% were removed outright from the dataset, and for the rest of remaining variables exhibiting missing values, multiple imputation employing predictive mean matching (PMM) was executed. The One-Hot Encoding technique was applied to transform multiple categorical variables into dummy variables. Whereas continuous variables underwent normalization by a z-score.

Model Development and Validation

In order to select the variables that would eventually develop models, we used the LASSO algorithm to process variables that demonstrated statistical significance in univariate analysis. The variables retaining nonzero coefficients following the LASSO were selected as the final variables for model development.

Five machine learning classifiers—logistic regression (LR), random forest classifier (RFC), extreme gradient boosting (XGB), multi-layer perceptron (MLP), and CatBoost—were employed. Their performance was compared using evaluation metrics such as AUROC, AUPRC, accuracy, sensitivity, specificity, F1 score, and calibration curves. The optimal parameters for each algorithm were determined by the grid search algorithm coupled with 10-fold cross-validation. The test set in our study provided data for internal validation to demonstrate the generalization of the machine learning model, ensuring the model, developed from the training data, maintained its applicability.

Model Evaluation

Due to the imbalance within our dataset, AUPRC was selected as the principal evaluation metric, which has been proven to be superior to AUROC, especially in the context of imbalanced data. In addition, accuracy, recall, precision, sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), and the F1 score were served as metrics to evaluate the models. A calibration curve analysis was also carried out to visually evaluate the degree of agreement between the model’s predicted and actual probabilities.

Model Interpretation

SHAP method was used to evaluate variable importance and provide an explanation for the predictions generated by ML algorithms in order to gain a better understanding of the machine learning model’s predictive process. SHAP is a useful tool that helps boost the model’s credibility and acceptability in clinical environments, which illustrates how each variable contributes to and influences the risk of 3-month nonunion.

Statistical Analysis

Baseline characteristics were compared between the control group and the 3-month nonunion cohort. The Shapiro–Wilk test was used firstly to determine whether continuous variables were normally distributed. Descriptive statistics were presented as means with standard deviations (for normally distributed variable), or medians with IQR (for non-normally distributed variable), and frequency (percentages) for categorical variables, respectively. We used the Student’s t-test or the Mann–Whitney U-test to assess the differences in continuous variables, while for categorical variables, the Chi-squared or Fisher’s exact tests were performed. The above statistical analysis conducted with the statistical package R, version 4.1.3 (R Development Core Team, Auckland, New Zealand), and p-values less than 0.05 was considered statistically significant.

Results

Baseline Characteristics

We analyzed the data of 248 patients treated for UDCFs from January 2015 to May 2023. In general, the overall median age was 55 years, interquartile ranges (IQR): 47–63 years, and with males constituting 60.1% of the cohort. 3-month nonunion was observed in 76 patients (30.6%, median age: 55 years, number of male patients: 52), finally 6 cases had bone nonunion after extended follow-up time, and underwent secondary surgery. Whereas 172 patients exhibited primary healing post-procedure (69.4%, median age: 54.5 years, number of male patients: 97). After data split, 173 patients were allocated to the training set and 75 to the test set. The detailed baseline information of the training set is shown in Table 1 and the training set and the testing set were similar in baseline characteristics.

Table 1 Baseline Characteristics of Patients in Normal and Delayed Union Groups in the Training Set

Feature Selection

Seven of the 47 variables we gathered that may be connected to 3-month nonunion were statistically significant after univariate analysis. Furthermore, after exclusion of variables with zero coefficients through the Least Absolute Shrinkage and Selection Operator(LASSO) process, we ultimately used four variables for model construction: Coracoclavicular distance(CCD), anesthesia method, High-density lipoprotein(HDL), and blood loss. The detailed coefficients of these non-zero coefficient variables are detailed in Supplementary Table 1.

Model Performance

All machine learning models that have been constructed have demonstrated similar performance, with no significant differences in AUROCs, as determined by the DeLong test (Supplementary Table 2). Among the five predictive models, the Category Boosting (CatBoost) model attained the greatest AUROC (0.863, 95% CI: 0.762–0.964) and Area Under the Precision-Recall Curve (AUPRC 0.801, 95% CI: 0.592–0.918), the highest accuracy (74.7%), F1 score (0.689), and PPV (55.3%) in the test set. AUROCs and AUPRCs for both the training and test sets are depicted in Figure 1. In addition, the calibration curve, presented in Figure 2, demonstrates a good alignment between the predictions of the CatBoost model and the actual outcomes. Evaluation metrics for all models are compiled in Table 2.

Table 2 The Performance of the Models in Training and Test Set

Figure 1 Receiver operating characteristic (ROC) curve and precision-recall curve (PRC) of the five models for predicting the risk of 3-month nonunion. (a) The ROC curve of the training set. (b) The ROC curve of the testing set. (c) The PRC of the training set. (d) The PRC of the testing set. Abbreviations: LR, logistic regression; RFC, random forests classifier; XGB, eXtreme gradient boosting; MLP, multi-layer perceptron; CatBoost, category boosting.

Figure 2 Calibration curve of the five models. Abbreviations: LR, logistic regression; RFC, random forests classifier; XGB, eXtreme gradient boosting; MLP, multi-layer perceptron; CatBoost, category boosting.

Model Interpretation

We offer Figure 3 as the global interpretation to the prediction behavior of the CatBoost model using SHapley Additive exPlanations(SHAP). Our model identified CCD as the main driving factor based on the mean absolute SHAP value as Figure 3a shows. Figure 3b summarizes the direction of effects for each variable. Feature values are indicated by a spectrum with blue representing the lowest value and red color indicates higher risk of 3-month nonunion while blue color shows lower risk. As a result, the variables included in the CatBoost are all risk factors except for the anesthesia method of nerve block.

Figure 3 Shapley additive explanations (SHAP) plots of the selected variables in the CatBoost model. (a) The mean absolute SHAP value ranks the relative importance of the variables in the model. (b) The direction of effects for each variable in the model. Abbreviations: CCD, coracoclavicular distance; HDL, high density lipoprotein.

From our result, this study is the first to employ ML techniques for predicting 3-month nonunion in UDCF patients treated with ORIF within a Chinese cohort, providing novel insights into both risk stratification and potential avenues for clinical intervention.

Discussion

Despite the clinical efficacy of diverse surgical approaches in managing UDCFs, the issue of postoperative nonunion persists, influenced by numerous factors pre-, intra-, and post-operatively. Accurately predicting the risk of nonunion following ORIF remains an unresolved challenge. Developing a prediction model for the risk of 3-month nonunion among UDCFs patients aids in identifying risk factors and facilitates early intervention to reduce incidence.

A significant advantage of our model is its outstanding performance. This study employed five distinct machine learning methodologies to develop a predictive model for the risk of 3-month nonunion in Chinese patients with UDCFs undergoing ORIF, based on four critical variables: CCD, anesthesia method (nerve block), HDL, and blood loss. Among these models, the CatBoost model emerged as the most effective, achieving an AUPRC of 0.801, a critical measure for assessing the predictive accuracy of models trained on imbalanced datasets. In addition, other evaluation metrics such as sensitivity and F1 score also achieve better performance. Subsequently, risk stratification was executed using the optimal threshold defined by the CatBoost model, enabling the high-risk group identified by the model to benefit from preventive interventions targeting CCD, HDL and blood loss. The results showed that CCD, HDL levels, and blood loss were associated with an elevated risk of 3-month nonunion. Conversely, the application of nerve blocks, including postoperative block, was correlated with a reduced risk.

Risk Factors Affecting Fracture Union

It is well known that many factors can affect bone union. Qvist et al ²⁸ conducted a retrospective analysis of 150 clavicle fracture cases, assessing eight factors: age, sex, smoking status, comminuted fractures, fracture shortening exceeding 2 cm, Disabilities of the Arm, Shoulder and Hand (DASH) scores, Visual Analogue Scale (VAS) scores, and the VAS ratio (four-week VAS score/two-week VAS score). Their findings revealed that: (1) a rise in the absolute pain score 4 weeks post-fracture correlated with a heightened risk of nonunion at 6 months; (2) patients with a VAS ratio above 0.6 faced an 18-fold increased relative risk of nonunion compared to those with a VAS ratio below 0.6. Nicholson et al ²⁹ conducted a prospective follow-up of 200 patients with clavicle fractures, assessing them 6 weeks post-fracture to predict the likelihood of nonunion after 6 months. The AUROC of the final established model was 0.873. This study gathered data on 7 patient-related factors, 3 fracture characteristics, 5 outcomes from physical examinations, and 3 results from subjective functional evaluations. Analysis via a conditional stepwise regression model indicated that a QuickDASH score ≥ 40, absence of callus formation in X-ray images, and fracture displacement observed during physical examinations were predictors of nonunion. Takahashi et al ³⁰ used ML to predicted osteoporotic vertebral fracture nonunion, the ML architecture utilized in this study included a logistic regression model, decision tree, extreme XGBoost, and random forest RF, and found that MRI findings, anterior height ratio, kyphotic angle, posterior wall injury, fracture level, and smoking habit ranked as important features in the ML algorithms. Leister et al ³¹ used XGBoost algorithm to identify treatment-related risk factors, found that the odds of odontoid fracture nonunion increased significantly with age. Similar to the aforementioned studies, our research serves as a risk alert for 3-month nonunion; however, we discovered that preoperative HDL levels, intraoperative blood loss, and postoperative CCD were linked to the risk of nonunion, diverge from those of previous research, likely due to its foundation on distinct risk factors. These predictors are readily ascertainable via laboratory tests, imaging examination and surgical documentation, and the developed model based on these risk factors has relatively good prediction performance (AUROC of the Catboost is 0.863 and the AUPRC is 0.801). This significantly aids clinicians in making tailored and accurate treatment decisions for high-risk individuals, thereby mitigating the risk of 3-month nonunion and offering broad applicability in clinical settings.

Predicted Factors

Our results indicate that an increased CCD is associated with a higher risk of 3-month nonunion. It is widely recognized that conservative treatment of UDCFs carries a higher risk of nonunion and may result in shoulder instability.^32–34 The coracoclavicular ligament is pivotal in stabilizing the acromioclavicular joint, consisting of the trapezoid and conoid ligaments. These components are instrumental in preventing forward and backward displacement, as well as upward rotation of the clavicle. Notably, the trapezoid ligament exhibits a significantly enhanced capacity to restrict backward displacement compared to the conoid ligament.^35,36 An increase in the CCD is predominantly attributed to injuries of the coracoclavicular ligament, which leads to stress concentration at the distal fracture end, thereby diminishing stability. Fleming et al ³⁷ highlighted that combined coracoclavicular ligament repair more effectively restores the CCD and stabilizes the acromioclavicular joint. Surgical strategies to restore the CCD typically encompass ORIF for anatomical realignment, Coracoclavicular Screw Fixation in cases where ligaments are intact, Ligament Reconstruction with grafts to replicate shoulder biomechanics, Hook Plate Fixation for provisional stability, and minimally invasive Arthroscopic Techniques utilizing various devices for less invasive interventions.³⁸ Therefore, radiological images reveal significantly increased CCD in the preoperative or intraoperative, which suggest that it is necessary to reevaluate the stability and surgical strategy.

Second, our findings indicate that anesthesia impacts bone nonunion three months post-surgery, with nerve blocks, including postoperative nerve block, potentially serving as a protective factor against nonunion outcomes. A variety of anesthesia can be employed in clavicular surgeries,^39–42 however an increasing body of literature advocates for the use of nerve blocks. Nerve blocks alleviate pain by inhibiting nerve cells from transmitting electrical signals that the brain interprets as pain. This effect is accomplished through anesthetics that obstruct the nerve receptors tasked with detecting damage or injury. Additionally, nerve blocks diminish inflammation, which aids in nerve recovery and further lessens pain.^1,39 Certain studies have suggested that mitigating absolute pain may be advantageous for the union outcome following unstable distal clavicle fractures.⁴³ A separate retrospective cohort study revealed that employing regional anesthesia in ORIF for clavicle fractures correlated with a higher rate of same-day discharge.⁴⁴ Ryan et al ⁴⁵ discovered that the combined use of brachial plexus regional anesthesia with a modified superficial cervical plexus block constitutes a dependable and effective approach, potentially offering advantages in terms of anesthesia initiation and overall case duration. Our retrospective analysis suggesting that opting for nerve block anesthesia, including postoperative nerve block, could diminish the risk of 3-month nonunion.

Third, HDL have a significant impact on the pathology and management of conditions such as atherosclerosis, morbid obesity, nonalcoholic fatty liver disease, type 2 diabetes, and various central nervous system disorders.⁴⁶ Researchers have uncovered a novel function of HDL in the development of degenerative and metabolic bone diseases through experimental mouse studies.^47,48 The findings indicate that reduced and dysfunctional HDL levels may contribute to a higher incidence of these diseases by influencing the molecular mechanisms involved in bone synthesis and degradation. Studies have reported that elevated HDL-C levels are linked to osteoporosis ⁴⁹ and an increased risk of fractures.^50,51 The relationship between high HDL-C levels, low bone mineral density, and osteoporosis,⁵² along with a genome-wide association study linking high HDL-C to low bone mineral density,⁵³ may offer a pathophysiological explanation. A case-control study by Quan et al ⁵⁴ identified osteoporosis as an independent risk factor for bone nonunion. Furthermore, another study presented high-quality evidence of a negative correlation between HDL-C levels and bone mineral density.⁵⁵ Integrating these findings with our research suggests that an increased HDL-C levels may be associated with a heightened risk of nonunion. However, this view conflicts with other opinions that assert no association between HDL-C levels and osteoporosis.⁵⁶ Thus, whether high HDL levels exacerbate the risk of 3-month nonunion requires further investigation.

Fourth, our research identified a correlation between intraoperative blood loss and the risk of 3-month nonunion. Research has demonstrated that formation of callus beyond the fracture site predominantly relies on the vascular supply to the periosteum and adjacent soft tissues. Furthermore, factors like trauma, excessive displacement at the fracture site, comminuted fractures, and both open and closed soft tissue injuries can impair the local blood supply. Surgical interventions can further influence the vascular supply to the fracture site, necessitating procedures like extended wound exposure, periosteal stripping, and varying levels of damage to surrounding soft tissues. Following a fracture, the blood supply to the affected end is compromised, and full restoration of vascular supply to this area may take over 6 months. ⁵⁷ Insufficient blood supply prevents callus formation and increases the risk of nonunion. To mitigate the risk of postoperative nonunion, a multifaceted approach is essential for managing intraoperative hemorrhage: (1) Minimizing surgical trauma and employing minimally invasive techniques are crucial for preserving the peripheral blood supply, especially the protection of periosteum; ³⁸ (2) The application of advanced hemostatic methods, including electrocoagulation and topical hemostatic agents, is recommended; ⁵⁸ (3) It is imperative to secure stable fracture fixation to minimize displacement at the fracture site; (4) Postoperative care should encompass optimal nutrition and the initiation of controlled activities to enhance circulation; (5) The administration of anti-fibrinolytic agents, for instance, triamcinolone, is advocated to curtail bleeding.⁵⁹ Employing these comprehensive strategies is beneficial for sustaining an adequate blood supply at the fracture site, thereby diminishing the likelihood of 3-month nonunion.

Practical Implications of Predictive Modeling

Our results suggest that while some predictors like HDL and intraoperative blood loss are not directly modifiable during surgery, their identification is vital for postoperative management. For instance, elevated preoperative HDL and significant intraoperative blood loss are indicators for intensive monitoring and intervention. These might include nutritional adjustments, optimization of physical therapy regimens, and stricter hematological surveillance to promote healing and mitigate the risk of nonunion. Moreover, CCD can be directly modified during the operation. Understanding the role of CCD in nonunion provides actionable insights for preoperative planning. When the CCD is significantly increased by analyzing radiological images, it suggests a potential reevaluation of surgical techniques to ensure optimal alignment and stability.

Enhancing External Validity

Acknowledging the high 3-month nonunion rate observed in our study, finally 6 cases had bone nonunion after extended follow-up and underwent secondary surgery. We are undertaking collaborative studies to validate these findings across multiple centers with varied 3-month nonunion incidences. This initiative will help determine whether the identified predictors hold universally or if their influence is context-specific. Such validation is crucial for enhancing the robustness and applicability of our predictive model in global clinical settings.

Limitations

This study is subject to certain limitations that warrant consideration in subsequent research. Firstly, the predictive model did not include functional outcomes, as another important indicator of delayed nonunion, besides radiological signs of nonunion. Secondly, similar to numerous retrospective analyses, our model’s efficacy requires prospective validation. Thirdly, prediction model was not developed based on 6 patients with final bone nonunion. Fourthly, the dataset for this research was sourced from single medical institution, which may not provide as comprehensive a perspective as a multicenter study would. Although external validation data were utilized, these originated from the same institution too, lacking validation cohorts from diverse regions and nations. Subsequently, additional datasets and prospective multicenter clinical trials are essential to confirm the reliability of our findings and model.

Conclusion

In summary, our retrospective analysis demonstrates that ML-particularly the CatBoost model-can effectively predict 3-month nonunion in patients with unstable distal clavicle fractures treated with ORIF. Key predictors identified (CCD, HDL levels, and intraoperative blood loss) offer actionable insights that can inform intraoperative adjustments and postoperative management. Integrating these predictive models into clinical practice could potentially reduce nonunion rates and improve patient outcomes by enabling more personalized treatment strategies.