Machine Learning-Based Mortality Risk Prediction Model in Patients with Sepsis

Introduction

Sepsis is a systemic inflammatory response syndrome caused by infection, characterized by rapid progression, which often leads to multiple organ failure and can be life-threatening in severe cases.¹ Elderly patients, due to immunocompromised states and the presence of multiple comorbidities, experience a significantly increased mortality rate from sepsis.²

Early and accurate assessment of sepsis severity in elderly patients, followed by timely and proactive therapeutic interventions, is crucial for improving patient outcomes.^3,4 Currently, clinical biomarkers commonly used to assess sepsis severity include C-reactive protein (CRP) and procalcitonin (PCT).^2,5,6 These markers play a role in traditional evaluation models; however, these models have limited calibration and relatively low predictive performance. Furthermore, international sepsis guidelines emphasize the importance of early identification and timely intervention, recommending the use of various biomarkers and scoring systems to enhance diagnostic and predictive accuracy. Cytokines such as IL-6, IL-8, and vascular endothelial growth factor (VEGF) are significantly elevated in sepsis patients and correlate with disease severity. Research indicates that combining these novel markers with traditional markers can significantly enhance the predictive accuracy for sepsis.^7,8

In recent years, advancements in statistical methods and computational technologies have led to the development of advanced clinical prediction models.^9–11 These machine learning methodologies demonstrate higher accuracy and efficiency in sepsis assessment.¹² For instance, LASSO regression models and nomogram-based predictive models have shown superior performance compared to traditional models in several studies.^13,14

Therefore, our study aims to develop and validate a new Machine learning-based mortality risk prediction model for sepsis by integrating advanced statistical methods with clinical data, incorporating both traditional and novel biomarkers, as well as scoring systems. We anticipate that the improved machine learning model will allow for more accurate identification of high-risk patients at an early stage, facilitating early intervention and ultimately improving clinical outcomes for elderly sepsis patients.

Materials and Methods

Study Population

Our study was conducted at Beijing Chao-Yang Hospital, Capital Medical University, from August 2021 to August 2023. Our study complied with the Declaration of Helsinki and received approval from the Ethics Committee of Beijing Chaoyang Hospital (Approval No.: 2021-S-636). Written informed consent was obtained from all patients or their families, and patient information was kept strictly confidential. The inclusion criteria for our study were patients aged ≥18 years who met the diagnostic criteria for sepsis 3.0 and were admitted to the emergency department resuscitation room.¹ The exclusion criteria were as follows:

1. Patients treated in the emergency department for less than 24 hours or those transferred from other hospitals. 2. Patients with a history of acute or chronic renal failure. 3. Trauma patients. 4. Patients with a history of malignancy, hematologic diseases, or connective tissue diseases.

Data Collection

Data were extracted using a structured query language.

Patient Demographics

Basic patient information: ID number, gender, and age.

Vital Signs and Laboratory Parameters

Vital signs: body temperature, heart rate (HR), mean arterial pressure (MAP), and respiratory rate (RR). Laboratory parameters: white blood cell count (WBC), hemoglobin (HGB), hematocrit (HCT), platelet count (PLT), blood pH, partial pressure of oxygen (PaO₂), partial pressure of carbon dioxide (PaCO₂), lactate (LAC), blood urea nitrogen (BUN), creatinine (CR), Na⁺, K⁺, total bilirubin (TBIL), aspartate aminotransferase (AST), alanine aminotransferase (ALT), albumin(ALB), PCT, CRP, oxygenation index (OI), and ejection fraction (EF). Novel serum biomarkers: Ferritin, Vascular Endothelial Growth Factor (VEGF), IL-6, IL-7, IL-8.

Disease-Related Scores

Sequential Organ Failure Assessment (SOFA), Acute Physiology and Chronic Health Evaluation II (APACHE II).

Major Comorbidities and Treatments

Heart failure, cardiovascular diseases, pulmonary diseases, renal diseases, diabetes, hypertension, and long-term bedridden (LTB).

Endpoint Events

The 28-day mortality rate was recorded for all patients. Follow-up was conducted to document survival status at 28 days.

A total of 38 variables were analyzed. For variables measured multiple times, only the first measurement was included in the analysis.

Model Development, Validation, and Statistical Analysis

Statistical analysis and model development were performed using R software (version 4.3.2) to compare the relationship between monitoring variables and the prediction of 28-day mortality in sepsis patients. For normally distributed data, t-test were performed, and the variables were described using mean and standard deviation. Conversely, for non-normally distributed variables, the Wilcoxon rank-sum test was used, and descriptive statistics (median and interquartile range) were employed to describe the variables. Categorical variables were expressed as numbers and percentages and compared using the chi-square test.

Variables were screened through univariate analysis and LASSO regression. The discriminative ability of the model was evaluated using receiver operating characteristic (ROC) curve analysis. A nomogram model was developed using multivariable logistic regression. The accuracy of our model was further verified by bootstrap validation using computer resampling for 1000 repetitions of simple random sampling with replacement. Calibration curves were used to assess the consistency between the model’s predicted probabilities and the actual outcomes. P < 0.05 was considered statistically significant.

Results

Comparison of Basic Clinical Data Between Survivors and Non-Survivors in Sepsis Patients

A total of 180 sepsis patients were included in this study (Table 1). The survivor group consisted of 60 patients (33.3%), including 39 males (65.0%) and 21 females (35.0%), with an average age of 71 (64, 83) years. The non-survivor group included 120 patients (66.6%), with 71 males (59.0%) and 49 females (41.0%), with an average age of 76 (66, 83) years. Compared to the survivor group, the non-survivor group showed statistically significant differences in SOFA score, APACHE II score, MAP, ALB, LTB, and VEGF levels (P < 0.05) (Table 1).

Table 1 Comparison of Patient Baseline Data

Establishment of LASSO Regression and Predictive Model

Pearson correlation analysis revealed that several variables had correlation coefficients greater than 0.5, indicating a high degree of correlation and the presence of multicollinearity, which could distort or make accurate estimation of the model challenging (Figure 1). Therefore, LASSO regression was employed for variable selection, adding a penalty term to shrink the regression coefficients. Through ten-fold cross-validation, when λ = 0.044, seven factors were selected: SOFA, APACHE II, MAP, ALB, PCT, VEGF, and LTB status (Figure 2). The predictive model was constructed using the LASSO-logistic regression equation:

Figure 1 Correlation heatmap of variable differences. The heatmap shows the correlation coefficients between different variables. Blue indicates a positive correlation and red indicates a negative correlation. The size of the circles represents the strength of the correlation, with larger circles indicating stronger correlations. Asterisks denote the level of statistical significance: *p<0.05, **p<0.01, ***p<0.001.

Figure 2 (A) LASSO Coefficient Path Plot. The horizontal axis represents the lambda (λ) values, and the vertical axis represents the partial regression coefficients. Each line represents the trajectory of a variable’s coefficient. As the value of log (lambda) increases, some coefficients shrink to zero, indicating the exclusion of the corresponding variables from the model. (B) LASSO Cross-Validation Plot. This plot displays the binomial deviance of the model for different values of the regularization parameter (log lambda). Red dots represent the deviance for each log(lambda) value with error bars indicating the standard error. The lower horizontal axis represents the lambda (λ) values, the left vertical dashed line represents the lambda value with the minimum binomial deviance, and the right vertical dashed line represents the largest lambda value within one standard error of the minimum deviance. The upper horizontal axis represents the number of variables, and the vertical axis represents the binomial deviance.

The prognosis probability for sepsis patients can be estimated using a nomogram (Figure 3). The performance of this nomogram was measured using ROC curve analysis, with the model achieving an area under the ROC curve (AUC) of 0.845 (95% CI: 0.786, 0.905). The model demonstrated good diagnostic performance, with a Brier score of 0.148, sensitivity of 75.9%, and specificity of 85.0% at the optimal cut-off point (Figure 4).

Figure 3 Nomogram for Predicting 28-Day Mortality Risk in Elderly Patients with Sepsis. This nomogram predicts the 28-day mortality risk based on clinical variables. To use the nomogram, locate the patient’s value for each variable on the corresponding axis. Draw a vertical line from this value to the “Points” axis to determine the individual points for that variable. Sum the points for all variables to obtain the total points. Finally, draw a line from the total points axis down to the lower line of the nomogram to determine the 28-day mortality risk probabilities.

Figure 4 Receiver operating characteristic curve. Abbreviation: AUC, Area under the receiver operating characteristic curve.

Model Validation

The model was further validated using internal bootstrap methods. The ROC curve was measured through 1000 bootstrap repetitions, resulting in a bootstrap-corrected AUC of 0.857 (95% CI: 0.799, 0.912), indicating similar statistical power to the initial model (Figure 5). The internal bootstrap validation calibration curve revealed that the nomogram-derived curve might overestimate the risk of prognosis at probabilities of 0–0.4 and 0.7–0.9, and underestimate probabilities at 0.4–0.7. Overall, our model exhibited good fit and calibration compared to the ideal curve. Furthermore, decision curve analysis showed that the predictive model had a good positive net benefit for threshold probabilities below 0.9, suggesting that the model has good potential clinical utility (Figure 6).

Figure 5 Internal validation of the nomogram using the bootstrap sampling. The ROC curve was measured by bootstrapping for 1000 repetitions, and the AUC of the bootstrap stepwise model was showed.

Figure 6 (A) Calibration curve for predicted probability. This calibration plot evaluates the accuracy of the predictive model by comparing the predicted probabilities of 28-day mortality with the actual observed probabilities. (B) Decision curve analysis for the prediction model. This decision curve analysis evaluates the clinical usefulness of the predictive model by comparing the standardized net benefit across different high-risk thresholds. The horizontal axis represents the high-risk threshold, while the left vertical axis represents the standardized net benefit. The blue line (Model) shows the net benefit of using the predictive model, the grey line (All) represents the net benefit of treating all patients, and the black horizontal line (None) indicates the net benefit of treating no patients.

Discussion

Sepsis is a severe organ dysfunction caused by a dysregulated host response to infection and is potentially life-threatening. This condition involves a systemic response, which may be associated with an excessive increase in cytokines and other inflammatory mediators released into the systemic circulation during the disease course.^1,15 The symptoms and signs of sepsis vary depending on the specific organ systems affected, such as altered mental status in the nervous system, hypoxemia in the respiratory system, shock in the cardiovascular system, reduced urine output and/or elevated serum creatinine levels in the renal system, thrombocytopenia in the hematologic system, and abnormal liver function.^16–18 Organ failures, particularly acute kidney injury (AKI), are independent risk factors for increased mortality in hospitalized patients.^19,20

In recent years, data mining and machine learning have been widely applied in clinical settings, aiding emergency intensive care physicians in making more precise clinical decisions. Our team has successfully developed a clinical prediction model for estimating the probability of AKI in ARDS patients, which was well-validated using the MIMIC-IV dataset.²¹ In the current study, we analyzed clinical data from 180 emergency sepsis patients, encompassing a total of 38 potential variables. Due to the aging population and the different types of patients in regional hospitals, we found that over 90% of patients diagnosed with sepsis were middle-aged and elderly. Therefore, the average age of patients in our study was 74 years (95% CI:65, 83), reflecting the predominance of middle-aged and elderly patients in our sample. Using machine learning algorithms, we further predicted the 28-day mortality rate in elderly sepsis patients and developed an efficient clinical prediction model that includes seven features: SOFA, APACHE II, MAP, ALB, PCT, LTB, and VEGF. The stepwise regression method demonstrated that the model has good diagnostic performance, with an AUC of 0.845 and a Brier score of 0.148 (Figure 4). At the optimal cut-off point, the sensitivity was 75.9% and the specificity was 85.0%. Internal bootstrap validation further confirmed that its statistical power is consistent with the initial model. Decision curve analysis indicated that the model has good potential clinical utility.

Among the seven features of the model, VEGF is one of the newly introduced markers and plays a significant role. VEGF is a highly specific pro-angiogenic factor that promotes increased vascular permeability, extracellular matrix degradation, endothelial cell migration, proliferation, and angiogenesis.^22,23 The activation and dysfunction of endothelial cells are critical factors in the progression of infection to sepsis. Endothelial cell injury can lead to vascular leakage and edema, as well as severe complications such as shock, micro thrombosis, and multiple organ failure (MOF).²⁴ Studies have shown that damage to the endothelial barrier affects the regulation of leukocyte migration and invasion, impacting the process of leukocyte migration to sites of inflammation. Moreover, disruption of the endothelial barrier results in the leakage of intravascular components into the extravascular space, causing tissue and organ edema and leading to insufficient organ perfusion.^1,25,26 In an animal model study, septic mice induced by cecal ligation and puncture (CLP) showed significantly elevated plasma VEGF levels. This increase was associated with the secretion of VEGF by pulmonary endothelial cells through various MAPK-dependent pathways, promoting the development of non-cardiogenic pulmonary edema in sepsis-related acute lung injury (ALI).²³ A meta-analysis by Tang et al of 20 studies involving 2242 patients indicated that baseline VEGF levels were higher in non-survivors and critically ill patients compared to survivors and non-critically ill patients, suggesting that high VEGF levels are associated with poor clinical outcomes in sepsis patients.²⁷ Our findings are consistent with these reports.

Mean arterial pressure (MAP) is a crucial hemodynamic parameter that reflects tissue perfusion throughout the body. Maintaining an appropriate MAP is essential for the prognosis of sepsis patients.^3,28,29 Increasing MAP generally enhances tissue perfusion and oxygen delivery. Organs such as the brain and kidneys possess autoregulatory functions, where perfusion and MAP display a linear relationship. When MAP falls below a certain threshold, perfusion decreases.¹⁵ In this study, MAP is a significant feature in the predictive model. Low MAP in sepsis patients leads to inadequate tissue and organ perfusion, resulting in cellular hypoxia and metabolic disturbances. This triggers the release of inflammatory mediators and increases vascular permeability, potentially causing multiple organ dysfunction syndrome (MODS) and even death. An analysis of 14,607 sepsis patients at Beth Israel Deaconess Medical Center found that MAP below 68.6 mmHg is a strong predictor of mortality, with each 10 mmHg increase in MAP reducing the risk of sepsis-related death by 52.1%.²⁸

Procalcitonin (PCT) is a widely recognized biomarker for sepsis evaluation. Despite its potential lack of sensitivity in predicting sepsis outcomes, PCT is a significant feature in our predictive model. PCT has been extensively used in clinical sepsis prediction, including neonatal sepsis, abdominal trauma sepsis, burn sepsis, and emergency elderly sepsis.^2,4,30 Albumin (ALB) is another critical factor in predicting patient outcomes. In sepsis patients, increased capillary leakage leads to the exudation of albumin and other components into the interstitial space, lowering plasma albumin concentration and colloid osmotic pressure. Additionally, endotoxin-induced inflammatory mediators inhibit hepatocyte albumin synthesis.^31,32 Intravenous albumin infusion is necessary and beneficial for correcting hypoalbuminemia in sepsis patients.^6,15 SOFA and APACHE II scores are vital for assessing changes in critically ill patients and organ function in emergency and ICU settings. LTB status is defined as continuous bed rest for more than seven days. The model achieved an AUC of 0.845 (95% CI: 0.786, 0.905), which is higher than what is typically reported for SOFA (0.677, 95% CI: 0.593, 0.761) and APACHE II (0.800, 95% CI: 0.731, 0.869) alone. This suggests that the model has the potential to outperform conventional risk scores in predicting 28-day mortality in elderly sepsis patients (Supplementary Figure 1).

The clinical prediction model developed in our study holds significant value for risk estimation, enhancing clinicians’ risk assessment of critically ill patients, and informing clinical decisions. We used stepwise regression to select seven features and establish a nomogram to predict the 28-day mortality risk in elderly sepsis patients. Although the study included only 180 patients, the model demonstrated relatively good performance. Qi et al constructed a clinical prediction model for sepsis prognosis in 5727 patients and found that the random forest model performed well in both training and validation sets, with age, albumin, and lactate as the most important features.²⁶ Similarly, Mirijello et al used various models to predict ICU sepsis prognosis and found that quick-SOFA, procalcitonin, MAP, and SOFA were key features, with the classic multivariable logistic regression model being the most concise and calibrated.

Furthermore, our model’s nomogram serves as a statistical tool to calculate the 28-day mortality risk for elderly sepsis patients, providing an early visual warning signal for clinical use. The nomogram is convenient and quick to use in clinical settings, offering valuable references for clinical decision-making and care, enabling timely interventions to improve patient outcomes. Overall, machine learning algorithms have the potential to enhance prediction accuracy and guide clinical decisions. With larger and more diverse datasets, the performance of these models is expected to improve further, providing valuable insights for clinicians managing this complex condition. Future research will include more models for comparative validation to better serve clinical needs.

There were several limitations. First, the relatively small sample size may weaken the analysis results. Second, the nomogram lacks robust external validation, although internal bootstrap validation and decision curve analysis indicate good net benefit. These results need to be confirmed in future studies with larger sample sizes and additional model validation. Third, the model has not been prospectively validated. We plan to conduct a multicenter prospective validation study of the sepsis prognosis prediction model. This will help to confirm the robustness and applicability of our model in different clinical settings.

Conclusion

The predictive model constructed with seven features—SOFA, APACHE II, MAP, ALB, PCT, LTB and VEGF—shows great potential in predicting the 28-day mortality risk for elderly sepsis patients. The visual nomogram is convenient and quick for clinical use, providing valuable references for clinical decision-making and care, enabling timely interventions to improve patient outcomes.