Forecasting Hospitalization for Adult Asthma Patients in Emergency Departments Based on Multiple Environmental and Clinical Factors

Asthma is a severe global health issue and ranks as the world’s second most common chronic respiratory disease,¹ affecting 1–18% of the population in various nations.² In China, the overall prevalence of asthma in adults over the age of 20 is estimated at 4.2%, representing 45.7 million individuals. Among these, 15.5% have visited the emergency department (ED) at least once in the past year due to worsening respiratory symptoms.³ Following the COVID-19 pandemic, the demand for emergency medical services has surged, intensifying the challenges faced by emergency medical systems. Asthma-related visits have exacerbated the shortage of emergency medical resources. Although most asthma exacerbations are managed in outpatient settings or EDs, severe cases may require hospitalization, significantly impacting healthcare expenses.^4–7 Therefore, prompt identification of patients needing hospitalization is necessary to reduce ED boarding times,⁸ optimize resource allocation, and ensure early and intensive care for those at a high risk of admission.

With the growing availability of large data sources and advancements in computational power,^9–11 predicting the risk of hospitalization for asthma has entered a new era. Zein et al¹² employed a machine learning (ML) approach, finding that the light gradient boosting algorithm was optimal for predicting hospitalization for asthma in outpatients. Patel et al¹³ concluded that the gradient-boosting algorithm surpassed other algorithms in predicting pediatric asthma patient hospitalizations in the ED. Similarly, Goto et al¹⁴ reported that the random forest (RF) algorithm demonstrated the highest discriminative ability and sensitivity in predicting hospitalization for adult asthma patients in the ED.

However, although meteorological factors and airborne pollutants are recognized as asthma risk factors,^15–18 no study has yet combined clinical, meteorological, and air pollution data to predict hospitalization for adult asthma patients in EDs. This study aims to fill this void. We believe that this approach will enable clinicians to determine quickly whether asthma patients require hospitalization, and thus will reduce ED boarding times, accelerate patient recovery, and promote rational use of emergency medical resources.

Materials and Methods

Study Design and Setting

A retrospective analysis was conducted utilizing data from the electronic health records (EHRs) of a single healthcare system. The study protocol received approval from Peking University Third Hospital Medical Science Research Ethics Committee (Approval No: IRB00006761-M2021582).

Study Samples

All ED visits by adult patients (aged ≥ 18 years) diagnosed with asthma by a physician from November 1, 2016, to July 1, 2023, were included. Pregnant women and patients without EHRs during their ED visits were excluded.

Predictors

Input variables included demographic information, vital signs, laboratory test results, comorbidities, and initial illness severity at triage. If multiple laboratory tests were conducted during ED visits, only the results from the first test were used. The severity of the initial illness was assessed using the Chinese Emergency Triage Scale (CETS), which categorizes the urgency of a patient’s condition on a four-point scale, with one indicating the highest urgency.¹⁹ Covariates were selected based on biological plausibility and evidence from prior research.

Additional input variables included external environmental factors. Concentrations of ambient air pollutants (O₃ and PM_2.5) were sourced from Tracking Air Pollution in China (TAP), a near real-time air pollutant concentration database.²⁰ Concentrations of other pollutants (CO, NO₂, SO₂, and PM₁₀) were obtained from ChinaHighAirPollutants (CHAP), which provides long-term, comprehensive, high-resolution, and high-quality datasets of ground-level air pollutants in China.^21,22 Meteorological variables, including temperature and relative humidity, were derived from the fifth generation of European ReAnalysis (ERA5), provided by the European Centre for Medium-Range Weather Forecasts (ECMWF).²³ Geocoding of each subject’s residential street into latitude and longitude was performed using the programming interface of the mapping application Amap (AutoNavi Software). Specifically, by matching geographic coordinates and the dates of the visit to the ED, we obtained the initial 32 environmental variables, including levels of PM_2.5, O₃, CO, NO₂, SO₂, and PM₁₀ at 24, 48, 168, and 336 hours prior to the patients’ visits to the ED (hereafter designated as 24 h prior, 48 h prior, 168 h prior, and 336 h prior, respectively).

Data Pre-Processing

The data pre-processing procedure encompassed outlier detection, handling of missing values, standardization, and re-sampling to address class imbalances.

A multivariate model approach known as “Cook’s distance”²⁴ was employed to identify outliers, which were removed from the dataset as necessary. Missing values were imputed using the median statistic for each respective column. To mitigate the impact of varying scales among different features on model training, the values of each continuous variable were standardized to a range of 0 to 1. After these pre-processing steps, the dataset comprised 1140 observations and 54 variables, including 22 base in-hospital variables and 32 external environmental factors. The initial dataset revealed that 21.8% of the samples belonged to the hospitalization class, indicating a class imbalance. To enhance the model’s performance across both classes, a technique called SMOTEENN²⁵ combining undersampling and oversampling was applied to balance the dataset.

Feature Engineering

The initial input variables captured conditions at specific instants but lacked information on dynamic changes. To address this and further explore the impact of environmental conditions on asthma severity, the study incorporated feature engineering of environmental factors. Specifically, for each of the environmental factors, we calculated the difference in values at 24, 48, 168, and 336 h prior. These differential variables enabled the examination of the dynamic influences of environmental factors on asthma. Consequently, the final dataset was expanded to include 102 variables, encompassing 48 new differential variables.

Statistical Analysis

The dataset was randomly partitioned into two subsets: (1) a training dataset comprising 80% of the patients, utilized for model training, and (2) a test dataset consisting of 20% of the patients, used to evaluate the final performance of the model. The proportion of positive cases was maintained equally in both the training and test datasets. A five-fold cross-validation procedure was applied to each model within the training dataset. The model performance was assessed using the test dataset.

The performance of the models was evaluated by calculating the area under the receiver operating characteristic (ROC) curve (AUC), sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV).

Predictive models were developed using various techniques, specifically the extreme gradient boosting (XGBoost), light gradient boosting machine (LightGBM), support vector machine (SVM), logistic regression (LR), and RF techniques. To enhance the interpretability of the results, the Shapley additive explanations (SHAP)²⁶ approach was employed. This method represents the average contribution of each feature to the overall predictions made by the model through its mean absolute SHAP value. Additionally, the cumulative effect study method was utilized to further improve model interpretability. Figure 1 provides a comprehensive overview of the procedures established for each predictive model, covering data pre-processing, feature engineering, model training, and evaluation.

Figure 1 Overview of Model Design.

The research was conducted using Python v3.12.4 with the following key dependency versions: scikit-learn v1.5.2 for ML model development and pipeline construction, XGBoost v2.1.4 for gradient boosting implementation, pandas v2.2.3 for data processing and manipulation, NumPy v2.0.2 for fundamental numerical operations, and SHAP v0.46.0 for explainable analysis.

Outcome Measures

The outcome measure in this study was the hospitalization event, defined as admission to a respiratory ward, an internal medicine ward, or an intensive care unit. A binary outcome (Yes/No) was determined based on whether a patient experienced a hospitalization event during the study period.

Results

Patient Characteristics

Between November 1, 2016, and July 1, 2023, a total of 1507 patients with asthma met the inclusion criteria for the study. From this group, exclusions were made for 293 patients who lacked electronic medical records during ED visits, 44 patients who were return visitors to the ED, and 30 pregnant patients (e-Figure 1). Consequently, 1140 ED visits were included in the analysis.

The median age of the included patients was 51.0 years, with an interquartile range of 31.0 to 67.0 years; 56.5% (644 patients) were female. After receiving ED treatment, 21.8% of the patients (249) required hospitalization. The epidemiological data of the patients included in the study are detailed in Table 1.

Table 1 Epidemiological Data of Patients

Hospitalization Without External Environmental Factors

The models were initially trained and evaluated using 22 in-hospital variables. Table 2 illustrates the prediction performance of five models on the training dataset, assessed via five-fold cross-validation. XGBoost achieved the highest mean sensitivity, although RF exhibited a slightly superior maximum sensitivity of 0.558 (compared with 0.5512 for XGBoost). Both SVM and LR demonstrated specificities greater than 0.96. The PPV for LR and RF exceeded 71%, while XGBoost and LightGBM presented NPVs better than 0.85. All models yielded relatively low F1-Score results, with SVM registering the lowest at 0.3. The RF algorithm achieved high accuracy and AUC (both exceeding 0.82); however, it encountered challenges in terms of high variance and slightly reduced stability.

Table 2 Comparison of Receiving Operator Characteristic Curves of Five-Fold Cross-Validation Based on Training Data

Table 3 compares the prediction performance of the five models based on the test dataset. RF performed at high levels on all metrics except Specificity and PPV, achieving an optimal AUC of 0.8272 compared with the other models. Figure 2 presents the ROC curve for each model.

Table 3 Comparison of Receiving Operator Characteristic Curves Based on Test Data

Figure 2 Receiver Operating Characteristic (ROC) Curves for Models Excluding External Environmental Factors. This figure compares the performance metrics Negative Predictive Value (NPV), Positive Predictive Value (PPV), and Area Under the Curve (AUC) for various models: Extreme Gradient Boosting (XGB), Light Gradient Boosting Machine (LightGBM), Support Vector Machines (SVM), Logistic Regression (LR), and Random Forest (RF).

Cumulative Effect of Environment Variables

We sequentially integrated ambient air pollutant and meteorological features to enhance the RF model’s predictive capability. Initially, the 22 in-hospital variables formed the baseline. We then progressively added environmental variables at different time points, incorporating eight variables in each phase to achieve total counts of 30, 38, 46, and 54 variables. To minimize the impact of random variation and enhance robustness, the model was trained five times for each configuration.

Figure 3 illustrates the trends in cumulative effects as environmental characteristics were integrated at various time intervals. The blue dashed line represents the mean of five AUC scores, while the light blue shaded area indicates the range of the means ± standard deviations. Notably, the cumulative effect reached its peak at 168 h prior, indicating the optimal integration of temporal environmental data for predicting hospitalization. Based on this finding, the subsequent phase of model training included a total of 46 features.

Figure 3 Trends in the Cumulative Effects of Environmental Features at Different Time Points. This figure depicts the influence of environmental factors introduced at different times on the model’s performance. Baseline (Base) represents predictions without environmental data. Different times are specified as 24 h, 48 h, 168 h, and 336 h prior to the ED visit.

Hospitalization with External Environmental Factors

Based on the findings outlined in the previous section, the input dataset for the analysis was composed of 46 variables, which included 22 hospitalization and 24 environmental variables. Following the feature engineering processes detailed in the Data Pre-processing section, an additional 48 features were incorporated to capture the dynamics of the environmental conditions. As a result, the total number of variables used in the analysis increased to 94.

Table 4 presents a comparison of the prediction performance of five models based on the training dataset, assessed through five-fold cross-validation. Both SVM and LR exhibited the lowest values for Sensitivity, NPV, F1-Score, Accuracy, and AUC. Table 5 details the prediction performance comparison of the five models based on the test dataset. As shown in Table 5, the RF model consistently outperformed the others across all evaluated metrics, achieving an accuracy of 0.8491 and an AUC of 0.8555. Consequently, the RF model is established as the optimal model for this study. Figure 4 displays the ROC curve for each model, further illustrating the comparative performance of the predictive models. The hyperparameters and configurations of all models are summarized in Table 6.

Table 4 Comparison of Prediction Performance Across Five Models Using Training Data and External Environmental Factors

Table 5 Comparison of Prediction Performance Across Five Models Using Test Data and External Environmental Factors

Table 6 Hyperparameter Combination of Each Model

Figure 4 Receiver Operating Characteristic (ROC) Curves for Models Incorporating External Environmental Factors. This figure displays ROC curves for models enhanced with environmental data. Abbreviations: XGB, Extreme Gradient Boosting; LightGBM, Light Gradient Boosting Machine; SVM, Support Vector Machines; LR, Logistic Regression; RF, Random Forest; NPV, Negative Predictive Value; PPV, Positive Predictive Value; AUC, Area Under the Receiver Operating Characteristic Curve.

SHAP Analysis of RF Model

Figure 5 displays the SHAP value distribution for the top 20 features, ranked by importance and shown on the vertical axis to the left, with the SHAP values plotted on the horizontal axis. The color bar on the right indicates the actual values of the features. The analysis reveals that CETS is the most significant feature, where lower values are strongly predictive of hospitalization. This is followed by oxygen saturation and age, then heart rate, lymphocytes, and eosinophils. Among the top 20 features, three engineered attributes are presented: the difference in NO₂ levels 48 h and 168 h prior, the difference in PM₁₀ values 24 h and 48 h prior, and the difference in CO values 24 h and 48 h prior. These attributes represent changes in pollutant levels, where higher values (closer to red) correlate with increased pollution and a greater predicted likelihood of hospitalization. The inclusion of these engineered features in the top 20 highlights their practical value in enhancing the model’s predictive accuracy.

Figure 5 Prediction Model Using Random Forest (RF). This figure outlines the importance of different features in the RF model for predicting hospitalizations. Features are ranked by the magnitude of their SHAP values; higher absolute values indicate greater influence. Conditions include COPD, Hypertension, and Allergic Rhinitis (coded as 1 for present, 0 for absent). Abbreviations: CETS, Chinese Emergency Triage Scale; COPD, Chronic Obstructive Pulmonary Disease; NO2, nitrogen dioxide; PM10, particulate matter with an aerodynamic diameter ≤10 μm; CO, carbon monoxide; ΔNO2, difference in values of NO2 concentrations at 48 h and 168 h before the ED visit; ΔPM10, difference in values of the PM10 concentrations at 24 h and 48 h before the ED visit; ΔCO, difference in values of the CO concentrations at 24 h and 48 h before the ED visit.

Figure 6 illustrates the relative risk of hospitalization associated with the top three features and a significant air pollution attribute, as follows:

Figure 6 Relative Risk of Different Outcomes with Independent Variables in Hospitalization Prediction Models. (A) Relative risk of different outcomes with CETS; (B) Relative risk of different outcomes with Oxygen saturation; (C) Relative risk of different outcomes with Age; (D) Relative risk of different outcomes with ΔNO2. Abbreviations: CETS, Chinese Emergency Triage Scale; ΔNO2, the difference in values of the NO2 concentrations at 48 h and 168 h before the ED visit.

(a) CETS (Figure 6a): This is a crucial feature in assessing the risk of hospitalization for asthma. A CETS score below 2 significantly increases the likelihood of hospitalization, indicating a high risk.

(b) Oxygen Saturation (Figure 6b): This is another crucial indicator. A saturation level of 93% is identified as the critical threshold. The blue dots in the figure represent patients with oxygen saturation below this level, indicating a heightened risk of hospitalization.

(c) Age (Figure 6c): This plays a significant role in hospitalization risk, with 58 years identified as a critical turning point beyond which the probability of requiring hospitalization increases.

(d) NO₂ (Figure 6d): This feature represents the difference in the NO₂ concentrations at 48 h and 168 h prior. A differential value of 0 is the turning point, with values exceeding this mark significantly raising the probability of hospitalization. This indicates that increases in NO₂ levels over time correlate with a higher risk of hospitalization.

Top-20-Feature Cumulative Effect

To examine the impact of different variables on our prediction model, we utilized cumulative feature effect methods. We began by identifying the top 20 features through a ranking based on both SHAP values and the inherent feature importance of the RF model. The experiment involved removing these top 20 features from a set of 94, then training five base models and calculating the mean values of the metrics to establish the base values. Features were incrementally reintroduced in ascending order of importance, undergoing five training sessions and subsequent metric evaluations. This approach yielded 21 sets of outcomes, with feature counts progressively increasing from 74 to 94, as depicted in Figure 7. We applied min-max scaling to normalize each metric for enhanced visualization. The normalized results were plotted along the horizontal axis, with zero as the midpoint; the left side depicted features selected based on SHAP values, and the right side depicted those based on the importance of the RF features.

Figure 7 Top-20-Feature Cumulative Effect. Abbreviations: CETS, Chinese Emergency Triage Scale; COPD, Chronic Obstructive Pulmonary Disease; NO2, nitrogen dioxide; PM10, particulate matter with an aerodynamic diameter ≤10 μm; CO, carbon monoxide; ΔNO2, difference in values of the NO2 concentrations at 48 h and 168 h before the ED visit; ΔPM10, difference in values of the PM10 concentrations at 24 h and 48 h before the ED visit; ΔCO, difference in values of the CO concentrations at 24 h and 48 h before the ED visit; ΔSO2, difference in values of the SO2 concentrations at 48 h and 168 h before the ED visit.

The diagram shows that the top three features—CETS, oxygen saturation, and age—remained consistent across both model interpretation methodologies, highlighting their strong correlation with hospitalization for asthma. The primary difference between the methodologies was observed in the variable Allergic Rhinitis and in the difference in SO₂ levels at 48 h and 168 h prior. Notably, the difference was significant in the classification process of the RF model, affirming the importance of this engineered feature.

Discussion

In this study, prediction models were initially trained and evaluated using 22 in-hospital variables. RF performed at high levels on all metrics except Specificity and PPV, achieving an optimal AUC of 0.8272. We sequentially integrated ambient air pollutant and meteorological features to enhance the model’s predictive capability. And the RF model consistently outperformed the others, achieving an AUC of 0.8555. To the best of the authors’ knowledge, this is the first study to use multiple sources of data to construct predictive models that assist clinicians in identifying adult asthma patients at high risk of hospitalization.

Over the past decade, the expansion of large clinical data sources and advances in computational power have fueled the growth of ML applications in healthcare.⁹ The line between ML and statistical modeling has been described as a continuum.⁹ Unlike statistical models that rely on theory and assumptions, ML models learn directly and automatically from data.²⁷ No definitive threshold exists at which a model is considered to be a “machine learning model”, as all methods exist on a continuum depending on the degree of human assumptions imposed.⁹ Compared with traditional methods, ML algorithms enable a more flexible relationship between predictor variables and outcomes, rendering them preferable for predictive rather than explanatory studies.¹⁰ Our study demonstrated that the RF algorithm exhibited superior performance in predicting hospitalization outcomes among adult asthma patients presenting to the emergency department. The algorithm’s effectiveness stems from its ensemble approach, which utilizes multiple decision trees to capture complex hierarchical relationships and interaction effects among variables. This enables it to handle high-dimensional datasets with collinear features proficiently, as seen in our study integrating clinical, meteorological, and air pollution data.

ML models are often called “black boxes”, as they input data and produce outcomes without visible internal processes.²⁸ To address this issue, the SHAP method was used in this study to explain the outcomes of the prediction model. Emergency triage is crucial in high-acuity EDs, and since the 1990s, several triage scales have been implemented in developed countries. In China, emergency triage faces unique challenges compared to developed countries.¹⁹ CETS, which relies on vital complaints and objective data, is a reliable and valid method for ED triage in mainland China.¹⁹ This study corroborated previous findings that lower CETS scores, indicating greater illness severity, were associated with a higher risk of hospitalization for asthma. Consistent with earlier research,⁸ initial oxygen saturation was also a significant predictor of hospitalization.^13,14,29 Older age, which is linked to reduced lung function and a higher prevalence of chronic disease comorbidities, increased the likelihood of hospitalization.^2,30,31 Eosinophil levels typically drop during severe asthma exacerbations,¹² because the strong inflammatory response recruits eosinophils to the airways and tissues. Air pollution factors, notably NO₂ and PM₁₀, have been significantly associated with asthma exacerbations,^32,33 potentially because of oxidative stress, airway hyper-responsiveness, and remodeling.^34–38

The rapid development of electronic medical record systems presents a unique opportunity to create clinical decision support systems (CDSS). These are computer programs that apply knowledge to data stored in EHRs to facilitate intelligent applications such as auxiliary diagnosis, treatment assistance, risk warning, medical record quality control, and infectious disease management; they have proved to be a valuable and effective tool in clinical practice.^39–41 A previous study developed a CDSS by integrating EHRs with BMJ Best Practice, achieving accuracy rates of 75.46% in first-rank diagnosis and 87.53% in top-three diagnoses;⁴² this highlights the significant clinical application potential of CDSS in EHRs.⁴² The predictive models developed in this study could be embedded in a CDSS to guide clinicians in making informed decisions about hospitalizing asthma patients and to enhance resource utilization.

This study has several limitations. First, the sample size was limited, and future research will require larger samples to enhance model accuracy. Second, data on ambient air pollutants and meteorological variables may not fully reflect individual exposure levels due to the omission of activities conducted away from home. Finally, the retrospective design of this study is constrained by missing data; for instance, only 31.6% of patients had recorded respiratory rate data, a factor associated with hospital admission.¹³

In conclusion, ML models based on clinical, meteorological, and air pollution data can rapidly and accurately predict hospitalization of adult asthma patients in EDs. This study lays a foundational framework for future research where similar methodologies can be applied to other chronic conditions influenced by environmental factors. Future work will focus on incorporating these models into clinical decision tools to assist clinicians in determining the need for hospitalization of asthma patients and to improve resource utilization for maximizing ED efficiency.