Dynamic Nomogram for Predicting the Fall Risk of Stroke Patients: An Observational Study

Introduction

In 2019, the annual number of new cases, total number of cases, and number of deaths from stroke were 12, 101 million, and 7 million, respectively.¹ The mortality caused by stroke rank the second worldwide.² Moreover, stroke is the leading cause of death and disability among adults in China, with an estimated 17.04 million people aged ≥40 years suffering from stroke. Meanwhile, disability-adjusted life years of stroke are much higher in China than in some developed countries.³

Stroke patients are more prone to falls compared to healthy person, due to reduced walking stability because of their inability to control trunk deflection and peak trunk velocity.⁴ Approximately 70% of stroke patients experience falls within 6 months after discharge,⁵ and approximately 27% of stroke patients have experienced recurrent falls within 1 year.⁶ Falls may not only lead to deterioration of clinical symptoms, increased dependence on care, increased financial burden, and even death,^7,8 but they may cause fear of falling (FOF), anxiety, depression, and other adverse consequences.^9,10 Studies have shown that the fastest early recovery of stroke patients occurs within 3 months,¹¹ followed by 3–6 months, and recovery can be maximized in 90% of patients within 6 months.¹² The adverse consequences of a fall can seriously affect the recovery and prognostic outcomes of patients. Therefore, it is particularly important for stroke patients to prevent fall during recovery.

Fall risk assessment is the first part of fall prevention. At present, although some researchers have used wearable inertial sensors or dual-force platforms to analyze the gait or posture of patients after stroke, so as to predict the fall risk of patients, most of the fall assessment tools used in stroke patients are traditional assessment scales due to high requirements for equipment.^13,14 For example, the Morse Fall Scale (MFS),¹⁵ Hendrich II Fall Risk Assessment Model (HFRM),¹⁶ St Thomas’s Risk Assessment Tool (STRATIFY),¹⁷ and John Hopkins Fall Risk Assessment Tool.¹⁸ Unfortunately, there are few studies on the predictive effect of HFRM and JHFARAT in the fall risk assessment of stroke patients, MFS and STRATIFY also show poor predictive performance in the fall risk assessment of stroke patients. For example, in China, Xu et al¹⁹ retrospectively analyzed 2093 cases using the MFS and found that the MFS alone could not accurately predict fall risk in stroke patients, there was no significant difference in MFS between the fall group and the non-fall group (P>0.05). Smith J et al²⁰ used the STRATIFY to assess the fall risks during rehabilitation in stroke patients; however, the results showed that STRATIFY underestimated the probability of falling in stroke patients.

The dynamic nomogram is an online scoring system generated based on a static column chart, which simplifies the traditional complex prediction-model formula into a visual form and can directly predict the probability of a fall in a patient at a certain point in time. It is fast and simple to operate with a clear interface. Presently, dynamic nomograms are used to predict the risk of various events and prognosis of patients, such as the risk of asthma, neonatal white matter damage, coronavirus-2019, and preoperative and postoperative peripheral lymphocyte difference in patients with hepatitis B virus-related hepatocellular cancer. Although, it is rarely used for fall risk assessment among stroke patients, there are also some nomogram prediction models for the prognosis of stroke patients at home and abroad that show good predictability and practicability, such as depression, mortality, nutritional risk, stroke-associated pneumonia, etc.^21–24 Dynamic nomograms can help health professionals predict the individual probability of relevant events in patients.^25–28 The dynamic nomogram is helpful for developing personalized measures for the treatment and prognosis of patients.

Summarily, a fall may hinder the rehabilitation progress of stroke patients, and the traditional fall risk assessment scale has a poor prediction effect on fall risk in stroke patient. A dynamic nomogram prediction model for the fall risk has not been developed, and a dynamic nomogram can quickly and intuitively predict fall risk in patients. Thus, the study aimed to develop and verify a dynamic nomogram fall risk prediction model for stroke patients during rehabilitation and provide a basis for the safe management of stroke patients.

Methods

This study has been registered in the Chinese Clinical Trial Registry (registration number ChiCTR2200057641).

Study Design and Participants

An observational study was conducted. Questionnaires and related functional tests were used during the last week of hospitalization. Patients or family members caring for the patients were followed up through telephone for fall events within 2 months after the patients were discharged from the hospital. Moreover, the Bootstrap method was used for the modelling population and internal validation of the model. Patients admitted to the neurology and rehabilitation wards of a tertiary hospital in Haikou City from March 2022 to September 2022 were enrolled using the convenience sampling method. The inclusion criteria were: (1) diagnosed with stroke; (2) within 6 months of disease onset;⁵ (3) aged ≥18 years; (4) being able to walk independently (with/without assistive tools) for at least 10 m; and (5) voluntarily participating the study. The exclusion criteria were: (1) severe visual impairment, resulting in the inability to see the road surface; (2) severe aphasia, hearing impairment, and cognitive dysfunction, resulting in the inability to cooperate with the survey; (3) neurological disorders or lower limb fracture resulting in motor dysfunction; (4) serious chronic diseases, such as heart failure, respiratory failure, and severe malnutrition.

Sample Size

According to the Guidelines for Reporting Specifications for Predictive Modelling Studies²⁹ and Risk of Bias Assessment Tool,³⁰ the number of events of the independent variable (Events Per Variable) should be at least 10. Whether a fall occurred or not was used as the dependent variable. Based on the literature, the maximum number of independent variables to be included in the prediction model was predicted to be 17, and the incidence of falls in patients within 6 months of discharge from the hospital was approximately 37%–73%,³¹ which was 40% of the estimated incidence of fall in stroke patients. Therefore, the sample size of the study was at least 425, and the final sample size was set at 472 after increasing the sample size by 10% to account for lost visits and invalid questionnaires.

Indicators and Measurements

Basic Information and Fall Risks in Stroke Patients During Rehabilitation

A questionnaire on the fall risks in stroke patients during rehabilitation was developed after literature research and two sessions of previous expert correspondence, which included the general and disease-related information of the patients. The general information includes the diagnosis, lesion site, sex, age, education level, marital and residential status, per capita monthly family income, height, weight, vision, and hearing of the patient. Disease-related information includes onset of stroke, risk of falling, number of episodes, side of limb hemiplegia, muscle strength, muscle tone, cognitive function, speech function, dizziness, visuospatial neglect, mobility aids, history of falls in the last 3 months, FOF, medications taken, presence of multiple comorbidities, urinary disturbances, pain, ability to perform daily living, anxiety and depressive conditions, balance function, and locomotor function. During the last week of hospitalization, the patients were instructed to fill out the questionnaires and complete relevant tests.

Cognitive Function

The cognitive function of stroke patients was evaluated using the Chinese version of mini-mental state examination (C-MMSE) scale. The C-MMSE translated and revised by Zhou Xiao-xuan in 2015 was used in this study. The Cronbach’s α was 0.833, and the test–retest reliability was 0.924.³² It covers the cognitive domains of orientation, immediate memory, attention and calculation, short-term recall, and language and visuospatial structural ability, with 11 entries and a total score of 30. Depending on the literacy level of the participants, the scale was classified as follows: patients whose education background are illiteracy, primary school, and junior high school, with score no more than 17 points, 20 points, and 24 points, respectively, are considered cognitive impairment.

Activities of Daily Living

The activities of daily living (ADL) of the patients were assessed using the modified Barthel index (MBI). MBI was developed in 1989 by Shah et al based on the initial Barthel index.³³ The Chinese version of the MBI was translated in the late 1990s by Hong Kong Polytechnic University.³⁴ It includes 10 items related to grooming, bathing, eating, dressing, bowel and urinary control, toileting, bed and chair transfers, ambulation, use of a wheelchair, and walking up and down stairs. Each item is divided into five grades, and each grade has a corresponding score (15/12/8/3/0; 10/8/5/2/0; 5/4/3/1/0). A higher score indicates the stronger the independent ability of a patient. The scale has a total score of 100 points and a score of ≥60 points indicates that the patient can take care of himself/herself. The MBI scale has a good interclass correlation coefficient rating of 0.968–0.997, internal reliability intra-class correlation coefficient of 0.866–0.990, and reliability and sensitivity, and can effectively evaluate the ability to perform ADL in stroke patients.³⁵

Mental State

The Hospital Anxiety and Depression Scale (HADS) was used to assess the mental state of the participants. The HADS was derived from a study by Zigmond A S and Snaith RP in 1983.³⁶ The Chinese version of the HADS was translated by Wang.³⁷ The HADS consists of 14 items. Seven odd-numbered items indicating anxiety (HAD-A) and seven even-numbered items indicating depression (HAD-D), with each entry scored 0–3. According to the recommendations of the original scale, a score of 0–7, 8–10, or ≥11 on the respective scale is considered negative, doubtful, and positive results, respectively. The total HADS had a Cronbach’s αcoefficient of 0.879–0.904, and the HADS has good reliability and validity.^38,39

Balance Function

Balance function was assessed using the Berg Balance Scale (BBS), which was reported by Katherine Berg in 1989 and is currently the most widely used balance scale.⁴⁰ The 14-item scale includes four aspects of standing up, sitting down, standing independently, and getting up. Each item was scored from “0” to “4” and the total score was 56 points. A higher BBS score indicated the better balance ability. The score with 0–20, 21–40, and 41–56 points suggesting poor balance function, some balance ability, and better balance function, respectively, and the score with <40 points predicting risk of falling.⁴¹ The BBS has a Cronbach’s α coefficient of 0.864 and good intrinsic reliability and concurrent validity in stroke patients.⁴²

Motor Function

Motor function was tested using the timed up and go test (TUGT).⁴³ Participants sat and leaned on the back of a 45-cm-high chair. After hearing the start message from the tester, they stood up as quickly as they could, walked to the 3-m marking line in front of them, turned around when they reached the marking line, returned to the chair, turned, and sat down. Time recording was stopped after the participants leaned their backs against the chairs. During the test, the participants were allowed to use a walking aid, and the tester was not allowed to give direct physical assistance but was present to ensure safety and prevent falls. The time that the participant to finish the test was recorded.

Data Collection

The research purpose and process of the survey were explained to the participants and their family members, and an informed consent form was signed before conducting the survey. During the last week of hospitalization, patients were instructed to fill out the questionnaires and complete relevant tests. The participants and their families were trained before the patients were discharged from the hospital to make sure they understood the definition of “fall” and judge fall events correctly.

Follow Up After 2 Months

The participants or family members were followed up to ask about fall events after the participants discharge from the hospital through telephone after 2 months. Fall is defined as the body position all of a sudden, involuntary, unintentional to the ground or lower plane.⁴⁴ In this study, a fall was considered to be either (1) when the patient fell for various reasons with or without injury or (2) when the patient found support in time and did not fall or prevent a fall with the help of others.

Statistical Analysis

The SPSS (Version 22.0) was used to analyze the data. The mean ± standard deviation and t-test were used when the continuous data complied with normal distribution. Otherwise, the median and interquartile spacing, and the Mann–Whitney U-test were used. The rate and percentage, and the chi-square test or Fisher’s exact probability method was used when the data belonged to categorical variables. Independent variables with P < 0.05 in one-way analysis of variance (tolerance) <0.1 and variance inflation factor (VIF) >10 were used to determine multicollinearity. Independent fall risk factors in stroke patients during rehabilitation were screened using multifactorial binary logistic regression analysis (P < 0.05). The rms package and DynNom package in R language software (4.0.2) were used to build column-line diagrams and dynamic nomogram prediction models. Internal validation of the model was performed using the Bootstrap method with 1000 repetitive samples, and differentiation was assessed using the area under curve (AUC) of the receiver operating characteristic curve (ROC). The calibration of the model was assessed by plotting calibration curve graphs and performing the Hosmer–Lemeshow goodness-of-fit test (HLGT). The clinical utility value of the model was assessed using decision curve analysis (DCA).

Results

Basic Characteristics of the Study Population

The flow diagram is shown in Figure 1. We enrolled 488 stroke patients who were undergoing rehabilitation. During follow-up, three patients died, and 16 failed the follow-up, with a failure rate of 3.89%. The patients who failed the follow-up were excluded from the statistical analysis. The basic characteristics of the study population are shown in Table 1.

Figure 1 Study flow diagram.

Table 1 Univariate Analysis of Risk Factors for Falls in Recovering Stroke Patients

Occurrence of Falls in Stroke Patients During Rehabilitation

Of the 469 patients who completed the follow-up, 115 patients experienced a fall, with a fall incidence rate of 24.4%; 32 patients experienced two or more falls, with a fall recurrence rate of 6.8% and a cumulative number of fall events of 163.

Univariate Analysis of the Fall Risk Factors in Stroke Patients During Rehabilitation

The results of univariate analysis showed that the differences were observed in 10 variables between the no-fall and fall groups (P<0.05), including age, history of falls in the last 3 months, FOF, lesion site, taking anxiolytic-depressants, anxiety, depression, MBI score, BBS score, and TUGT. The results are shown in Table 1.

Multifactorial Analysis of the Fall Risk Factors in Stroke Patients During Recovery

Multifactorial logistic regression analyses were further performed on the 10 variables that were statistically significant in the univariate analyses. The fall or no-fall were as the dependent variable (Table 2 presents specific variable assignments). Prior to this, a multicollinearity test was performed on the 10 variables as independent variables. The results of Table 3 shown that the tolerances of all variables were >0.1, and VIFs were <10, suggesting no multicollinearity among the variables. The results of multifactorial logistic regression analyses (Table 4) showed that the fall risk was higher in 60–69 (OR = 2.049, P = 0.034), 70–79 (OR = 2.795, P = 0.002), and ≥80 (OR = 3.650, P = 0.004) age groups than in <60 age group. Patients with a history of falls within the last 3 months had a higher fall risk (OR = 3.282, P < 0.001). Patients who awere anxiety-positive had a higher full risk compared with patients who were anxiety-negative (OR = 2.908, P = 0.018). Patients with BBS score <40 had a higher fall risk (OR = 3.596, P < 0.001).

Table 2 Table of Independent Variable Assignments

Table 3 Diagnosis of Covariance

Table 4 Multifactorial Logistic Regression Analysis of Risk Factors for Falls in Recovering Stroke Patients

Constructing a Dynamic Nomogram Prediction Model for the Fall Risks in Stroke Patients During Rehabilitation

Based on the results of the multifactorial logistic regression analysis, the rms package in the R language software and DynNom package were used to draw the nomogram (Figure 2) and generate an online scoring system at https://aaapredictedaaa.shinyapps.io/DynNomappdd/. For example, a 76-year-old stroke patient with a history of falls within the last 3 months was rated negative for anxiety and had a BBS score of 38. The probability of this patient having a fall was 0.700 (95% CI: 0.539–0.823), and the interface is shown in Figure 3. A ROC was plotted to evaluate the discriminatory power of the model, which had an AUC of 0.756 (95% CI: 0.715–0.795), with sensitivity and specificity of 66.09% and 73.16%, respectively, with a P<0.001 (Figure 4).

Figure 2 Nomogram of fall risk in recovering stroke patients.

Figure 3 Dynamic Nomogram of fall risk in recovering stroke patients.

Figure 4 ROC (receiver operating characteristic) curve of fall risk prediction model for stroke patients in rehabilitation period.

Internal Validation of the Prediction Model

The AUC was 0.761 (95% CI: 0.760–0.763) after repeating the sampling 1000 times by the Bootstrap method, which indicated that the model had good discrimination. The calibration curve was plotted (Figure 5), and the results showed good agreement between the predicted probability of occurrence and the actual probability of occurrence of the model. The results of the Hosmer–Lemeshow fitting test showed that there is not significant difference between the predicted probability of the prediction model and the actual probability of occurrence (χ²=2.040, P=0.958), which indicated good goodness of fit.

Figure 5 Calibration plot of the prediction model.

Clinical Utility of the Prediction Model

The clinical decision curve is shown in Figure 6, where the x-axis is the threshold probability, the y-axis is the net benefit rate after subtracting disadvantages from advantages, and black horizontal and grey diagonal lines parallel to the x-axis represent the two extreme cases. The black horizontal line assumes that none of the patients had a fall and did not receive any intervention, and the net benefit was 0. The grey diagonal line is the net benefit assuming that all the patients had a fall and received an intervention, and the red curve represents the net benefit of the prediction model constructed in this study. As illustrated in the figure, the red curve is located in the upper right of the two extreme lines, indicating that the model constructed in this study has a high clinical utility and net benefit values when the threshold probability is 10%–81%.

Figure 6 DCA (decision curve analysis) diagram of the prediction model.

Discussion

In this study, we developed a questionnaire, collected data, and constructed a prediction model based on the pre-established assessment system of fall risk factors in stroke patients during rehabilitation, which addressed the shortcomings of previous studies. Of the 469 patients who completed the follow-up, 115 experienced falls. The incidence of falls and repeated falls were 24.4% and 6.8%, respectively. The final predictors in this study included age, history of fall in the last 3 months, anxiety, and BBS score. Meanwhile, a dynamic nomogram model with good prediction performance (Link:https://aaapredictedaaa.shinyapps.io/DynNomappdd/) for the fall risks in stroke patients during rehabilitation was developed based on the predictors and was validated.

In this study, the incidence of falls was 24.4% during a 2-month follow-up period in stroke patients who were within 6 months of onset. However, the incidence was lower than the 30.56% reported by Zheng et al.⁴⁵ The difference may be due to the short follow-up duration of this study and the relatively young age of the included population, which may be related to the high awareness of fall prevention among the included population and climate of Hainan.

The results of the study show that the older the stroke patients, the greater the fall risks. The fall risks among patients aged ≥80 years is 3.650 times that of patients aged <60 years. With increasing age, skeletal muscle metabolism in the body slows down, and lower limb muscle strength, balance function, joint mobility and coordination decrease.⁴⁶ Simultaneously, chronic inflammatory mediators in the body increase accordingly, neuroendocrine activity is affected, cognitive function of the patients declines, probability of comorbidities increases, and various disease and medication factors may increase the fall risks in patients. In addition, post-rehabilitation functional impairment in stroke patients increases with age, which makes age an independent fall risk factor in stroke patients.⁴⁷

This study showed that the fall risks in stroke patients with a history of falls was 3.282 times higher than in those without a history of falls. The risk of persistent FOF in patients with a history of falls in the last 6 months is 3.499 times higher than that of patients without a history of falls.⁴⁸ FOF can cause adverse psychological and physical effects on patients, such as reducing the confidence of patients in daily activities and rehabilitation exercises, leading to limited activities, hindered rehabilitation, and increased fall risks in patients.

The findings of the study showed that anxiety is a predictor of the fall risk during recovery in stroke patients, with patients that are anxiety-positive having a 2.908 times higher risk for a fall than patients that are anxiety-negative. Balaban et al⁴⁶⁻⁴⁹ showed that anxiety can negatively affect balance-related neural pathways through the vestibular-parabrachial nucleus network of the brain, thereby impairing balance function and gait in individuals. The prevalence of post-stroke anxiety is approximately 10%–40%,^47–50 which mainly manifests as excessive worry and uncontrollable tension post-stroke, and anxiety symptoms are factors influencing the trajectory of change in the fear of falling in stroke patients.⁴⁸ Similar to FOF, anxiety makes patients less motivated to act on their own, and prolonged activity limitation leads to lower motor function and ultimately to an increased fall risks.

Balance dysfunction is one of the most important risks of fall factor in stroke patients during rehabilitation. In this study, we used the BBS to assess patients’ balance function, and the results showed that the fall risks in patients with BBS scores <40 was 3.596 times higher than that in participants with BBS scores ≥40. In patients with hemiplegia and stroke, the loss of control of the higher center over the lower center, followed by weakened balance reflexes, increased muscle tone, and loss of coordination between muscle groups, decreased patient’s ability to control static and dynamic postures, resulting in an increased fall risk.

In this study, the area under the ROC curve was used to evaluate the discriminatory ability of the model, which was the ability of the model to distinguish between the occurrence and non-occurrence of falls. The results of AUC for 1000 repetitions of sampling using the Bootstrap method, suggesting that the predictive model constructed in this study has a better discriminatory power. In this study, the calibration of the model was assessed by plotting the calibration curve graph and performing the HLGT. The plotted calibration curve coincided with the diagonal dashed line (y = x) with a slope of 1, indicating that the model predicted the probability of fall consistently with the actual incidence of falls, suggesting that the predictive model has a good calibration ability. In addition, the results of HLGT suggested that the model was well fitted.

For the constructed prediction model, we focused on the prediction performance of the model itself. Considering the clinical application of the model, we were concerned about whether patients could benefit from it, that is, the clinical utility of the model. Model prediction cannot be 100% accurate, as there will always be false positives and negatives. We need to judge whether to intervene based on model predictions, as both false positives and negatives may reduce the net benefit. Ideally, an intervention based on the model’s results should be more beneficial to the patient than an “all-accepted intervention” programme or “no intervention” at any time. Assisting decision-makers in assessing the risks and benefits of interventions to find the approach with the greatest net benefit is dependent on the model’s clinical utility. In this study, the clinical utility of the constructed model was evaluated by plotting the DCA; the results showed that the predictive model had a high net benefit value when the threshold probability is 10%–81%, indicating that the net benefit of the model was higher than that of the two extremes over a wide range of thresholds and that the large range of thresholds ensured the safety of the model’s predictions.

In the current study, we constructed a dynamic nomogram prediction model for the fall risk in stroke patients during rehabilitation, different from the commonly used fall risk assessment tools. We comprehensively considered stroke-related factors, and the prediction model developed in this study identified four predictors: age, history of falls in the last 3 months, anxiety, and BBS score <40, which addressed the limitations of MFS, such as age and anxiety psychological scales, which is more targeted than in traditional scales and helps healthcare professionals develop individualized interventions for each patient to reduce the incidence of outcome events. In addition, the dynamic nomogram is an online scoring system that transforms complex regression equations into a visual form and can directly calculate the predicted probability of a fall, with the advantages of fast and simple operation and a concise and clear interface.

Because our prediction model requires professional assessment of balance function and anxiety, it may be less appropriate for use in home care and more appropriate for use in healthcare settings. In addition, considering the existing devices, such as wearable inertial sensors or dual-force platforms, may have the advantages of shorter time consumption and more accuracy in evaluating the balance function of stroke patients. Future studies can try to replace the BBS score in our prediction model to measure the balance function of patients, and then evaluate whether the predictive and practical of the prediction model can be improved.

The study existed some limitations. The study sample were only from one hospital, which might limit the representativeness and generalizability of the results. A multicenter prospective study is needed in the future to validate the stability and extrapolation of the established predictive model. Due to time constraints, this study did not compare the established prediction model with common clinical fall risk evaluation tools. Hence, it is requisite to compare the effectiveness of the prediction model with common clinical fall risk assessment tools in stroke patients during rehabilitation.

Conclusion

The dynamic nomogram model established in this study identified four predictors. The model has good differentiation, calibration, and clinical utility. It is a simple and practical predictive tool that can help healthcare professionals quickly identify patients with a high fall risk in the early stage of stroke rehabilitation and thus prevent fall events.