Development and Validation of a Nomogram Prediction Model for Key Symptoms of Post-Intensive Care Syndrome-Family in Family Members of Critically-Ill Patients: Focusing on Sleep Disturbance, Fatigue, Anxiety, and Depression

Introduction

Post-intensive care syndrome-family (PICS-F) refers to a series of physiological and psychological functions, including sleep disturbance, fatigue, anxiety, depression, and post-traumatic stress disorder (PTSD), which may occur in the family members of patients admitted to the intensive care unit (ICU) or transferred out of the ICU, particularly in clinical decision-makers and main caregivers.¹ Family members usually act as surrogate decision-makers and face various pressures during the admission of critically ill patients to the ICU.^2,3 After the patient is transferred out of the ICU, the family members, as the main caregivers, bear a heavy burden of care, rendering the family members susceptible to PICS-F. Prior research has demonstrated that the incidence of PICS-F ranges from 20% to 60%.⁴ Family members of critically ill patients commonly suffer from poor sleep quality for two months after the patients are transferred out of the ICU.⁵ In a cohort study, 77% of family members reported sleep problems.⁶ A survey was conducted on 94 family members of critically ill patients, and the results indicated that 57.6% of the participants reported moderate to high levels of fatigue.⁷ The prevalence of family anxiety during the patient’s ICU stay ranged from 40% to 70%,⁸ decreasing to 10% to 50% three to twelve months after the patient’s ICU discharge.⁹ The prevalence of depressive symptoms during the patient’s ICU stay ranged from approximately 10% to 35%.¹⁰ However, surveys have demonstrated that up to 43% of family members continue to experience elevated levels of depressive symptoms one year after the patients are discharged.¹¹ Poor health status of family members not only affects their own lives but also leads to the decline of care ability, which significantly impacts the rehabilitation of patients. Therefore, early identification of family members at high risk of PICS-F is significant.

The investigation and analysis of factors influencing PICS-F is a highly salient topic in research on PICS-F in family members. A multitude of factors have been identified as contributors to the occurrence of PICS-F, including patient-related factors, family-related factors, healthcare system-related factors, and other factors.^8,12 Family member-related factors associated with increased risk of PICS-F include female gender,¹³ external pre-existing stressors,¹⁴ the role of a spouse or parent,^15,16 limited social support,¹⁷ low socioeconomic status,⁸ and high caregiver burden.¹⁸ Patient-related factors associated with increased risk of PICS-F include the severity of illness,¹⁹ long-term impairments,²⁰ older or younger age,^14,20 and coronavirus disease 2019.²¹ Previous studies have mostly examined the influence of demographic factors; however, the life event of a patient’s admission to an intensive care unit is a stressor that can lead to a stress response in family members, which can be influenced by mediating factors such as social support.²² Therefore, this study incorporated the predictor of social support.

Previous research mainly focused on the risk factors of PICS-F in family members, but no relevant prediction model has been developed to predict the probability of PICS-F. Nomogram models provide a visualization of the predictive factors and graphically present the results of multi-factor logistic regression.²³ The objective of this study was to construct and validate a nomogram model to predict the risk of PICS-F among family members of patients transferred from the ICU. Specifically, this study constructed a family PICS-F line chart prediction model based on literature and clinical data, facilitating the early detection of family members at high risk of PICS-F.

Methods

Setting and Participants

The present study recruited family members of patients who had been transferred out of the general intensive care unit (ICU), emergency intensive care unit (EICU), and neurosurgical intensive care unit (NSICU) of Binzhou Medical University Hospital from December 2023 to June 2024, as study subjects. The family members of patients transferred out of three ICUs between December 2023 and April 2024 were selected as the modeling group, and the family members of patients transferred out of three ICUs from May 2024 to June were selected as the external validation group. The following were the inclusion criteria. Patient level: (1) Age ≥ 18 years old; (2) ICU stay ≥ 24 hours; Family level: (1) Undertook medical decision-making and main care responsibility of patients; (2) At least 18 years old, identified as the patient’s parents, children, spouse, etc. The exclusion criteria are as follows: Patient level: (1) Transfer out to another ICU; Family level: (1) Other severe traumatic stress events occurred in the recent 3 months; (2) organic diseases that prevented the completion of the questionnaire. All the subjects in this study provided informed consent forms. This study was conducted by the principles of the Helsinki Declaration and has been approved by the Ethics Committee of Binzhou Medical University Hospital (approval number 2023KT-255).

Sample Size Calculation

Based on the literature review and expert advice, 20 potential predictive factors were included in this study. The sample size of the prediction model is required to be 5 ~ 10 times the number of influencing factors.²⁴ Considering the documented prevalence of PICS-F of approximately 20% to 60%,⁴ accounting for a 20% sample loss, the minimum sample size of this modeling group was calculated as follows: (20 × 5 ÷ 60%) ÷ (1–20%) = 208 cases. Furthermore, the sample size of the modeling group was required to be at least 1/4 ~ 1/2 of that of the external verification group,²⁵ so the sample size of the external verification group was determined as at least 52 cases.

Study Tools

General Information Questionnaire

The general information questionnaire included general information about ICU patients, including age, sex, Acute Physiology and Chronic Health Evaluation II (APACHE II) score, admission of patients, etc. Moreover, general information about family members was collected, such as age, sex, educational level, working status, etc.

Perceived Social Support Scale

The Perceived Social Support Scale (PSSS) was a measure of the perceived social support of patients’ family members. The scale encompasses three dimensions: familial, amicable, and other forms of support. It comprises 12 items, each with a score ranging from 1 to 7, resulting in a total score between 12 and 84. A higher total score indicates a higher level of social support perceived by the subject.²⁶ The Cronbach’s alpha coefficient of this scale was 0.854.

Outcome Variables and Definitions

Pittsburgh Sleep Quality Index

The Pittsburgh Sleep Quality Index (PSQI) was developed in 1989 by Buysse et al²⁷ and represents a significant instrument for the assessment of sleep quality in family members. The PSQI is comprised of 19 self-rated items and 5 other-rated items, with only the initial 18 self-rated items contributing to the final score. A total score on the scale can range from 0 to 21, with higher scores indicating an increased likelihood of suffering from more severe sleep disorders. Among adults in China, those having a score of >7 points were judged as sleep impaired. The Cronbach’s alpha coefficient of this scale was 0.842, which indicates good reliability and validity.

The Subscale of Fatigue Assessment Instrument

The Subscale of Fatigue Assessment Instrument (FAI) was developed by Schwartz and so on.²⁸ The subscale of FAI was used to evaluate the degree of fatigue of family members in this study. The subscale of FAI is comprised of 11 items, specifically items 5, 18–22, and 24–28. The total scores of the scale can be classified as follows: <4 indicates no fatigue, 4 ~ 5 indicates mild fatigue, 5~6 indicates moderate fatigue, and ≥6 indicates severe fatigue.²⁹ The Cronbach’s alpha coefficient of the scale in question ranged between 0.78 and 0.92.

Hospital Anxiety and Depression Scale

The Hospital Anxiety and Depression Scale (HADS) comprises two subscales, namely anxiety and depression. The odd-numbered items are employed to assess anxiety, while the even-numbered items are utilized to evaluate depression. Each item is assigned a value of 0 to 3 points by the severity of the symptoms in question. The total score is thus calculated on a scale of 0 to 42 points. A score of ≥11 points indicates that the subjects have anxiety or depression. The Cronbach’s alpha coefficient of this scale was 0.890,³⁰ and research has shown good reliability and validity.³¹

The outcome variables comprise physiological (sleep or fatigue) and/or psychological (anxiety or depression) aspects. A positive result for either symptom indicated the presence of PICS-F in the subjects.

Data Collection

A research group was set up, including 2 nursing graduate students, 2 emergency ward nurses, and 3 neurosurgical ward nurses. Before data collection, the group members were trained by nursing managers and experts in the psychology field, and all of them collected basic questionnaires. Data collection was carried out in two stages. First, when the patient was transferred out of the ICU to the general ward, the team members filled in the general information of the patient according to the case system. Subsequently, general information about family members and the PSQI, FAI, and HADS questionnaires were collected via a face-to-face interview with family members in the ward within 48 hours of the patient’s transfer from the ICU. Before the investigation, the subjects were fully informed of the research purpose and signed an informed consent form. The participants followed unified guidelines, were informed of the filling requirements, and any doubts were clarified. After filling out the questionnaire, the researchers checked and collected the forms immediately.

Statistical Analysis

SPSS 26.0 and R 4.2.3 software were used to describe the statistical data. The quantitative data did not conform to a normal distribution. Hence, M (P₂₅, P₇₅) was used to describe the difference between the groups and the Mann–Whitney U-test was employed to compare the differences. Qualitative data was expressed by the frequencies and percentages, and the difference between groups was tested by the chi-square test (univariate analyses). The two-way stepwise regression method was adopted, the minimum AIC value was taken as the optimal model selection criterion, and the variables that were statistically significant on univariate analysis, ie family age, family education level, family work status, relationship with patients, per capita monthly income of families, patient age, APACHE II score of patients, and PSSS score were used as independent variables, and whether PICS-F occurred as the dependent variable was used as the dependent variable, and logistic regression was carried out to determine the predictive factors. R4.2.3 software was used to draw the alignment chart model.

Internal validation of the model was verified by the Bootstrap validation method, whereas external validation was carried out using the validating group data. The Hosmer-Lemeshow (H-L) goodness of fit test and calibration curve were used to evaluate the calibration of the model. The receiver operating characteristic (ROC) curve was employed to calculate the sensitivity, specificity, and maximum Youden index. The area under the curve (AUC) was employed to assess the model’s discriminatory power, while decision curve analysis (DCA) was utilized to investigate the model’s clinical efficacy. A two-sided P-value of less than 0.05 indicated that the difference was statistically significant.

Results

Characteristics of the Participants

A total of 432 family members of patients were included in the development model group, with an incidence of post-intensive care syndrome-fatigue (PICS-F) of 58.80% (254/432). In the external validation group, the incidence of PICS-F was 59.30% (80/135). The demographic and clinical characteristics of the participants in the modeling group are presented in Table 1. The baseline characteristics comparison between the modeling group and the external validation group is shown in the Table S1 .

Table 1 Baseline Characteristics of Family Members

Univariate Analyses

In the modeling group, univariate analyses were conducted on each predictive factor. The results showed a statistically significant difference (P<0.05) between the two groups in terms of family age, family education level, family work status, relationship with patients, per capita monthly income of families, patient age, APACHE II score of patients, and PSSS score. Details are presented in Table 1.

Multivariate Binary Logistic Regression Analysis

A two-way stepwise logistic regression analysis was conducted. The dependent variable in this analysis was whether PICS-F occurred in family members. The independent variables were those with a P-value less than 0.05 in the univariate analysis. Substitute the original values of the quantitative variables, including the age of family members, the age of patients, the APACHE II score of patients, and the PSSS score of family members. The assignment of dependent variables and categorical independent variables is as follows: PICS-F (No = 0, Yes = 1); Education of family (Primary school and below = 1, Junior high school = 2, High school/vocational school = 3, College or above = 4); Work status of family (Employed = 1, Unemployed = 2, Retire = 3); Relationship with patients (Spouse = 1, Children = 2, Parents = 3, Others = 4); Family per capita monthly income (Less than 1000 RMB = 1, 1000 ~ 3000 RMB = 2, 3001 ~ 5000 RMB = 3, More than 5000 RMB = 4). The multi-factor analysis results revealed that the age of the family members, the age of the patients, the APACHE II score, the monthly income per capita, and the PSSS score are all influencing factors, as shown in Table 2.

Table 2 Multivariate Analysis of Risk Factors for PICS-F

Construction and Estimate of the Nomogram

In this study, the column chart was used by software R 4.2.3, as shown in Figure 1. The area under the ROC curve of the model was 0.894 (95% CI: 0.864 ~ 0.924), indicating that the model had a good discriminative ability, as illustrated in Figure 2. The predictive model yielded a specificity of 0.854 and a sensitivity of 0.780, a maximum Jordan index of 0.634 and a corresponding optimal cutoff value of 0.603 in the training set. The critical score of the column chart was calculated to be 106 points. The family members were divided into two groups based on this result, including, the high-risk and low-risk PICS-F groups. The H-L goodness of fit test yielded a value of x²= 9.528 (P = 0.300), suggesting a strong alignment between the model’s predicted and actual risk, reflecting good calibration.

Figure 1 Nomogram to detect PICS-F for family members. APACHE II indicates Acute Physiology and Chronic Health Evaluation II; PSSS indicates Perceived Social Support Scale.

Figure 2 Nomogram ROC curves generated from the training dataset. The area under the ROC curve of the training dataset was 0.894 (95% CI: 0.864~0.924).

Validation of the Nomogram Model

Internal validation of the model was performed using the Bootstrap method with 1000 repeated samples, and the calibration curve is shown in Figure 3. The Brier value of the model was 0.01, suggesting that the difference between the predicted likelihood of PICS-F occurrence in family members and the actual outcome was small, and the accuracy was high. The probability of PICS-F in each family member was calculated by feeding external validation set data into the model. The results revealed an external validation AUC of 0.847 (ranging from 0.782 to 0.912), with a sensitivity of 0.745 and a specificity of 0.863, as depicted in Figure 4. Furthermore, the H-L test showed X²=9.625 (P=0.292), indicating that the model’s predictive performance was effective.

Figure 3 Calibration plot for the training dataset. The x-axis showed the predicted probability of the nomogram, the y-axis showed the actual prediction results.

Figure 4 Nomogram ROC curves generated from the externally verified dataset. The area under the ROC curve of the external validation AUC was 0.847 (ranging from 0.782 to 0.912).

Clinical Decision Curve Analysis of the Nomogram Model

The clinical validity of the model was evaluated using the DCA method, and the results are presented in Figure 5. The results showed that the overall net benefit rate of the DCA curve in the threshold range of 0 ~ 1 was higher than that of the two extreme cases of no intervention and equal intervention, indicating that the clinical utility of the predictive model was higher.

Figure 5 DCA curves for the training dataset. The decision curve graph showed the net benefit of the intervention measures taken by the established model on the families with post-intensive care syndrome-family.

Discussion

The prevalence of PICS-F varies across studies.³² The results of this study indicated that the incidence of PICS-F in family members of patients transferred out of the ICU was 58.80%, which is consistent with the research findings of Naef et al.¹⁴ In addition, Shibasaki et al³³ found that the overall incidence of PICS-F was 33%. The disparity in the research results on the incidence of PICS-F may be related to differences in PICS-F screening tools, screening criteria, and evaluation time points.^33,34 Patients receiving treatment in the ICU usually undergo a short-term, non-selective transition, during which their cognitive abilities are limited. Family members usually act as alternative clinical decision-makers,³⁵ bearing pressure from various aspects.³⁶ Due to the particularity of closed management in ICU wards, family members face major changes in the patient’s condition and are more likely to develop anxiety, depression, and other negative emotions. Moreover, prolonged care and waiting outside the ICU, as well as adverse environmental factors such as noise and light outside the ICU, can disrupt the biological clock of family members, affecting normal biological rhythms and resulting in physiological problems such as sleep disorders and fatigue.³⁷ Various multidisciplinary critical care organizations now recognise the detrimental impact of critical illness on the patient’s family and the need for holistic home-based care.³⁸ Hence, in future clinical practice, medical staff in the intensive care unit can optimize the ICU visitation system, develop flexible visitation policies, and enhance the sense of family participation.³⁹

PICS-F is a complex condition with a multitude of associated factors.⁸

This study revealed that the age of the family member was an independent risk factor for PICS-F occurrence. Patients admitted to the ICU exhibit critical and rapid changes to their condition. Relatives are required to stay outside the ICU to monitor the patient’s condition and make timely medical decisions. However, older family members have lower physical fitness and cognitive function, which affects their information comprehension ability, decision-making efficiency, and stress tolerance.⁴⁰

The findings of this study also demonstrated that younger patients were linked to an elevated risk of PICS-F in their family members, which was consistent with the findings of Bielek et al.⁴¹ The younger the patient, the greater their importance to the family.⁴² The caregivers of young ICU patients mostly comprise parents or spouses, with close relationships with the patients. Therefore, when the patient’s condition is critical and their survival and treatment process are uncertain, family members are concerned about the development and prognosis of their condition, resulting in a heavier psychological burden and a higher incidence of PICS-F.

The results revealed that the APACHE II score was an independent risk factor for family members developing PICS-F. The APACHEII score of a patient represented the severity of their condition, with a higher APACHEII score indicating a more critical condition.¹⁹ Patients with high APACHE II scores may experience greater fluctuations and uncertainty in their condition, and their family members may lack sufficient psychological preparation to face the potential possibility of death, leading to emotional tension, immense mental stress, psychological disorders, and aggravated physiological problems such as sleep disorders and physical fatigue.⁴³ Medical workers should pay special attention to the families of critically ill ICU patients and discuss long-term care plans if necessary, including end-of-life care services or palliative care that patients may need.⁴⁴

The results of this study suggested that the lower the per capita monthly income of family members, the higher the probability of PICS-F occurrence. Patients in the ICU are in critical condition, necessitating continuous monitoring of vital signs and advanced treatments. Medical costs for these patients far exceed those of patients in regular departments, placing a significant financial burden on their families. ⁴⁵ In addition, patients admitted to the ICU and subsequently transferred require the ongoing attention of their family members, ⁴⁶ which can disrupt their routines and income. Such circumstances make family members more susceptible to PICS-F.

Moreover, this study revealed lower scores in perceived social support in family members who experience PICS-F and those who do not. High perceived social support effectively reduces the occurrence of PICS-F in family members, consistent with the research findings of Serrano et al.¹² Family members are exposed to economic and mental pressure while waiting outside the monitoring room. Higher levels of social support indicate that family members perceive respect and support. A good level of social support helps to improve the psychological adaptation ability of family members, thereby reducing the occurrence of PICS-F.^47,48

This study constructed a column chart model for PICS-F of family members of patients who were transferred out of the ICU. The AUC of the model in both the modeling group and the external validation group was greater than 0.80. Moreover, the H-L goodness of fit test results showed P>0.05, indicating that the constructed column chart model has good discrimination and high predictive accuracy. The column chart model provides a visualization of the potential risk of family members developing PICS-F after the patient is transferred out of the ICU; the model facilitates the systematical monitoring of the influencing factors related to PICS-F, providing a certain degree of scientific significance.^26,49 In addition, the predictive factors included in this model can be measured using simple and specific methods, which promotes its clinical application. A calculated column chart score ≥106 points indicates that family members are at high risk of PICS-F. Nursing staff can customize nursing plans based on the calculated risk score and provide precise nursing strategies to prevent or reduce the occurrence and development of PICS-F in family members.

Nevertheless, the constraints of the present study must be recognized. Firstly, the cross-sectional study design inherently precludes the detection of dynamic changes in PICS-F among family members. Secondly, the nomogram prediction model was developed based on data obtained from China. The extent to which the findings of this study can be generalized to other regions and countries is contingent upon further validation.

Conclusion

The PICS-F nomogram prediction model constructed in this study demonstrated strong discriminative ability and good calibration. In future clinical work, nursing staff can use this model to efficiently predict the probability of PICS-F occurrence in family members, identify high-risk family members, and provide a reference for developing targeted prevention strategies. Future studies should focus on validating the model’s performance in multicenter, large-scale prospective cohorts to enhance its generalizability, as well as investigating the effectiveness of targeted interventions based on the model’s risk stratification in reducing PICS-F incidence.