The Effect of Clinical Ambiguity on the Decision-Making Process Among Intensive Care Unit Providers in Northern America Using Clinical Vignettes in Mixed Methods Study

Introduction

Critical illnesses demand a high level of skills, knowledge, and the ability to navigate complex decision-making processes.^1,2 Decisions are often made under significant uncertainty, as clinical data are frequently imperfect, biased, or challenging to interpret.^2–6 Ambiguity is a fundamental “risk” in the medical decision-making process, and a provider’s ability to tolerate this ambiguity and take risks becomes critical in care delivery.^7–12 This ambiguity can be managed in some well-established illnesses via established treatment protocols (sepsis, pneumonia), while other disease recommendations (for example, pancreatitis) are much less defined.¹³ Additional sources of risk include, but are not limited to, local deviations from standard practice or the ever-changing landscape of clinical guidelines.^13–16 Cognitive appraisal of the ambiguity of the situation will affect the decision-making process within the context of provider experience.^2,5 More experienced providers often rely on intuitive thinking, especially when dealing with less sophisticated, ambiguous data.¹⁷ Less experienced practitioners frequently engage in logical thinking as it enables decision-making in individuals with low tolerance for ambiguity or experience.^4,18,19 Varying levels of emotional response are triggered depending on whether logical or intuitive thinking is deployed.^{7–12,20–22} Furthermore, these emotional consequences can be mitigated by avoiding risky decisions even if deviation from evidence-based guidelines occurs.^8,23 Also, providers may deploy strategies focused on managing the emotional aftermath, such as denial.^23,24 This relationship between the perception of risk, tolerance of ambiguity, and the clinical decision-making process affects and the effectiveness of the care delivery.^{1,4,12,25–27} Physicians with higher risk-taking preferences have been associated with lower admission rates for patients with chest pain, lower imaging use in abdominal pain cases, and lower overall resource utilization.^28,29 If the subjective perception of risk is low, the provider engages in treatments with an excessive rate of complications.²⁶

This complex network has been infrequently tested in the intensive care unit (ICU) setting.^1,22 Maintaining the balance between risk perception, tolerance of ambiguity, and preferential engagement in decision-making style and strategies to cope with emotional aftermath are particularly critical for ICU patients and the well-being of their providers. In the case of one of the most morbid and common ICU illnesses, sepsis, the provider faces a high-ambiguity/high-risk/high-stake situation but can rely on clear guidelines to make decisions.^16,30 The standard of treatment for sepsis is to implement a “bundle” of therapies even if the threshold of suspicion is low as mortality increases exponentially with a delay in treatment.^13–15 However, a lack of improvement throughout sepsis places the provider in a more ambiguous situation since the recommendations are vague if there is no favorable response.^13–15 So, the provider faces two sequential situations here. Initially, it is a high-risk, relatively low-ambiguous situation with clearly established guidelines.^13,16,30 Here, the threshold can be determined by a simple Bayesian risk/benefits ratio with minimal emotional consequences for a provider.^1,4,5,22,31 This situation supports logical thinking, especially if providers are less experienced. However, if sepsis is not successfully treated, the situation becomes high risk, with high ambiguity necessitating intuitive thinking. Even more critical is the provider’s flexibility in decision-making in ambiguous situations.

Here, we examine healthcare professionals’ responses to sepsis, where initial treatment is followed by lack of response by a patient to therapy (Vignette #1). We also created a second scenario when the diagnosis was a much less obvious diagnosis on admission (Vignette #2). We specifically hypothesize that the initial presentation of sepsis will result in deploying highly structured therapy per guidelines, particularly with less experienced providers engaging in logical thinking.^1,6 When facing a situation where the illness does not respond adequately to prescribed therapy, there should be an increased engagement in intuitive thinking processes, particularly among more experienced practitioners, while leveraging denial to reduce stress.^1,2,4,6,24 Providers who balance risk-taking with a high tolerance for ambiguity and low reliance on denial mechanisms are better equipped to handle high-risk scenarios effectively, especially at the onset of high-risk/low-ambiguity scenarios.^17,24 As clinical situations evolve, the engagement of intuitive thinking and the deployment of coping mechanisms such as denial may play significant roles, particularly among seasoned practitioners.^1–4,17,22

Materials and Methods

In this study, we utilized a cohort survey design with a convenience sample of healthcare providers (attendings, fellows, residents, and advanced practice providers) with different levels of medical experience. The study was approved by the IRB (#826741; 2017.02) of the University of Pennsylvania Health System. Electronic informed consent was obtained from all participants before study enrollment, in accordance with institutional guidelines and the Declaration of Helsinki. Unique identifiers were assigned to each participant.

Participants/ Study Setting

Healthcare providers from a multicenter academic hospital were invited via Email to participate in a survey on medical decision-making using convenience sampling. The invitation was sent using an existing mailing list and no incentives were offered to participants. The investigators do not control these distribution lists. Participants could open the link at their convenience in the condition appropriate to confidentiality. Inclusion criteria were 18 years or older, a healthcare professional in a healthcare-related field, able to read and understand English, and provide informed consent. There was no formal testing for these variables as we assumed this proficiency among providers in the major US healthcare system.

Psychological Variables

The psychological construct of the variables interacting with each other was created using existing literature.^{1,2,6–8,10,12} We deployed the following scales which were not modified to assess psychological determinants of decision-making considering their original validation. Reliability was assessed in all tools but only one^32,33 provided information across cultures. While some investigators address ethnicity,⁸ it is not enough to address cultural variability. Therefore, potential scoring biases may exist. The Tolerance of Ambiguity Scale (TOA) was developed to examine the ability to cope with clinical uncertainty using inter-item correlation and factor analysis. The scale was validated and adjusted using factor analysis with physicians and psychiatrists who were considering genetic testing. The resulting scale consists of seven items, with a higher score relating to a lower ability to tolerate clinical ambiguity.^8,10 Internal consistency was 0.78. The IBS Rationality/Emotional Defensiveness Scale (R/ED) was deployed to assess the intensity of repression/ denial.^32,33 The scale consists of 12 items. The scale was validated on a patient population, but the universal nature of this denial mechanism allows for more generalization. A higher score correlates positively with a lower intensity of the repression/emotional defensiveness mechanisms. Psychometric characteristics of IBS-R/ED were satisfactory with Cronbach α=0.77.^32,33 Correlational analyses conducted in the Netherlands and the US, need further studies to assess cultural variability. The Anxiety Due to Uncertainty (ADU) 13-item subscale of the Physician Uncertainty Reaction was used to measure anxiety related to ambiguity.^34,35 Higher scores signify an increased element of stress from clinical ambiguity. The scale was validated on a representative sample of physicians. Internal consistency was 0.80. Prevalence for risk-taking behavior was assessed using the Modified Jackson Personality Index (JPI), a six-item scale.³⁶ The higher score indicates more risk-taking behavior. Internal consistency varied between 0.66 and 0.79, depending on whether items were positively or negatively scored. The Decision Style Survey (DSS) measured the decision-making process on two dimensions: rational and intuitive processes.^37,38 This has been validated and consists of two subscales (five items each) with higher scores related to logical and intuitive decision-making processes.^37,38 This is a 10-item scale validated on five independent samples and correlated with the Big Five Traits of the International Personality Item Pool. Internal consistency for the total score was 0.68; for the rational decision-making scale, it was 0.84; and for intuitive decision-making, it was 0.77.³⁹ Life Orientation Test (LOT) was used to measure the traits of optimism. The psychometric features have been published before.⁴⁰ Internal consistency and test-retest reliability has been reported with a Cronbach’s alpha (α=0.76).⁴¹ To address cultural variability, this has been translated to Greek and studied in Greece (LOT-R) with a high Cronbach’s Alpha (α=0.71).⁴²

Survey Design

We collected basic demographic information (age, gender), healthcare role (attending, fellow, resident, or advanced practice provider (APP), years of professional experience, and the percentage of time spent with ICU responsibilities.^22,43

Two of the study authors developed two clinical vignettes from existing literature representing common clinical scenarios (Supplemental Material 1).^44–46 The vignettes were created with the idea of progressive treatment failure leading to patient demise, exposing the provider to patients with sepsis in the case of Vignette #1 and much more ambiguous diagnoses in the case of Vignette #2 based on published case reports.⁴⁷ Five non-participating and blinded providers evaluated the vignettes for realism and clinical relevance before embedding them into the study methods. These judges suggested the changes to vignettes. After these changes were implemented, vignettes were deployed for the study.

Each vignette consisted of eight one-day, predetermined treatments. Each day, participants could select therapies from a predetermined list based on current practice patterns and recommendations regarding the treatment of septic shock.⁴⁸ There was also an option to use free text, and these data were analyzed qualitatively. We inquired if participants had enough information to form a treatment strategy on a given day using standardized treatment options based on guidelines and current practice patterns.

Survey Administration/Data Collection Process

Surveys were self-administered via the RedCAP™ web-based survey instrument.⁴⁹ The survey invitations were distributed twice (one month apart) via Email link to healthcare providers via an internal mail distribution targeting individuals providing critical care at the academic medical center. A total of 1078 invitations were sent, 258 of them were opened by prospective participants. Of those 258 individuals, 138 individuals completed the survey. If there were missing data, the participants were removed from the study. The time for survey completion was not collected or assessed.

Statistical Analysis

Descriptive statistics described the data. The normality of distribution and parametric nature of the data were assessed using the Lilliefors test and QQ plot.⁵⁰ Chi-square (χ²[d∫;n]) test for independence was used to determine the difference in frequencies for categorical variables. Student’s t-tests (t[n] or Kruskal-Wallis (KS[n]) tests were used to evaluate the differences between the groups when two groups were considered for parametric and nonparametric data, respectively.⁵¹ ANOVA (F[d∫;n] was employed for multiple comparisons with the Dunnett test for post-hoc contrasts.⁵¹ In general, we deployed more conservative nonparametric statistics, as there is no consensus on the definition of data’s parametric nature.⁵⁰ Correlations momentum was expressed as r² Pearson value. Cluster analysis was conducted using agnostic k-mean of psychological traits to categorize them in groups in multidimensional variable environments. The initial center of correlations was not defined. Subsequent analysis looked for maximal distance between centers while maintaining reasonable number of cases similar to modified elbow technique.

The level of significance was set as a p-value of <0.05 for hypotheses unless specifically stated in the text. The free text field was analyzed using thematic content analysis (Supplemental Material 2). SPSS v.29 (IBM, Whalton, MA) and Graph Prizm (Prizma. Cambridge, MA) were used for statistical analysis.

Qualitative Analysis

The free text entries were collated to identify common themes by one of the authors using content analysis. Codes and themes were used to help support and explain the quantitative results. There were very few entries, but items were classified into several themes and then grouped based on the therapy types and presented in the form of a table Redundant entries were used for confirmation of codes and themes. Considering relative sparsity and consistency of the free text entries we decided against utilizing another person for qualitative analysis.

Results

Studied Population

There was a total of 138 participants, with a mean of 12.7±9.99 years of healthcare experience; the majority were female (54.3%), and approximately 46.5±38.73% of their time was spent performing ICU duties (See Table 1 for complete details). We found residents and fellows to be the youngest, with the least amount of experience in healthcare and the least number of people in households compared to attending physicians and APPs (Table 1). The percentage of clinical duties in the ICU was highest among the APPs (66.3±44.91) (Table 1).

Table 1 Basic Demographic and Professional Characteristics of Studied Group Demonstrated Difference as Expected from Our Studied Groups

Implementation of Sepsis Bundle and Other Therapies

We found that a minority of the providers engaged the sepsis bundle 48 hours after seeing the patients (Figure 1A), but this varied slightly according to the vignette. The full bundle was used in Vignette#1 earlier than Vignette #2. Regardless, the initial implementation rate of the sepsis bundle was relatively low, mostly due to the sparse use of antibiotics in vignette #2, while in vignette #1, fluids were rarely given (Supplemental Material 3) in the first 72 hours. By day three, mechanical ventilation (MV) and renal replacement therapies (RRT) were implemented, yet this varied according to vignettes until day five (Figure 1B and C). Recommendations for RRT and MV therapies were made more often by APPs than by other providers (Supplemental Material 3). There was significant use of bicarbonate and corticosteroids with minimal implementations of other therapies such as (Figure 2). Qualitative analysis revealed that providers frequently requested additional information, including history and physical exam findings, diagnostic testing, and lab work such as lactate levels, repeat cultures, and blood gases. In addition, they requested more support from consultative teams (renal, pulmonary and infectious diseases). Interestingly, a minimal number of palliative care consults and goals of care conversations were initiated on day four and six but only in vignette 1 and in clusters 1, 3 and 4.

Figure 1 There were significant differences in the deployment of the sepsis bundle between the two vignettes during the first three days of clinical scenarios (A) when day-by-day was considered (bar) or cumulative implementation (lines) (A). The deployment of mechanical ventilation was also significantly different on days 2nd – 4th, with vignette #1 triggering more frequent recommendations to deploy mechanical ventilation (B). The deployment of renal replacement therapy followed an identical trend, with a significant difference in recommendation between the two vignettes, which were focused on days 2n through 5th (C). * Denotes the level of statistical significance between two vignettes.

Figure 2 Deployment of several practice-rooted treatment modalities was limited in impact, but we noticed significant use of bicarbonate and corticosteroids leading to the intervention.

Psychological Characteristics of the Studied Population

Study participants were not very tolerant of ambiguity, used defensive mechanisms, and more often used rational decision-making rather than intuitive decision-making (Table 2). There were no statistically significant differences in psychological variables considering gender or provider role (Table 2). Years of healthcare experience correlated negatively with the stress of uncertainty, defensiveness, and rational thinking. On the other hand, optimism correlated positively with years of healthcare experience. The percentage of ICU responsibilities correlated to risk-taking and defensiveness. However, these correlations were weak (Figure 3).

Table 2 Distribution of Psychological Variables Across the Studied Population Showed No Differences in Distributions of the Studied Psychological Factors

Figure 3 There were few correlations between professional experience and psychological variables but the interplay between variables was low. Bolded and underlined correlations are also statistically significant.

The Effect of Psychological Variables on Treatment Strategies

We adopted three strategies to examine the relationship between providers’ psychological characteristics and the treatment strategies. First, we compared implementers to non-implementers of the septic process with respect to their psychological makeup. Next, we compared the implementation of the sepsis bundle across providers, considering the diversification of risk-taking behaviors and tolerance of ambiguity. Finally, we conducted a cluster analysis using the psychological traits and comparing the treatment patterns among providers.

First, participants were identified based on the amount of bundle implementation resulting in implementers, non-implementers, never implementers, or ever implementers (Table 3). Implementers were those performing early implementation of the sepsis bundle in the first 24 to 48 hours. Non-implementers were those participants who did not implement the bundle during the first 24–48 hours. Never implementers were providers who did not implement the bundle during the 8-day study period, while ever implementers implemented the bundle at any time during the 8-day study period. We identified 30 individuals who implemented the bundle within the first 48 hours and 39 who never implemented the bundle. Their demographic, professional, and psychological characteristics did not differ between groups except the amount of ICU responsibilities (Table 3).

Table 3 A Comparison of Individuals Who Implemented Sepsis Bundles Within 24 hours, 48 hours or Never Implemented Showed Only Difference in Percentage of Clinical Duties

Second, when we divided healthcare providers based on TOA and risk-taking into four groups based on the psychological testing results, we found that providers who implemented the bundle had either a predilection for risk-taking or average scores of TOA (Figure 4A). The decision to implement MV and RRT did not differentiate between TOA or risk-taking groups (Figure 4B and C).

Figure 4 When the participants were divided using cluster analysis according to the risk, tolerance and ambiguity, only difference was seen in case of Full Sepsis Bundle (A), but not in notation of Mechanical ventilation (B) or Renal Replacement Therapy (C). * Denotes the level of statistical significance between two vignettes. TFA = Tolerance of Ambiguity, FB = Full Sepsis Bundle, MV = Mechanical Ventilation, RRT = Renal Replacement Therapy.

Finally, a cluster analysis of all studied psychological variables was performed. All psychological variables in the initial cluster contributed significantly to the heterogeneity of the cluster analysis except rational decision-making style. Therefore, we removed rational decision-making and re-ran the cluster analysis. The final analysis yielded four clusters utilizing all psychological variables except rational decision-making (Table 4). Defensiveness was borderline in its variability between clusters. Cluster #1 represented an exceptionally high stress of uncertainty while being very low on risk-taking and relatively low on denial and optimism on the ISBRED. Cluster #2 represented the average scores of psychological traits. Cluster #3 had the lowest TOA and low stress of uncertainty scores. Cluster #4 demonstrated exceptionally high stress of uncertainty scores but the highest optimistic denial scores. Demographic variables did not differ among clusters (Table 4). Implementing the sepsis bundle, engagement of MV, initiation of RRT, or ancillary therapies did not differ across clusters (Figure 5A–C, Supplemental Material 4).

Table 4 Distribution of the Psychological Traits Across Clusters Showed That Cluster #1 Represented an Exceptionally High Stress of Uncertainty While Being Very Low on Risk-Taking and Relatively Low on Denial and Optimism on the ISBRED. Cluster #2 Represented the Average Scores of Psychological Traits. Cluster #3 had the Lowest TOA and Low Stress of Uncertainty Scores. Cluster #4 Demonstrated Exceptionally High Stress of Uncertainty Scores but the Highest optimistic Denial Scores

Figure 5 Shows different treatment approaches according to different clusters, including starting and adjusting Antibiotics (A), Heparin (B) and Mechanical ventilation (C), over time. Individuals were analyzed by psychological clusters. * Denotes the level of statistical significance between two vignettes.

Discussion

This is the first study utilizing extensive and simultaneous measurements of psychological variables to study their impact on the clinical medical decision-making process using clinical vignettes.^{1,21,25,47,52} We hypothesized that providers with a low TOA and specific risk-taking preferences would quickly adopt evidence-based strategies in high-risk situations, such as sepsis, as presented in Vignette #1.^7–9,21 These providers tend to favor structured approaches and rely on established protocols to manage critical conditions.^1,6 However, our findings did not support this hypothesis despite determining significant variation in treatment choices.⁴⁷ We also found little evidence of defensive and risk avoidance strategies. We did not observe changes in the utilization of logical and intuitive thinking and specific decision-making behaviors across several types of providers and their experiences.^38,53,54 Instead, delivery of the care seemed to be independent of any studied psychological variables. The only significant predictor of decision-making was the proportion of duties performed in an ICU setting. This unexpected result may stem from several factors. We rooted our hypothesis utilizing intuitive vs rational thinking theory, but it is not the only theorem out there.^52,55,56 Education of health care providers may counteract the effect of studied psychological variables on the decision-making process, though some work suggests that professional development continues well after completion of formal training.^52,53,57,58 Other factors that may influence the interaction between patients’ clinical complexity and provider psychological gestalt remain unclear or untested.^1,6,21,22 For example, resilience appears to play a key role in helping providers navigate the uncertainty of clinical data and the emotional demands of decision- making.^58,59 Resilient providers are better equipped to handle stress and ambiguity, avoiding the pitfalls of avoidance or denial. Alternatively, a lack of adherence to guidelines when working in ambiguous, pressurized, and risky contexts can derail decision-making due to the tendency to rely on psychological biases and faulty heuristics that override more rational processing like in Vignette #2.⁶⁰ It is unknown whether educational programs can influence the use evidence based guidelines. For example, using ‘representative heuristics’ to label a patient as ‘unlikely to do well’ in ICU based on prototypical knowledge about that patient type instead of more rational consideration of the specific qualities of that patient, an issue that is often exacerbated by time pressure to make these decisions quickly.^61,62 It is unlikely that changes in guidelines played a role in low adherence as most of them are similar over the years with special emphasis on early use of antibiotics and fluid therapy.^13–16

We found several interesting and novel findings. One of the most surprising was the low adherence to the sepsis bundle initially and over time. We also found that several traits are stable and not dependent on prior experience. This suggests that healthcare providers come with predetermined TOA, risk-taking, and thinking types instead of being amenable to change during professional training.^8,35 Consequently, professional educators must account for these variables while teaching future generations of healthcare providers.

Other investigators should study these results. It is possible that the vignettes were not convincing despite being written and rigorously reviewed by experts. Furthermore, we demonstrated a difference in sepsis bundle implementation between Vignette #1 and #2. Our study was also deployed in 2019. Although the Survive Sepsis Campaign was already deployed, it was not in full swing as seen today, though core recommendations showed less variation over time.^13–16 Sepsis may be perceived as a heterogeneous illness through current recommendations suggest uniform sepsis treatment. It is unclear how cultural context influenced the decision-making process. Furthermore, cultural context may severely limit the applicability of this study outside of Western medicine and belief systems. Vignette testing has its limitations, including a lack of flexibility in the therapies chosen.⁴⁷ However, providers were able to use free text to suggest therapies that were not included in the predetermined choices. Still, these were limited to the use of pressors, fever reduction modalities, electrolyte replacement, and sedation. Potential factors affecting the results include the level of training, as we did not assess providers’ knowledge of sepsis, despite the widespread implementation of the Sepsis Survival Campaign and sepsis bundle in healthcare systems. As we studied only one system, resource allocation is unlikely to be a factor in this case. However, in future cross-validation studies involving multiple institutions, the variable availability of resources and cultural influences will need to be considered.

This study has several strengths. There were differences in medical treatment between vignettes at the initial stages, as the study’s authors intended. These differences were consistent when we compared medical treatment, with Vignette #1 being more sepsis-oriented versus Vignette #2 being more ambiguous. As the clinical scenarios demonstrated progressive worsening, the clinical approach to the critical illness equalized. This is not surprising, as protracted and relentless illness often results in providers approaching several treatments based on their experiences and not rigid guidelines, with a noticeable increase in heterogeneity. This will result in a somewhat haphazard implementation of ancillary strategies, including corticosteroids, bicarbonate, or blood products. We deployed three different statistical and methodological approaches to link the psychological variables to decisions made by providers, including agnostic k-means cluster analysis. We used standardized tests to assess psychological variables. The internal validity of this study was confirmed by the correlation between psychological variables, as demonstrated in description of the psychological tests. However, some of the scales were part of a larger study. Statistical analysis was conservative, focusing only on relevant results. The data were inherently consistent. For example, demographic characteristics revealed an expected increase in age or change in marital state as more experienced providers reported their characteristics. Interactions between several variables were consistent with the psychological parameters of tests.

Conclusion

In conclusion, providers with more ICU experience combined with the level of preexisting trait of optimism seemed to influence the relatively low implementation of the sepsis bundle across the two vignettes when considering other providers’ demographic and psychological variables.