Refinement and Validation of the Team Effectiveness Scale for Nursing Units

Introduction

The widely recognized publication of “To Err is Human: Building a Safer Health System” prompted a significant reconsideration of safety in health care. They established “promoting effective team functioning” as one of the five principles to create safe hospital systems.¹ Since this report was published, there has been a notable surge in research dedicated to investigating team effectiveness,² particularly within multidisciplinary teams composed of medical, nursing, and allied health professionals.^3,4 Numerous studies have established a clear link between high team effectiveness and better health outcomes.^5–8 Team effectiveness in healthcare research has predominantly centered around interdisciplinary teams, measured by factors such as patient health outcomes, satisfaction, the team’s work performance process, and individual members’ work performance and satisfaction.²

A nursing unit is a type of working team that provides various nursing services (such as direct and indirect nursing and communication) for a specific number of patients.^9,10 In South Korea, nursing units are mostly composed of nurses, different from nursing teams in other countries composed of multidisciplinary teams. Nurses establish nursing plans according to nursing processes and carry out cooperative working in three shifts by handing over nursing tasks to each other in nursing unit. Therefore, the performance of the nursing unit needs to be managed in terms of team performance rather than nurses’ individual performance.¹¹ Studies suggest that assessing the effectiveness of a nurse-centered team should consider delicate and diverse factors such as the process of team performance and communication, which represent nursing characteristics.^11,12

Nursing units have unique characteristics that differentiate them from doctor-centered multidisciplinary teams, but the tools and frameworks used to measure team effectiveness have not fully accounted for these differences. For example, Lemieux-Charles and colleagues¹³ investigated variables influencing team effectiveness in health care organizations, measuring the effectiveness of the multidisciplinary team with objective variables such as patient’s functional status, patient satisfaction, mortality rate, incidence of complications, cost of services, length of stay, hospital admission rate, incidence of adverse events.

In the systematic review of papers that measured the effectiveness of nurse-centered team, only 10 papers were available in English and Korean languages.¹¹ Some studies have evaluated team effectiveness in terms of the effectiveness of the nursing service delivery system, team functionality, nurses’ job satisfaction, work process, and nursing roles.¹² Another study used a team effectiveness measurement tool that simply modified the terminology from the business field to the nursing setting.^14–17 Kash and colleagues⁶ analyzed 22 tools developed up to 2016 to measure the team effectiveness of multidisciplinary teams in health care and argued that it is necessary to develop valid tools that are sensitive to health care settings.

Given these gaps, it is evident that a more tailored approach is needed to assess the effectiveness of nursing units. The management of a nursing unit necessitates the evaluation of team effectiveness of nursing units.¹⁸ In measuring the team effectiveness of nursing units, it is necessary to understand the unit’s effectiveness using a comprehensive model that reflects the characteristics of the healthcare system. Moreover, it is important to measure the objective and subjective outcomes of the nursing unit as well as the team dynamics perceived by nurses.¹⁹ In other words, the evaluation of the team effectiveness of a nursing unit should include both process (eg, leadership, cohesiveness, nurse competency, and coordination) and outcome factors (eg, job satisfaction and productivity).^11,20–24

Integrated Team Effectiveness Model (ITEM) was developed by Lemieux-Charles and McGuire.¹³ This model presents the organizational context in which a team operates (ie, goals, structure, rewards, resources, and training environment) as well as the way in which the organizational context can indirectly influence team effectiveness. Moreover, it shows that organizational context can affect task design (such as team type, team composition, autonomy, and interdependence), which in turn influences team processes (such as communication, leadership, and decision-making) and psychosocial traits (such as cohesion and rules). The model assumes that the abovementioned aspects can directly influence team effectiveness.

The Team Effectiveness Scale for Nursing Units (TES-NU) was a significant step forward in this direction. Developed based on interviews with nurses and grounded in the ITEM framework, TES-NU aimed to provide a more accurate tool for evaluating the unique dynamics of nursing units.¹⁹ However, its initial development faced limitations, including a lack of confirmatory factor analysis and data drawn from a single nursing group, which restricts its broader applicability.¹⁹

This study addresses these limitations by refining and validating TES-NU for use across various nursing units in South Korea. By reducing the number of items and testing this tool validity and reliability, this study seeks to enhance the utility of TES-NU. This study aimed to provide fundamental data for the management of team effectiveness in nursing organizations by presenting evidence related to the validity of the tool to measure team effectiveness of nursing unit for all nursing units in South Korea.

Materials and Methods

Design

This methodological study was designed to refine the TES-NU by establishing its validity and reliability.

Participants

The study included 330 clinical nurses from six general hospitals in South Korea, selected via convenience sampling. Participants were randomly divided into two groups to be subjected to exploratory factor analysis and confirmatory factor analysis to verify the construct validity of the TES-NU. Random sampling was performed using the SPSS program to include 130 and 200 participants in samples 1 and 2, respectively. As the number of cases required to obtain stable factors in an exploratory factor analysis should be >100²⁵ and a minimum of 120 cases, which is 20 times the six factors of team effectiveness in nursing units, is required.²⁶ For confirmatory factor analysis, a sample size of 200 is appropriate.²⁷ So, the number of participants in the present study was sufficient for validating the tool.

Instruments

Team Effectiveness Scale for Nursing Units (TES-NU)

TES-NU, developed by Kim and Kim,¹⁹ was based on the ITEM of healthcare institutions and comprises 30 questions in 6 subdomains on a 5-point Likert scale. During the measurement of attitudes, the midpoints of the scale of items can mask disingenuous responses or undesirable thoughts.²⁸ Therefore, each item in the tool was scored on a 4-point Likert scale, with higher scores indicating positive perceptions of nursing unit team effectiveness. The Cronbach’s alpha reliability coefficient of the tool at the time of its development was 0.94, and the coefficients for each subdomain were as follows: 0.90, head nurse leadership; 0.88, cohesion; 0.87, job satisfaction; 0.82, nurse competence; 0.79, productivity; and 0.70, coordination.

Team Effectiveness

Ahn¹⁴ developed the tool for the business field and adapted it to target nurses. It consists of 23 items in 4 subdomains on a 5-point Likert scale, where higher scores indicate positive perceptions of team effectiveness. At the time the tool was developed, the Cronbach’s alpha reliability coefficients for each subdomain were as follows: 0.94, coworker satisfaction; 0.85, team satisfaction; 0.91, team engagement; and 0.86, team performance.

General and Job Related Characteristics

The general and job-related characteristics included sex, age, marital status, religion, education, nursing experience, and nursing experience in the current nursing unit.

Ethical Considerations

This study was approved by the researcher’s institution’s review board (7001066–202302-HR-007) to conduct a secondary analysis of the data collected in the study by Kim & Ko (in preparation). The institution’s review board approved the original study. The original study’s purpose was to identify team effectiveness in nursing units. We conducted this study in accordance with the principles outlined in the Declaration of Helsinki. We first obtained permission to conduct the study from the head of the nursing department of the hospital participating in the study. Nurses voluntarily participated in the study and signed a consent form. We coded, anonymized, and used the collected data only for research purposes. We will securely store these data for 3 years before destroying them.

Data Collection Procedure

Notably, for data collection, the researchers visited the nursing departments of six general hospitals and asked for their cooperation and permission in the survey. Data were collected from January to March 2023. A structured questionnaire was distributed to the nurses who fully understood the purpose and methodology of the study and expressed their willingness to voluntarily participate in the study; further, a small reward was given to those who participated in the survey.

Statistical Analysis

IBM SPSS Statistics 27 and Amos 27 Graphics (IBM Corp., Armonk, NY, USA) were used for data analysis, and the significance level for statistical analysis was set at 0.05.

General characteristics of the participants were analyzed in terms of frequency, percentage, and mean and standard deviation, and a t-test and χ2 test were performed to verify the homogeneity of the participants in samples 1 (n = 130) and 2 (n = 200).

To check whether the items measuring the same concept, item analysis and exploratory factor analyses were conducted for the data obtained from sample 1. The mean and standard deviation, skewness (<3), and kurtosis (<7) were measured for each item, and the correlation between the total score and the items was evaluated. Further, the Kaiser–Meyer–Olkin (KMO) measure and Bartlett’s test of sphericity were used to confirm the basic assumptions for exploratory factor analysis. To determine whether items measuring the same concept are grouped into the factor, exploratory factor analysis was performed using maximum likelihood and Varimax factor rotation methods. The number of factors was determined based on 60% or more of the total explained variance. During item extraction, items with a factor loading value of <0.40 or a cross-factor loading with a difference of <0.20 between factors were removed.²⁵

To review the number of factors on the scale, a confirmatory factor analysis was performed for data obtained from sample 2. To verify the goodness of fit of the model, the ratio of chi-square value to degrees of freedom (CMIN/DF) (<2), comparative fit index (CFI) (>0.90), Turkey–Lewis index (TLI) (>0.90), root mean square error of approximation (RMSEA) (<0.10), and standardized root mean square residual (SRMR) (<0.08) were analyzed. Further, to improve the fit of the model, covariance with the modification index [31] was also analyzed. In addition, convergent validity was assessed with standardized factor loading (>0.5), average variance extracted (AVE) (>0.5), and composite reliability (CR) (>0.7); moreover, discriminative validity was verified when the AVE value was greater than the square of the correlation coefficient.²⁹ To verify how the newly developed scale is related to other scales, Pearson’s correlation test was used to verify convergent validity, whereas reliability was assessed using Cronbach’s alpha coefficient (>0.7).³⁰

Results

Verification of Homogeneity of the Participants

Of the 330 participants, 95.2% were female. The mean age was 30.3 (±5.10) years. About 80% were single, and 65.2% had a bachelor’s degree. In addition, 76.4% had no religion, and 91.2% were general nurses. The average nursing experience was 6.28 (±5.09) years, and the average nursing experience in the current nursing unit was 4.07 (±3.71) years.

The results of the homogeneity test of the primary (n = 130) and secondary (n = 200) analysis groups for the preliminary questions were as follows: There were no significant differences between the two groups in terms of sex (χ2 = 0.467, p = 0.494), age (t = 1.702, p = 0.090), marital status (χ2 = 2.335, p = 0.127), education (χ2 = 3.584, p = 0.167), religion (χ2 = 0.523, p = 0.470), job position (χ2 = −0.313, p = 0.755), nursing experience (t = 1.033, p = 0.302), and nursing experience in the current nursing unit (t = −0.313, p = 0.755) (Table 1).

Table 1 Characteristics of the Participants (N=330)

Verification of Results

Item Analysis

The preliminary items met normality criteria and showed strong discriminatory power. The mean scores of 30 preliminary items ranged from 2.80 to 3.48, the standard deviation ranged from 0.45 to 0.74, skewness ranged from 0.90 to 0.69, and kurtosis ranged from 0.59 to 3.38, with each value ranging between 3 and 7 and thereby meeting the normality criteria. The correlation coefficients between the 30 items and the total score ranged from 0.51 to 0.74, with all values being >0.30, which confirmed the discriminatory power of the tool.³¹

Exploratory Factor Analysis

The preliminary items met the conditions of exploratory factor analysis and were analyzed as follows according to the criterion of factor loading of 0.40 or more. The KMO value of 30 preliminary items was 0.90; moreover, the results of Bartlett’s test of sphericity (p < 0.001) indicated that the items were suitable for factor analysis. The cumulative variance of the six factors extracted from the first exploratory factor analysis (leadership, cohesion, job satisfaction, nurse competence, work productivity, and coordination) was 68.68%, and this value was consistent with that reported during tool development. However, 8 items (head nurse leadership item 1, cohesion items 7 and 12, satisfaction item 18, competence items 29 and 30, and coordination items 20 and 21) were removed because their factor loadings were co-loaded on two factors at ≥0.40.²⁷

The KMO value of 22 preliminary items was 0.89, and Bartlett’s test of sphericity was p < 0.001, indicating that the items were suitable for factor analysis. The cumulative variance of the five factors extracted from the second exploratory factor analysis was 67.58%, and the 22 items’ commonality was >0.40. The explained variances were 16.7%, 14.7%, 14.5%, 11.8%, and 9.9% for factors 1, 2, 3, 4, and 5, respectively. Regarding the composition of the items in five factors derived from the exploratory factor analysis, the first factor was head leadership, with items 2, 3, 4, 5, and 6; the second factor was job satisfaction, with items 13, 14, 15, 16, and 17; the third factor was cohesion, with items 8, 9, 10, 11, and coordination, with item 28; the fourth factor was productivity, with items 24, 25, 26, and 27; and the fifth factor was productivity, with item 23, and competence, with items 19 and 22.

Based on the aforementioned findings, we examined the five subfactor items and compared them with six subfactors from Kim and Kim’s¹⁹ tool. We then named the subfactors as follows: factor 1, “head nurses leadership”, factor 2, “job satisfaction”, factor 3, “cohesion”, factor 4, “work performance”, and factor 5, “nurses competence” (Table 2).

Table 2 Exploratory Factor Analysis (n=130)

Confirmatory Factor Analysis

The confirmatory factor analysis initially showed suboptimal fit for the 5-factor 22-item model, but after adding covariances, the revised model achieved acceptable fit with improved indices. We conducted a confirmatory factor analysis to verify the fit of the 5-factor 22-item model, which we derived from the exploratory factor analysis. The initial model fit was as follows: CMIN/DF = 1.946, CFI = 0.910, TLI = 0.895, RMSEA = 0.069, and SRMR = 0.061. This indicated that some of the indices were not appropriate. To improve the fit of the model, covariance using the modification index²⁹ was added.

Further, the model fit of the revised model improved to CMIN/DF = 1.687 (<2), CFI = 0.936 (>.90), TLI = 0.924 (>.90), RMSEA = 0.059 (<0.10), and SRMR = 0.057 (<0.08), indicating that all model fit indices were acceptable (Table 3).

Table 3 Model Fit of Confirmatory Factor Analysis (n=200)

Convergent and Discriminative Validity Verification

The 22 items satisfied the criteria for both convergent and discriminant validity. Of the 22 items, the standardized factor loading ranged from 0.51 to 0.87 for 21 items, meeting the criterion of ≥0.50; further, after reviewing item 24, “Only a few work errors occur in the nursing unit”, it was included in the tool owing to the importance of patient safety in the work at the nursing unit and the fact that its factor loading (0.46) was not too low.²⁹ The AVE value for each subfactor ranged from 0.69 to 0.82, whereas the CR ranged from 0.90 to 0.95; these values confirmed the convergent validity of the items.

The discriminative validity test confirmed that the AVE values of the five subfactors were greater than the square of the correlation coefficients between them, indicating that the subfactors represented each attribute independently of each other. The correlations between the total team effectiveness score and the five subdomains were found to be 0.72–0.84, indicating that each subdomain adequately explained the concepts measured by the tool (Table 4).

Table 4 Convergent Validity Test of TES-NU

Item Analysis and Convergent Validity Test

The 22 items met the following criteria: item analysis, skewness and kurtosis, reliability, and convergent validity. In the item analysis of the finalized TES-NU, the mean of the 22 items was 3.20 ± 0.36 (out of 4), with item-total correlations of 0.53–0.75, which is greater than the cutoff of ±0.30. The mean score for each TES-NU subdomain was as follows: 3.26 ± 0.50, head nurse leadership; 3.05 ± 0.46, job satisfaction; 3.18 ± 0.42, cohesion; 3.12 ± 0.43, job performance; and 3.33 ± 0.45, nurse competence. Notably, skewness was 0.44 for the overall instrument, with the distribution of −0.08 to 0.32 for the subdomains; in addition, kurtosis was −0.45 for the overall instrument, with the distribution of −1.07 to 0.42 for the subdomains. The overall Cronbach’s alpha reliability coefficient of the tool was 0.92, and the coefficients for each subdomain were as follows: 0.88, head nurse leadership; 0.82, job satisfaction; 0.77, cohesion; 0.75, work performance; 0.79, nurse competency (Table 4).

Convergent validation revealed a significantly positive correlation (r = 0.69, p < 0.001) between the TES-NU scores developed in this study and the corresponding team effectiveness scores measured using Ahn’s¹⁴ instrument. According to the correlation analysis among the subfactors, there was a significant positive correlation between work performance of the nursing unit team effectiveness tool and Ahn’s¹⁴ team satisfaction (r = 0.62, p < 0.001), team commitment (r = 0.64, p < 0.001), team performance (r = 0.62, p < 0.001), and peer satisfaction (r = 0.56, p < 0.001) (Table 5).

Table 5 Correlations Among Subdomains of TES-NU and Team Effectiveness

Decision of Final Tool

The nursing unit team effectiveness measurement tool, which was refined through the aforementioned validation process, consists of 22 questions divided into 5 subfactors: The tool includes 5 questions on the leadership of head nurses, 5 questions on job satisfaction, 5 questions on cohesion, 4 questions on work performance, and 3 questions on nurses’ competence.

Discussion

This study aimed to confirm the validity of the tool among a large number of general hospital nurses, with the goal of refining the TES-NU, which was developed for a single nursing organization. Exploratory and confirmatory factor analyses were used to verify the validity and reliability of the tool,³² resulting in a refined tool that was simplified to 22 items in 5 subdomains. We discuss the study’s key findings below.

The first factor in the revised TES-NU was the leadership of the head nurse, also known as the middle manager. This domain comprises five items, including those related to whether the head nurse is familiar with the work of the nursing unit and nurses, whether the head nurse follows principles fairly, and whether the head nurse trusts and protects the nurses. In the ITEM of healthcare, the leadership of the head nurse, which is categorized under the team process, was identified as a factor affecting team effectiveness, both directly and indirectly (by interacting with the psychosocial characteristics of the team).¹³

In nursing organizations, managerial leadership is a multidimensional process that influences nurses to achieve organizational goals; in particular, the leadership of head nurses majorly influences nurses’ job satisfaction, organizational commitment, performance, and autonomy.¹⁶ The importance of nurse leadership as a subdomain of team effectiveness in the revised tool of the present study can be considered a result of the crucial roles played by nurses in bringing the nursing unit together as a team and in ensuring that the team operates effectively and efficiently. As a result, an effective strategy for increasing team effectiveness in the nursing unit is to develop leadership skills among the head nurses. For this purpose, nursing organizations must provide continuous education and training to nurse managers.

Job satisfaction was the second factor in the revised TES-NU. Job satisfaction is a subjective outcome in the ITEM of team effectiveness in healthcare and refers to team members’ attitudes toward job and professional growth. The job satisfaction domain comprises five items about whether nurses feel rewarded, confident, and sense of professionalism in their work; whether nurses actively participate in continuing education for their development; and whether nurses want to continue working in the same nursing unit. These items were similar to the team satisfaction items in the team effectiveness measurement tool in management,¹⁴ which involves questions regarding whether nurses feel a sense of accomplishment, challenge, and teamwork in their work within the team and whether they are satisfied while working with the team and want to continue working in it.

Generally, job satisfaction refers to employees’ perception of preferences and positive feelings toward the job itself, and it is recognized as the most important factor in quality of service, productivity, and employee loyalty.³³ In particular, the job satisfaction of nurses positively influences nursing job performance.³⁴ This finding suggests that nurses who enjoy and are positive about their work can improve nursing productivity.³⁴ Moreover, studies show a positive bidirectional correlation between nurses’ job satisfaction and team effectiveness, as measured by team goals, roles, team processes, team relationships, intergroup relationships, problem-solving, cohesion, and skills and learning.³⁵ When the nursing unit as a team works effectively and efficiently to achieve its goals, positive outcomes can be achieved, eg, nurses feel more professional and confident in their work, are more committed to developing their skills, and are more willing to continue working in the unit. Therefore, job satisfaction is a useful indicator and outcome variable of the team effectiveness of nursing units.

The third factor in the revised TES-NU was cohesion, which represents the team psychosocial traits that influence team performance in the ITEM model of team effectiveness in healthcare. In a team, cohesion refers to the emotional attachment between teammates and the collective cohesion among teammates to achieve goals.³⁶ The cohesion domain of the tool consists of five items regarding whether nurses in a nursing unit encourage, trust, get along with each other, and do not blame each other when problems arise, as well as whether work is predictable These items were similar to the measures of sports team cohesion, such as “I do not say anything to offend each other”, “I get along well with my teammates”, “I get along with my teammates”, “I listen to my teammates”, “I help my teammates improve their skills”, and “I talk to my teammates about techniques or strategies”.³⁷ Studies indicate that cohesion significantly influences the performance of sports teams.³⁸ Therefore, we expect cohesion among nurses in a nursing unit as a team to positively impact team effectiveness.

The fourth factor of the revised TES-NU was work performance. This factor was originally referred to as productivity in the pre-revised tool, but it was renamed to work performance during the tool refinement process to avoid confusion with the broader concept of productivity at the organizational level. Performance factors correspond to the subjective outcomes of team effectiveness, as perceived by nurses in the ITEM of team effectiveness in healthcare. The work performance domain comprises four items that measure the ability of nurses to efficiently complete tasks in an error-free manner within work hours to achieve the common goal of providing good-quality care to patients in the nursing unit. These items were similar to those used in the management field, which measure team effectiveness based on the efficiency of team operations, quality and accuracy of work, adherence to plans, and development of ideas,¹⁴ and to those used to measure team effectiveness based on personal growth and work effectiveness and efficiency.³⁹

Notably, the fourth factor (work performance) was refined to measure nurses’ perceived work performance as a subjective outcome variable of the nursing unit team effectiveness tool. Conversely, interdisciplinary teams at acute care organizations have used both objective (patient outcomes, patient satisfaction, cost-effectiveness, quality of service, and staff retention) and subjective (communication, information exchange, cohesion, satisfaction, work processes, accountability, and service issues) variables to measure team effectiveness.^2,11 Therefore, to effectively manage nursing unit team effectiveness, further research is needed to identify the subjective and objective outcome variables of team effectiveness that reflect the work characteristics of nursing units.

Finally, the fifth factor of the revised TES-NU was nurse competency. In the ITEM of healthcare, team composition, which includes task design as an input, directly affects team processes and team psychosocial characteristics. Moreover, it directly and indirectly affects team effectiveness as an outcome. In the revised tool, nurse competency comprises three items regarding whether nurses complete their assigned tasks during their shift and perform their role responsibly. Owing to the nature of nursing work, which involves working in shifts and handover tasks, nurses may need the competence of their fellow nurses to achieve the goals of the nursing unit. These characteristics differ from team effectiveness measures in the management field, where coworker satisfaction is measured as satisfaction with being in the same team as coworkers and emotional satisfaction with working together.¹⁴ Providing new nurses with sufficient education, training, and support to grow and become competent to responsibly perform their assigned tasks and roles can be a way to increase team effectiveness in nursing units.

With the refinement of the TES-NU, developed using the ITEM of healthcare described above, the revised tool was shortened from 30 questions in 6 subdomains (the pre-revision tool) to 22 questions in 5 subdomains. A measure of nursing team effectiveness, validated for reliability and validity in this study, can help identify and understand the multidimensional factors that influence the achievement of nursing unit goals. By applying TES-NU to clinical practice, nurses and managers can recognize the nursing unit as a team and be more interested in the performance of the team than individual nurses. Therefore, to enhance the performance of the nursing organization, nursing managers need to oversee the effectiveness of the team within the nursing units, in addition to managing the individual nurses’ work performance. Also, to increase the team effectiveness of nursing units, nursing organizations should measure and manage the head nurse’s leadership, job satisfaction, cohesion, work performance, and nurses’ competency in nursing units.

This study analyzed data collected from nurses working at six general hospitals by convenience sampling, which limits the generalizability of the results. We recommend future studies to compare nursing team effectiveness based on differences in structural variables (such as nursing delivery method and the nurse-to-patient ratio) and to identify subjective and objective variables associated with nursing unit team effectiveness.

Conclusion

This study refined TES-NU to develop a concise version of the tool and to verify its validity and reliability applicable to various nursing organizations. This tool would be useful in measuring and managing team effectiveness in nursing units. Nursing managers can use this tool to manage the performance of individual nurses as well as nursing units. Also, nursing managers need to consider head nurse’s leadership, job satisfaction, cohesion, work performance, nurses’ competency when improving team effectiveness of nursing units. Future studies are required to examine additional organizational characteristics that can influence the team effectiveness of nursing units and the relationship between nursing unit team effectiveness and objective outcome measures.