Factorial and Criterion Validities of the Chinese Version of Rosenberg Self-Esteem Scale Among Undergraduate Students

Introduction

Self-esteem, encompassing self-worth and self-confidence, refers to one’s positive or negative attitude towards oneself and is one of the most critical traits in estimate self-concept.¹ It is a pivotal trait for assessing self-concept, and serves as a crucial indicator of mental health and subjective well-being.² From the perspective of positive psychology, which is considered a vital protective and promotive factor for mental health, individuals with higher self-esteem tend to report greater self-acceptance and satisfaction. Conversely, various theories have identified low self-esteem as a contributing factor to depressive disorders.³

Several self-report questionnaires and interviews have been developed to effectively measure self-esteem. Among the questionnaires, the Rosenberg Self-Esteem Scale (RSES), the Self-Perception Profile, and the Self-Esteem Inventory are notable examples. Interview-based measures included self-evaluation, social support, and personal interviews. In particular, RSES has gained widespread usage owing to its efficiency and ease of administration.^1,4 Initially designed to gauge the global self-esteem of adolescents,¹ the RSES has demonstrated acceptable reliability and validity across diverse age groups and countries, suggesting its utility in assessing self-esteem across a wide spectrum of individuals.⁵

Early research by Rosenberg considered the RSES to be unidimensional¹ (see Figure 1, RSES-GSE Model). Subsequent studies, however, have revealed a bi-factorial structure that distinguishes between self-confidence and self-deprecation based on positively and negatively wording methods⁶ (see Figure 1, RSES-PN Model). Meanwhile, the wording effects introduced by positively and negatively keyed items would further result in method effects, that is, systematic errors related to the measurement process rather than to the scale structure, which then yield undesirable systematic variation and weaken the measurement of the constructs. To address the problems caused by method effects, correlated trait-correlated method (CTCM) and correlated trait-correlated uniqueness (CTCU) models derived from the multitrait-multimethod (MTMM) framework have been proposed.^7,8 Specifically, CTCM models treat items with the same wording method as a latent variable, which works as an independent factor along with the content factor.⁷ Therefore, CTCM models allow the estimation of the variance arising from method effects, yet may carry the risk of the model not being convergent. CTCU models, instead, correlate the residuals of items with the same wording method to each other, which helps CTCU models converge well,⁹ but cannot estimate the variance resulting from wording effects.⁷

Figure 1 Eight Alternative Models of RSES. RSES-GSE Model = a unidimensional model with a global self-esteem (GSE) factor; RSES-PN Model = two separate factors representing positively and negatively worded items; CTCM-PN Model= correlated traits-correlated method (CTCM) bi-factor model containing a GSE factor and two respective models describing positive and negative method effects; CTCM-P Model = CTCM model with a GSE factor and only a positive factor of method effect. CTCM-N Model = CTCM model with a GSE factor and only a negative factor of method effect; CTCU-PN Model = correlated uniqueness (CTCU) model including one trait factor and correlated uniqueness among positive and negative subscales; CTCU-P Model = CTCU model including one trait factor and correlated uniqueness among positively worded items; CTCU-N Model = CTCU model including one trait factor and correlated uniqueness among negatively worded items.

Using confirmatory factor analysis (CFA), researchers have tested the construct validity of the RSES, with various models receiving support. The CTCU model with negatively worded items (see Figure 1, CTCU-N Model) showed a better fit, whereas Mullen and his colleagues¹⁰ supported the CTCU model with positive item wording (see Figure 1, CTCU-P Model). In contrast, Gomez-Lugo¹¹ and Vasconcelos-Raposo¹² preferred the CTCU model with positive and negative items (see Figure 1, CTCU-PN Model). Boduszek regarded the CTCM model with positively and negatively worded items (see Figure 1, CTCM-PN Model) as the best fitting one.¹³ In contrast, other studies have approved the bi-factor model of the RSES with positive and negative factors^8,11 (see Figure 1, RSES-PN Model) and the bi-factor model with only positive or negative items.⁶ Overall, these competitive models of RSES with both pros and cons were supported by corresponding studies, indicating that the factorial structure of the RSES is contaminated to some extent by method effects.⁷ Hence, it is necessary to examine all the above models by performing CFA to compare the fitting of each model and obtain the best model for future studies. In addition to the inconformity of RSES dimensionality, whether gender differences exist in self-esteem remains controversial. Support for their existence has been provided by several studies employing the unidimensional model,¹⁴ bi-factor model,⁸ and CTCU models with positive items. Other studies, however, yielded no significant differences in self-esteem across genders using unidimensional and CTCM models with negatively worded items.¹⁵ As suggested by prior research, the method effects associated with the wording of items may vary in populations or groups, for example, between males and females. For example, DiStefano and Motl¹⁵ found that gender differences in RSES total scores might be related to different responses to negatively worded items. Therefore, when comparing intergroup differences in the RSES, models controlling for method effects would be preferred first. Further comparisons of latent mean differences should be conducted after cross-group measurement invariance of the model has been established, while directly comparing the observed scores may still suffer from systematic errors.

The establishment of measurement invariance in the factorial structure is a prerequisite for the subsequent analysis of group differences in factor scores, as it confirms that the attributes measured by the scale are invariant across groups.⁹ Therefore, the gender invariance of the factorial structure and model of the RSES should be confirmed before comparing gender differences, based on which the apparent difference between males and females in self-esteem can truly reflect the discrepancy in the evaluative component of self-concept.¹⁶ Some previous studies have focused on gender invariance in the RSES; however, their results remain blended. Several studies found that the RSES was gender-invariant in undergraduates for both trait and method factors using the CTCM model with negatively worded items,^15,17 while some studies reported that the CTCM model of positive and negative factors showed a strong invariance across gender.¹⁶ One study found that the CTCU model with positively and negatively worded items had partially weak invariance across gender.¹² Blended results may stem from differences in the model selection and sample size, warranting further investigation.

Since its translation into Chinese, the RSES has been widely applied in China to evaluate the self-esteem level of individuals and its relationship with other psychological variables.¹⁸ Several studies have examined its factorial structure and compared alternative models, supporting the bi-factor model as the best fit in Chinese populations, such as adolescents and patients with acute coronary syndrome patients.¹⁹ However, only one study has examined the measurement invariance among Chinese urban and rural adolescents.¹⁹

As stated above, low self-esteem is a significant clinical risk factor for depression. The vulnerability model of depression, a prominent framework for elucidating the link between self-esteem and depression, posits that low self-esteem is a stable personality trait that predisposes individuals to experience depressive symptoms. It shapes an enduring susceptibility to mental health challenges,^20,21 suggesting that individuals with low self-esteem are not only directly more likely to develop depression but also have an increased sensitivity to stress, thereby heightening the likelihood of depressive reactions under adverse conditions. Empirical evidence also supports the roles of self-esteem and depressive symptoms in bridging the relationship between stress and life satisfaction.²² Additionally, the cross-lagged model has revealed that self-esteem not only mediates these effects but also predicts life satisfaction over time, highlighting its dynamic influence on well-being trajectories.¹⁴ However, the specific mechanisms through which self-esteem interacts with stress, depression, and life satisfaction remain unclear. To address this gap, this study aimed to verify the criterion validity of the RSES by re-examining the complex interplay among self-esteem, stress, depression and life satisfaction.

In summary, although the RSES has been widely used to estimate individuals’ self-esteem levels in China, no studies have systematically evaluated its factorial validity, let alone gender invariance. Hence, using a Chinese undergraduate sample, this study aimed to verify the factorial and criterion validities of the RSES by (1) comparing the factorial structures of eight alternative models of the RSES and choosing the optimum fitting model using CFA; (2) testing the gender invariance of the optimum model of the RSES using multigroup CFA; (3) verifying whether the RSES has acceptable internal consistency reliability; and (4) examining the criterion validity of the RSES by comparing the group differences in anxiety and depression (high self-esteem vs low self-esteem) and by verifying the vulnerable model of depression.

Materials and Methods

Procedure and Participants

A priori power analysis was conducted to determine the sufficient sample size for the intended statistical analyses with G*power software.²³ It was estimated that a total of 436 participants would be necessary to achieve a statistical power of 95% for the Goodness-of-Fit tests (39 degrees of freedom).

Participants were recruited from a university in Hunan Province, mainland China, using convenience sampling. To ensure a diverse and representative sample, this research included participants from all the four undergraduate year levels, ranging from first year to senior students. The administration of data collection was conducted under the supervision of two psychology graduate students per class. All participants acknowledged and signed the informed consent form at the beginning of the study.

A total of 1493 participants were enrolled in this research. After excluding those who failed to complete all items and those suspected of random responding, a total of 1415 participants (mean age = 20.52 ± 1.03), including 696 males (mean age = 20.71 ± 0.98) and 719 females (mean age = 20.34 ± 1.04), were included in the analysis. Their scores for subjective economic status (mean = 5.52) and subjective social status (mean = 6.15) both ranged from 1 to 10 and were normally distributed.

This research complies with the ethical standards of the Declaration of Helsinki and has been approved by the Ethics Committee of the Second Xiangya Hospital of Central South University.

Measure

The Rosenberg’s Self-Esteem Scale (RSES)

The RSES is a 10-item self-report scale that measures self-esteem level.¹ Each item is scored on a 4-point Likert scale ranging from 1 (totally disagree) to 4 (totally agree). Notably, the original RSES contains five positive wording items and five negative wording items, of which Item 8 was scored in reverse. However, considering the cultural setting, item 8 of the Chinese version of the RSES was scored positively, since the Chinese tended to understand it in a positive way.²⁴ Therefore, in the Chinese version of the RSES, Items 3, 5, 9, and 10 are scored in reverse, so that higher scores on these four items indicate lower self-respect levels. The total RSES score ranges from 10 to 40, with higher scores indicating positive self-esteem or a general perception of self-worth.

The Short Form of Mood and Anxiety Symptoms Questionnaire (MASQ-SF)

MASQ-SF was used to evaluate a range of symptoms related to depression and anxiety.²⁵ The MASQ-SF includes 62 items, with the score for each item ranging from one (strong disagreement) to five (extreme agreement). The items were divided into four subscales: General Distress (GD), Anhedonic Depression (AD), General Anxiety (GA), and Anxious Arousal (AA). The Chinese version of the MASQ-SF has been shown to have adequate psychometric properties. In this study, the Cronbach’s α α values of the MASQ-SF was 0.94 in the total sample, 0.94 in males and 0.93 in females, respectively.

Perceived Stress Scale (PSS)

PSS was employed to evaluate the level of perceived stress during the past month. It consists of 10 items rated on a Likert 5-point scale (0–4) with a score range of 0–40, with higher scores reflecting a higher level of perceived stress. Cronbach’s α of the PSS was 0.73 in the whole sample, 0.74 in males and 0.72 in females.

Quality of Life Enjoyment and Satisfaction Questionnaire-Short Form (Q-LES-Q-SF)

The Q-LES-Q-SF is a 16-item self-report questionnaire that evaluates overall enjoyment and satisfaction experienced in the past week.^26,27 The responses of the Q-LES-Q-SF were scored on a 5-point scale (1–5), where the first 14 items summed the total score, while the 2 last items involving medications and overall life satisfaction were considered independently. Higher total scores indicate better life satisfaction (ranging from to 14–70). The Cronbach’s α α of the Q-LES-Q-SF was 0.88 in the whole sample, 0.88 in males and 0.87 in females.

Statistical Analysis

Statistical analyses were conducted using SPSS 26.0, and Mplus 7.4. The skewness and kurtosis values for each item of RSES, PSS, Q-LES-Q-SF and majority items of MASQ-SF were within acceptable limits, with absolute values less than 1, indicating that the data of these items were normally distributed. The tolerance values (0.73 ~ 0.86) and variance inflation factor (VIF) values (1.16 ~ 1.37), calculated based on total scores of each scale, were within acceptable ranges (tolerance > 0.1, VIF < 10), indicating no risk of multicollinearity. Additionally, Harman’s single factor test, conducted using all items across four scales, revealed 18 factors with eigenvalues greater than 1 before rotation, and the first factor explained 18.20% of the variance, which was below the critical threshold of 40%, suggesting that common method bias was not a severe concern in the data for this study.

A series of Robust Maximum Likelihood CFAs were first conducted to examine the factorial structure of the eight competitive models of the RSES. As shown in Figure 1, the RSES-GSE Model is unidimensional with a global self-esteem (GSE) factor. The RSES-PN Model has two relevant factors that represent positively and negatively worded items, respectively. The CTCM-PN Model is a CTCM bi-factor model containing a GSE factor and two respective factors that describe the positive and negative method effects. The CTCM-P Model and CTCM-N Model are CTCM bi-factor models with a GSE factor and only a positive OR negative factor for the method effect. The CTCU-PN Model is a CTCU model included one trait factor and correlated uniqueness among positively and negatively worded items. The CTCU-P Model and CTCU-N Model are CTCU models with trait factors and correlated uniqueness among positively or negatively worded items, respectively. The variance of factors was restricted to 1 in all models, and factors were forbidden from being correlated in the CTCM-PN, CTCM-P, and CTCM-N models. All factor loadings were freely estimated. The following indices were selected to evaluate the model fit: ratio of χ² to df (χ²/df), Comparative Fit Index (CFI), Tucker Lewis Index (TLI), root mean square error of approximation (RMSEA), Standardized Root Mean Squared Residuals (SRMR), Akaike Information Criterion (AIC), and Bayesian Information Criterion (BIC). Generally, χ²/df < 5, CFI > 0.90, TLI > 0.90, RMSEA < 0.08, SRMR < 0.08, and AIC and BIC values that were as small as possible were considered acceptable.²⁸ The best-fit model was selected by comparing the fitting indices.

Second, the gender invariance and latent means invariance of the factorial structure of the RSES were tested with the best fitting model using Robust Maximum Likelihood multigroup CFAs. The baseline model was set with no constraint for configural invariance, the weak invariance model was configured to have invariant factor loadings between groups to confirm that the RSES shared identical structure and meaning across gender; the strong invariance model added the intercept invariance of items while the weak invariance was satisfied, and the strict invariance model then imposed the residual invariance of items upon the assumption that the strong invariance was held. A successful demonstration of these models would yield measurement invariance across sexes. Models of variance-covariance invariance were then examined to test for factorial invariance. Subsequently, latent mean invariance was examined to determine whether the latent means were equal across the samples. If all the models were established, the optimum model of the RSES was confirmed to have factor invariance. Model fit was assessed using several indices: BIC, χ², χ²/df, CFI, TLI, and RMSEA with significance χ², CFI > 0.90, TLI > 0.90, and RMSEA < 0.08 are regarded as satisfactory. For model estimation, changes in CFI (ΔCFI) ≤ 0.01, changes in TLI (ΔTLI) ≤ 0.01, and a smaller BIC are considered acceptable invariance.²⁹ Finally, manifest means that differences in RSES scores across the sexes were examined.

Third, to examine the internal reliability of the RSES, the Cronbach’s α and the McDonald’s ω coefficients were both computed.

Fourth, criterion validity was confirmed by comparing group differences in anxiety and depression (high vs low self-esteem) and validating the vulnerability model of depression. Participants with the highest 25% of the total RSES scores were defined as the high self-esteem group, whereas those with the lowest 25% of the scores were defined as the low self-esteem group. The independent samples t test was then used to compare group differences in MASQ-SF scores. Cohen’s d value was used to estimate the effect size.³⁰ As a specific instance of structural equation modeling, path analysis was employed to examine the mediating roles of self-esteem and depressive symptoms in the relationship between perceived stress and life satisfaction. We initially tested several candidate models: perceived stress → life satisfaction; perceived stress → self-esteem / depression → life satisfaction; perceived stress → self-esteem → depression → life satisfaction. Based on the model fit indices and the significance of individual paths, we refined our models to arrive at the most parsimonious and well-fitting model. The model-fitting criteria for path analysis were χ²/df < 5, CFI > 0.90, TLI > 0.90, RMSEA < 0.08, and SRMR < 0.08.

Results

Model Fitting of RSES

As seen in Table 1, the RSES-GSE Model showed a poor fit, with both CFI and TLI lower than 0.90 and RMSEA (= 0.164) much above the listed criterion. The fit of the RSES-PN Model improved, but its CFI, TLI, and RMSEA did not meet these standards. The fit indices became much more satisfactory after the method effects were considered, except for the CTCU-PN Model, which could not be identified using Mplus. All remaining models met the criteria of CFI and TLI (all > 0.90), while only the CTCM-PN and CTCU-P models met RMSEA norms (0.061 for Model 3, 0.082 for Model 7, both < 0.100). Overall, the CTCM-PN Model, with a significantly smaller RMSEA and χ²/df and a larger df, was the optimum fitting model (χ²/df = 6.237, CFI = 0.986, TLI = 0.974, RMSEA = 0.061). As the CTCM-PN Model was selected, the fit indices for each gender indicated that the fit in males (CFI = 0.988, TLI = 0.979, χ² = 74.420) was better than that in females (CFI = 0.979, TLI = 0.962, χ² = 128.938), as shown in Table 2. Additionally, the loadings on the GSE factor range from 0.323 to 0.890.

Table 1 Confirmatory Factors Analysis for Alternative Models

Table 2 Fit Indices of the CTCM-PN Model in Each Gender

Measurement Invariance Across Gender of the CTCM-PN Model

As shown in Table 3, the results of the measurement and structural invariance analyses of the CTCM-PN Model across genders all met the criteria (all χ²/df < 5, CFI > 0.90, TLI > 0.90, RMSEA < 0.08, and SRMR < 0.08). Moreover, weak invariance, strong invariance, strict invariance, factor variance-covariance, and latent mean invariance of the CTCM-PN Model between males and females were established, with ΔCFI and ΔTLI < 0.01. The difference in manifest self-esteem levels between males and females was not statistically significant (P = 0.34, t = 0.95).

Table 3 Fit Indices for Measurement Invariance Tests of CTCM-PN Model of RSES Across Gender

Reliability of RSES

The Cronbach’s α of the RSES was 0.92 for the whole sample, with 0.91 for males and 0.92 for females. The Cronbach’s α of the positively worded items of the RSES was 0.91 for the whole sample, 0.91 for males, and 0.92 for females. Correspondingly, the Cronbach’s α α of the negatively worded items of the RSES was 0.83 for the whole sample, 0.83 for males, and 0.83 for females. Meanwhile, the McDonald’s Omega (ω) coefficients were from 0.84 to 0.92 for each subscale of the RSES.

Criterion Validity of RSES

According to the criteria mentioned above, 333 subjects were classified into the high-level self-esteem group (total scores of RSES in the top 25%), and 338 subjects were classified into the low-level self-esteem group (total scores of RSES in the last 25%). The results of the independent samples t test indicated significant group differences in MASQ-SF’s scores, with the effect size ranging from 0.45 to 1.19 (see Table 4).

Table 4 Differences in MASQ-SF’s Scores Between the High-Level and the Low-Level Self-Esteem Groups

As illustrated in Figure 2(a) and Table 5, the model fit indicators of the original model all met the fit criteria, while the path from self-esteem to quality of life was not significant. Therefore, that path was removed, and the model fit was reversed. The fit results of the modified model are presented in Figure 2(b) and Table 5, where all paths were significant and the model fit indicators fulfilled the fit criterion.

Table 5 Standardized Indirect Effects and 95% Confidence Intervals of Structural Equation Model

Figure 2 Results of Path Analyses. (a) Results of the original model; (b) Results of the modified model. Notes: ***P < 0.001

Discussion

In the current study, we comprehensively investigated the factorial and criterion validities of the Chinese version of the RSES in undergraduates using the following analyses. Initially, we compared the model fittings of eight competitive models using CFA with a sample of undergraduates, revealing that the CTCM-PN Model best fitted the data. We then confirmed that the gender invariance of the CTCM bi-factor model was established at configural, metric, scalar, and strict levels, and further confirmed latent means invariance. Next, we measured the Cronbach’s alpha α of the RSES and confirmed that it had acceptable reliability among Chinese undergraduates. Subsequently, we verified the criterion validity of the RSES by comparing anxiety and depressive symptoms between individuals with high and low self-esteem. Finally, we provided further robustness to the criterion validity of the RSES by unraveling the essential contribution of self-esteem to alleviating depressive symptoms and ultimately enhancing life satisfaction from the perspective of vulnerable models.

Consistent with prior research, the current study supported the CTCM bi-factor model, including a GSE factor with two positive and negative factors (CTCM-PN) that fitted optimally.¹⁶ In the CTCM-PN Model, all items had significant and considerable loadings on the GSE factor as they all reflected self-esteem levels. With this model, it is possible to distinguish the items affected by wording effects. In addition, it was observed that negatively worded items were dominant in CTCM models, whereas the opposite pattern existed in CTCU models in this study. However, in most previous studies, both CTCU and CTCM models showed a tendency for models with negatively worded items to have a better fitting.^5,10 The different dominance of items with different valences in this study could be attributed to the unsatisfactory fitting of CTCU models, which were less stable and unable to analyze method effects directly, whereas CTCM models were more suitable for reflecting convergent, discriminant, and criterion validity, where negatively worded items were more important.⁹

Because the CTCM-PN Model of RSES was the optimum model, gender invariance analyses were then performed using multigroup CFAs. Controlling for method effects introduced by negatively and positively wording, the CTCM-PN model of the RSES was configurationally, weakly, strongly, and strictly invariant across genders, implying that self-esteem measured by the Chinese version of the RSES shared identical potential theoretical constructs and psychological implications across genders. Measurement invariance of the RSES across genders has been confirmed in other countries,^15,16 and the present study further confirmed the factorial invariance by verifying the factor variance-covariance invariance, which not only confirms the robust measurement properties of the RSES in the Chinese population but also establishes the foundation for further applied research.

Differences in latent means and manifest scores were then examined upon the establishment of factorial invariance, which uniformly suggested that self-esteem levels of male and female undergraduates were comparable. This finding was consistent with the results of previous research showing latent mean invariance of RSES in undergraduates.¹⁷ However, it contradicted with several studies comparing observed scores, which reported significantly higher self-esteem levels in females than in males.^14,31,32 This disparity may stem from the comparison of arms, which means that comparisons based on measurement invariance eliminate systematic errors caused by different units or reference points across groups, reflecting authentic group differences.³³ However, direct comparisons of manifest means used in prior research could not eliminate the systematic errors caused by method effects; therefore, a more careful interpretation of manifest mean differences is required.

The findings of the criterion validity analyses provided further evidence for the extended utility of the RSES within the realm of mental health assessment. Consistent with prior research,¹⁶ the current study revealed that individuals with comparatively lower levels of self-esteem would experience higher levels of anxiety and depression. Typically, individuals with lower self-esteem and negative self-perception are at higher risk of self-derogation, sociopathic behavior, anxiety, depression, and other mental problems.¹⁶ To further investigate the implications of self-esteem on mental health and daily life of undergraduate students, a vulnerability model for depression was validated in the current research. Consistent with the results of previous studies,²⁰ our findings revealed a well-fitted vulnerability model that explained the relationship between self-esteem and depression. Undergraduates with high self-esteem are more likely to alleviate stress-induced depressive symptoms and ultimately enhance their life satisfaction. This finding provides ideas for subsequent efforts to improve the physical and mental health of undergraduates, especially in what is still the post-epidemic period with relatively high levels of mental stress.

Limitation

Despite the advantages of this study, it has several limitations must be mentioned. First, the participants included in this study were drawn from a demographically homogeneous sample of college students, suggesting that future research with more diverse and heterogeneous samples would be beneficial in ascertaining the broader applicability of the CTCM bi-factor model across various populations. Second, the equivalence analysis was limited to the gender dimension in this study, further research should extend this analysis to include other dimensions, such as temporal stability, age variations, and clinical populations. Third, this study did not specifically address gender differences in self-esteem, future research should endeavor to increase sample size to allow for a more detailed examination of gender disparities in self-esteem levels. This examination, grounded in positive psychology, could inform targeted interventions to enhance self-esteem and well-being across gender. Such inquiry is crucial not only for elucidating the complex interplay between gender and self-esteem but also for developing nuanced strategies that promote psychological resilience among diverse populations.

Conclusion

This study systematically confirmed the factorial and criterion validities of the RSES within a Chinese undergraduate sample by comparing the model fitting of eight models of the RSES, establishing the gender invariance and latent mean invariance of the CTCM bi-factor of the RSES, and verifying the criterion validity of the RSES. These findings pave the way for future research on self-esteem measurement within the Chinese cultural context, contributing to a deeper understanding of the nuances of self-esteem in diverse cultural settings.