Mapping EQ-5D-5L Score From SGRQ in Patients with Asthma and/or COPD in NOVELTY

Introduction

Preference-based utility measures are often used in economic evaluation of health technologies as they enable the comparison of such technologies across different disease areas.¹ An example of a preference-based instrument that is widely used for utility elicitation is the EuroQol 5-Dimension 5-Level (EQ-5D-5L).² EQ-5D-5L is a simple, generic measure of health status covering the following five domains: mobility, self-care, usual activities, pain/discomfort and anxiety/depression, with five response levels ranging from experiencing no problems to experiencing extreme problems.² In contrast, disease-specific symptom or functional scores are used in clinical studies across disease areas, for example, as key endpoints in clinical trials.³ For airways diseases, the St George’s Respiratory Questionnaire (SGRQ) has been widely used in clinical studies as a comprehensive and validated disease-specific instrument for the evaluation of patient health status.^4–8 The SGRQ is a 50-item questionnaire with questions covering three domains of symptoms, activity, and impacts (psychosocial).^9,10

Where EuroQoL 5-Dimension (EQ-5D) utility scores from validated instruments are not available, they can be predicted by mapping responses to disease-specific scores.¹¹ Given the extensive adoption of SGRQ to evaluate patient health status in multiple clinical studies,^4–8 algorithms for mapping SGRQ score to preference-based utility measures can have broad applicability. One such mapping algorithm has been developed and validated by Starkie et al in patients with moderate-to-very severe chronic obstructive pulmonary disease (COPD).¹² Starkie et al proposed a mapping formula that included only SGRQ total score, its squared value, and sex.¹²

The Starkie et al algorithm was derived using data from Towards a Revolution in COPD Health (TORCH), a clinical trial of patients with COPD from around two decades ago.¹² The validity of this mapping algorithm in more contemporary samples, as well as in patients with asthma, is not known, even though the algorithm has subsequently been used in patients with severe eosinophilic asthma.¹³ Furthermore, more nuanced mapping, such as algorithms that use SGRQ domains rather than total score and include additional patient characteristics, has the potential to enhance the accuracy of mapping and its applicability to broader populations with obstructive airways disease.

The NOVEL observational longiTudinal studY (NOVELTY; NCT02760329) provides a unique opportunity to study a large, diverse, contemporaneous cohort of patients with physician-assigned asthma and/or COPD managed in primary and secondary care settings across 18 countries.¹⁴ The NOVELTY study has demonstrated substantial heterogeneity within, and overlap between, populations of patients with asthma and COPD.¹⁵ While marked heterogeneity has been observed in patients with COPD versus asthma, heterogeneity within diagnosis groups, such as for health-related-quality of life, suggests current diagnostic and severity classifications poorly differentiate between clinically-important phenotypes.¹⁵ As such, access to this diverse cohort is of particular benefit when testing the generalizability of SGRQ to EQ-5D-5L mapping algorithms for health economic evaluations across a broad spectrum of patients with chronic airways disease.

Concordantly, the aim of this study was to validate the Starkie et al algorithm¹² in patients with asthma and/or COPD in NOVELTY, and examine whether the inclusion of additional covariates would improve the accuracy of predictions.

Materials and Methods

Study Design and Population

NOVELTY is a global, prospective observational study of patients with a physician-assigned diagnosis of asthma and/or COPD.¹⁴ Details of the NOVELTY study design and patient population have been reported previously.^14,15 Briefly, NOVELTY included patients aged ≥18 years (or ≥12 years in some countries) from primary care and specialist centers in 18 countries in the Americas, Asia, Europe, and Oceania.

Patients completed a baseline visit and subsequent yearly visits for three years. Patient-reported measures, including the EQ-5D-5L and SGRQ, were completed by patients at baseline and at each follow-up visit. The EQ-5D-5L scale included statements across five domains and used an algorithm that converted the EQ-5D-5L questionnaire responses to EQ-5D-5L scores.^16,17 The SGRQ is a 50-item questionnaire scored on a 0–100 scale, with higher scores indicating worse health status.⁹

Baseline and Year 1 data were used as the development sample. Year 2 and Year 3 data were used as the validation samples. The intention was for all individuals to contribute to the parameter estimation to optimally use the available data.¹⁸ To be included in the development or validation samples, a patient was required to have both EQ-5D-5L and SGRQ scores available at the yearly visit in question. Given the small fraction of missing data for physician-assessed severity, a complete case approach was used whereby patients with missing data for this variable were excluded from the development of models 3, 4, 5 and 6, and from the validation samples. This methodology is in line with Jakobsen et al 2011, which specifies that a complete case analysis may be used if the proportions of missing data are below approximately 5%.¹⁹

Longitudinal Linear Mixed Models

EQ-5D-5L scores were calculated using patient responses to the five questions and the R package eq5d,²⁰ applying the five-level crosswalk algorithm^16,17 and the UK value set.²¹

Six longitudinal linear mixed models were fitted to the development sample (baseline and Year 1), with EQ-5D-5L as the response variable. Each model had a different combination of covariates. Model 1 included the same covariates as those in the Starkie et al algorithm:¹² SGRQ total score, its squared value, and baseline sex. Model 2 included individual SGRQ domains (symptoms, activity, impact), their squared values, and sex. Model 3 included the same covariates as model 1 as well as five additional baseline covariates: age, body mass index, physician-assigned diagnosis (asthma, asthma+COPD, COPD), physician-assessed severity (mild, moderate, severe), and country (18 countries; 17 dummy-coded variables). Model 4 included the same covariates as model 2 as well as the five additional covariates included in model 3. Models 5 and 6 used backwards selection to sequentially remove the least significant baseline covariate from models 3 and 4, respectively, until all were statistically significant at the α <0.05 level.

Mixed model repeated measures methodology was used to enable the accommodation of within-patient correlation among measurements.²² Restricted maximum likelihood and an unstructured covariance matrix were used to fit all models. An unstructured covariance matrix is the most general form of covariance structure.²² The development sample included only two time points, so this structure can be easily identified. Therefore, there is no requirement to consider reduced covariance structures, such as using random intercepts and slopes.²² Models accounted for correlations from repeated measures in the same patient over time by specifying that residual errors are correlated,²² and ceiling effects in EQ-5D-5L scores were not accounted for. More sophisticated models that do accommodate ceiling effects are possible²³ but were not adopted here. This was because our estimation sample was longitudinal and methods of this type require extension to accommodate correlated data.²³ In addition, our primary interest lay in whether Starkie et al’s linear model could be improved by including additional covariates. SGRQ scores were time varying covariates, with scores taken at either baseline or Year 1 corresponding to the visit of the EQ-5D-5L response variable.

Model performance was assessed by computing mean square error (MSE) values for predictions made in the two validation samples (Year 2 and Year 3) in the main and subgroup analyses (below); this methodological approach is supported by Brazier et al.²⁴ Smaller MSEs indicate better performing models. Validation sample predictions and MSEs were computed in subsets of patients who had data for all covariates included in the models, to ensure that all six models were validated in the same set of patients. In addition to MSE, models were also compared in terms of the proportion of variation in EQ-5D-5L explained.

Subgroup Analyses

Subgroup analyses were performed according to the physician-assigned diagnosis (asthma, asthma+COPD, or COPD), whereby models were developed in the subgroups and were subsequently validated. Patients with asthma+COPD were included in both the asthma and COPD subgroups.

Sensitivity Analyses

To assess the impact of missing EQ-5D-5L and SGRQ data, we performed a sensitivity analysis where these data were imputed. To assess the impact of alternative disease severity classification, we performed a sensitivity analysis where disease severity was determined based on the Global Initiative for Asthma (GINA) treatment step classification, as opposed to physician-assessed severity in the main analysis. In this analysis, the GINA 2017 treatment steps (1–5) were used as a categorical baseline covariate for severity instead of physician-assessed severity in patients with asthma and asthma+COPD.²⁵ These treatment steps represented the GINA guideline recommendations that applied at the time the data were collected. Patients for whom GINA treatment step was unknown were excluded from models 3 and 4, the stepwise procedure giving rise to models 5 and 6, and from the validation samples to ensure that all six models were validated using the same validation datasets.

In their original report, Starkie et al rounded coefficients of intercept, SGRQ, SGRQ², and sex to four decimal places. An additional comparison was performed in which the coefficients from the model that was subsequently selected for further analysis were rounded to the same number of decimal places,¹² to evaluate the potential loss of prediction accuracy due to rounding.

Results

Study Population

In total, 11,192 patients were included in the NOVELTY baseline population (Table 1 and Figure 1). Of these, 6813 patients with available EQ-5D-5L and SGRQ data were included in the baseline sample (mean age, standard deviation [SD] 60.3 (14.5) years; 47.7% male); 3546 had asthma, 872 had asthma+COPD, and 2395 had COPD. The Year 1 sample included 5607 patients with both EQ-5D-5L and SGRQ data (Tables 1 and 2) and together with the baseline sample comprised the development sample. The Year 2 validation sample and the Year 3 validation sample included 4593 and 4643 patients with both EQ-5D-5L and SGRQ data available, respectively (Tables 1 and 2). A summary of the EQ-5D-5L outcome data is shown in Table 2; the median (interquartile range) EQ-5D-5L score in the baseline sample was 0.795 (0.648–1), at Year 1 was 0.827 (0.664–1), at Year 2 was 0.836 (0.668–1), and at Year 3 was 0.819 (0.671–1). The mean (SD) SGRQ score in the baseline sample was 35.4 (22.2), at Year 1 was 32.3 (22.8), at Year 2 was 31.6 (23.0), and at Year 3 was 30.4 (22.5).

Figure 1 Number of patients included in this analysis. Abbreviations: COPD, chronic obstructive pulmonary disease; EQ-5D-5L, EuroQoL 5 Dimensions 5 Levels Health Questionnaire; SGRQ, St George’s Respiratory Questionnaire.

Table 1 Baseline Demographics and Clinical Characteristics for Patients Included in the Baseline, Year 1, Year 2, and Year 3 Samples

Table 2 EQ-5D-5L Outcome Data and Missing Data in the Development and Validation Samples

Model Development

In models 1 and 2, sex, which was the only baseline covariate included, was statistically significant (Table 3). Backwards stepwise selection removed baseline age from models 3 and 4, resulting in models 5 and 6.

Table 3 Model Performance in All Patients, Patients with Asthma (and asthma+COPD), and Patients with COPD (and asthma+COPD)

Model Validation

Validation MSE values indicated good performance and were similar across models 1–6 (Year 2: 0.0302–0.0308; Year 3: 0.0272–0.0277) and were smaller than those obtained using the Starkie et al algorithm¹² (Year 2: 0.0340; Year 3: 0.0296), indicating better performing models (Table 3). EQ-5D-5L score sample variance was 0.0556 for Year 2 and 0.0524 for Year 3, indicating that just under 50% (Year 2: 45–46%; Year 3: 47–48%) of the total variation in the EQ-5D-5L scores is explained by the models.

Since validation MSEs were similar across all models, model 1 was chosen for further analysis given that this was the most parsimonious model. Model 1 performed well with estimated coefficients for all patients differing from those of the Starkie et al algorithm¹² (Table 4). Predictions were similar for model 1 and the Starkie et al algorithm when rounding to four decimal places, which is consistent with the rounding used by Starkie et al.¹² Furthermore, predictions lay in a feasible utility range (Figure 2).

Table 4 Estimated Coefficients for Model 1 Predictions in All Patients, Patients with Asthma (and asthma+COPD), and Patients with COPD (and asthma+COPD), Compared with Starkie et al

Figure 2 Comparison of predictions from model 1 with those from the Starkie et al algorithm. Abbreviations: EQ-5D-5L, EuroQoL 5 Dimensions 5 Levels Health Questionnaire; SGRQ, St George’s Respiratory Questionnaire. Note: Starkie et al algorithm from: Starkie et al. Value Health 2011;14:354–6012. Grey dots depict all available data points from both validation samples (Year 2 and Year 3) to support in interpreting the predictive power of the model.

Model Performance Within Disease Groups

For the asthma subgroup, 4,418, 3,578, 2,900, and 3,075 patients with asthma (and asthma+COPD) were included in the baseline, Year 1, Year 2, and Year 3 samples, respectively. For the COPD subgroup, 3,267, 2,762, 2,321, and 2,193 patients with COPD (and asthma+COPD) were included in the baseline, Year 1, Year 2, and Year 3 samples, respectively. As with the main analysis, sex was a statistically significant baseline covariate in models 1 and 2 in both the asthma and COPD subgroup analyses. Backwards stepwise selection removed different baseline covariates from models 5 and 6 in each subgroup analysis (Table 3).

Model 1 performed well in the patient subgroups, with performance similar in all models in the asthma and COPD subgroups (Table 3). As with the main analysis, estimated coefficients for model 1 predictions in patients with asthma (and asthma+COPD) and patients with COPD (and asthma+COPD) differed from those of the Starkie et al algorithm¹² (Table 4).

Estimated coefficients were different for the COPD subgroup compared with the main and asthma subgroup analyses; however, model 1 predictions were similar across the main and subgroup analyses (Figure 3), although some noticeable differences were observed for low (<20) and high (>60) SGRQ scores, due to the SGRQ quadratic coefficients having the most impact at the data extremities. To explore if patients with COPD require different coefficients, a final model was fitted to all patients as in model 1 but also including an indicator for the patient having COPD at baseline as a covariate, as well as interactions between this indicator and the other model coefficients. The main effect of the COPD indicator was estimated to be –0.024 and was statistically significant (p = 0.014). This results in lower predictions for patients with COPD versus other diagnostic groups (Figure 3) and suggests that use of the lower intercept for patients with COPD may be important. The strength of the evidence for interactions of the COPD indicator with other covariates varied (sex p = 0.499; SGRQ total score p = 0.037; SGRQ total score squared p = 0.005).

Figure 3 Comparison of subgroup analyses of model 1 predictions. Abbreviations: COPD, chronic obstructive pulmonary disease; EQ-5D-5L, EuroQoL 5 Dimensions 5 Levels Health Questionnaire; SGRQ, St George’s Respiratory Questionnaire. Note: Grey dots depict all available data points from both validation samples (Year 2 and Year 3) to support in interpreting the predictive power of the model.

For example, using model 1 (all patients) for prediction, envisage a male patient with a SGRQ total score of 40. From the coefficients reported in Table 4, the predicted EQ-5D-5L is given by (to four decimal places). Supplementary Table 1 provides additional information about the model fits in Table 4 to allow calculation of covariances and standard errors for predictions and simulation of data from these models.

Sensitivity Analyses

While there were missing data for both EQ-5D-5L and SGRQ outcomes in the NOVELTY population (Figure 1), the sensitivity analysis that involved imputation of missing data showed that the key findings are robust when this is taken into account (see Supplementary Material — “Imputation of missing EQ-5D-5L and SGRQ values” and Supplementary Tables 2 and 3). Furthermore, the imputed EQ-5D-5L scores are not at the ceiling, suggesting that the lack of accommodation for ceiling effects in our modelling is not a paramount limitation.

In the asthma subgroup sensitivity analysis, the GINA treatment step was unknown for 494 patients with asthma and 59 patients with asthma+COPD, and these patients were excluded from models 3 and 4, the stepwise procedure giving rise to models 5 and 6, and from the validation samples as described in the sensitivity analysis section of the Methods. The conclusions from the asthma subgroup analysis were found to be robust when using GINA 2017 treatment step instead of physician-assessed severity as a covariate. Backwards stepwise selection removed the baseline covariate GINA 2017 treatment step from models 3 and 4 (Supplementary Table 4).

Discussion

In this work, we developed an algorithm to map the SGRQ to EQ-5D-5L and evaluated model performance against a previously published algorithm across a spectrum of patients with chronic airways disease. Validation MSEs indicated good performance and were similar across models. The models had reasonable predictive power, explaining approximately half of the variation in EQ-5D-5L scores. The covariates used by Starkie et al¹² (SGRQ total score, its squared value, and baseline sex) were adequate for mapping in this patient population. Including additional baseline covariates and SGRQ domains resulted in a similar model performance to the Starkie et al algorithm, although the coefficients used in our analysis may be more suitable for situations where the patient population is more similar to that included in the NOVELTY study, versus those used in the Starkie et al algorithm.¹²

Our results have important implications for research. We confirm that the Starkie et al mapping algorithm, developed based on data collected >15 years ago,¹² performs well using contemporary, longitudinal, multi-country data. The observation that the adaptations made in this analysis only slightly improved the performance of the Starkie et al algorithm,¹² suggests that it is adequate for mapping in patients with asthma and/or COPD, and since the SGRQ is widely used in clinical research in airways diseases,^4–8 the mapping algorithm can have wide applicability. The Starkie et al algorithm¹² was developed in patients with moderate-to-very severe COPD from the TORCH clinical trial, whereas our analysis included a much broader population of patients from NOVELTY with physician-assigned asthma, asthma+COPD, or COPD, with a range of physician-assessed severities, from real-world primary care and specialist settings in 18 countries. Although the findings for the subgroup analyses are slightly unclear in terms of whether or not patients with COPD require their own set of coefficients, data suggest that the algorithm derived from all patients, regardless of their diagnosis, can also be applied to patients with COPD. Conclusions were robust when using GINA 2017 treatment steps instead of physician-assessed severity as a covariate but the use of GINA treatment steps resulted in more missing covariate data versus physician-assessed severity and, as GINA treatment step was not selected in models 5 and 6, it does not appear to be a useful predictor.

A noteworthy observation is that Starkie et al’s¹² decision to only report their coefficients to four decimal places, and so the coefficient of SGRQ² to the first significant figure, may have resulted in a loss of model performance. The Starkie et al¹² predictions lay in between our model, where we report all decimal places for the coefficients provided by the statistical software, and our rounded model, where we only report coefficients to four decimal places; predictions for our model were more similar to those of Starkie et al when rounding to four decimal places. This highlights that numerical precision of reporting coefficients is critical when making predictions, as SGRQ total score squared can become large. When making comparisons, one methodological advantage of our study is the use of mixed model repeated measures and using their ability to predict future patient outcomes to validate them; when making predictions in the validation samples, no use of the estimated correlation structure is made, resulting in a more equitable comparison between the Starkie et al model¹² and our model.

Some limitations of this work should be acknowledged. As NOVELTY is a longitudinal study, the development and validation samples were generated using data for the same patients across different yearly timepoints. The drawback of this approach is that the development and validation samples were not independent of each other, resulting in data correlation between the samples, making comparisons with other algorithms derived from entirely different datasets difficult. However, splitting the development and validation samples in this way can improve the model’s predictive power by taking advantage of the longitudinal nature of the data.¹⁸ Future mapping analyses should use samples independent from those reported here and by Starkie et al¹² to validate the mapping algorithms and coefficients should be reported to a greater numerical accuracy than four decimal places. Furthermore, accounting for the ceiling effect of EQ-5D-5L in the modeling might improve the accuracy of the mapping, albeit at the cost of added complexity. Additionally, although validation across 18 countries means that our algorithm is likely applicable to diverse settings, inclusion of country as a covariate with the UK tariff for mapping may also generate conceptual challenges. However, an advantage of using the most parsimonious model (model 1) is that the resulting model predictions are widely applicable as they do not depend on variables such as country that may make their application more challenging in other datasets. While our findings show that EQ-5D-5L utility scores can be predicted through mapping from SGRQ total score, as previously described by Starkie et al,¹² it should be noted that EQ-5D-5L utility scores derived directly from validated instruments should be used, where possible.

In conclusion, our analysis validates the use of Starkie-type mappings in patients with asthma and/or COPD. The inclusion of additional baseline covariates and SGRQ domains resulted in a similar model performance. Our updated coefficients derived from NOVELTY have potential for use in health economics models due to the broader recruited population versus Starkie et al.¹² In addition to marginal improvement in the accuracy of mapping (in terms of lower MSEs), a particular advantage of NOVELTY-derived coefficients in health economic modeling is their applicability to patients with asthma and asthma+COPD. Our mapping algorithm could also be considered for use in other respiratory diseases in the future.