Developing a Process for Preference Measures in Pediatric Growth Hormone Deficiency: Challenges and Solutions

Introduction

Patient experience data (PED) capturing the patient voice is gaining increasing recognition to provide potential evidence across the drug development continuum and for use in risk/benefit analysis to evaluate new drugs and inform reimbursement and pricing decisions.

Patient experience data is defined as data that are collected by any persons and are intended to provide information about patients’ experiences with a disease, treatment, or condition and includes the experiences, perspectives, needs, and priorities of patients.¹ The United States (US) Food and Drug Administration’s (FDA) position on the importance of PED is echoed by the European Medicines Agency, which states, “Throughout the drug development process, working with patients and caregivers to learn about patient perspectives can be valuable in addressing specific questions to inform development programs and related regulatory decision making”.²

One type of PED is patient treatment preference information (PPI) which includes factors such as efficacy, side effects, and impacts on daily life and functioning. PPI defined by the FDA guidance is the relative desirability (what is valued most) or acceptability (perspective on risk/benefit) to patients and care-partners of alternative health interventions. Simply defined, PPI evidence is an assessment of acceptability of drug A compared to drug B, based on rating desired attributes.^3,4 Methodologies for assessing preference can be either qualitative or quantitative, ranging from focus groups to discrete choice experiments, with approximately 32 different methodologies identified in the research literature.^4,5 Many of these methods may require a person to make a series of judgments regarding treatment attributes in “trade off” scenarios which can be complex to design and have been criticized for being difficult for patients to complete and for policy makers to interpret.^6,7 Further, some conjoint analysis preference methodologies require stated preferences for hypothetical scenarios based on attributes which may or may not have been experienced by the person completing the questionnaire.^8,9 Although no one methodology may be applicable across the drug development process, greater debate and consensus on methodology for the development and use of PPI questionnaires in a systematic and scientifically valid way is needed.^10–12

Pediatric growth hormone deficiency (PGHD), is a rare condition that occurs when inadequate growth hormone (GH) is produced. Patients with PGHD typically experience growth failure with potential psychosocial impacts. They may also face long-term health risks, including increased abdominal fat from altered lipid metabolism, dyslipidemia (elevated LDL, triglycerides, reduced HDL), decreased lean body mass, and reduced bone density.¹³ The standard-of-care treatment for PGHD is daily subcutaneous (SQ) injections of recombinant human GH. Long-acting GH therapies providing a once weekly SQ injection anytime during the day have recently been approved.^14–16 In addition, novel oral GH secretagogues are currently being investigated. Each therapy has a different mode and timing of administration and scheduling requirements. Thus, understanding preferences for PGHD treatment can be a critical factor in improving treatment adherence,^17,18 evaluating the treatment risk/benefit profile, and providing evidence for regulatory, clinical practice, and pricing decision-making. There are limited disease-specific measures of treatment preference for PGHD available, and those available either do not ask about the attributes which form preferences or have not been developed including the patient voice.

The aim of this study was to delineate a rigorous prototype process for developing easily administered and interpretable patient-centric treatment preference questionnaires, which can be used when respondents have experienced either one or multiple treatment options. Pediatric growth hormone deficiency treatment was used as a model for the study.¹⁹ This process draws from aspects of best practices for patient-reported outcome (PRO) measure development,²⁰ as well as the underlying concept of attributes on which preferences are based on in discrete choice methodologies. Child GH treatment is used as the case study for the questionnaires’ development.

Methods

Independent Review Board (IRB) approval for study protocol LP22-PGHDPrefQ (IRB Tracking Number: 20230357) was received from WCG IRB. The research was conducted in accordance with the 1964 Helsinki Declaration and its later amendments or comparable ethical standards. Informed consent included participant’s agreement to publication with individual identities remaining confidential.

Establishing Content Validity Process

Methodology for this study included a literature review and concept elicitation (CE) interviews with clinical experts, caregivers of children with PGHD, and children with PGHD. These data were analyzed and used to develop a treatment preference questionnaire and a treatment attribute questionnaire. Intended meanings of all instructions and items were described in a draft item definition table. These questionnaires then underwent a transability assessment, cognitive debriefing (CD) assessment, were refined, and finalized. The item definition tables were developed to assist with interpretation of items for translators by providing definitions and intended concepts for all instructions and items, as well as lists of alternative wording when required for a given language. The study process flow chart is presented in Figure 1.

Figure 1 Study Process Flow Chart. Concept Elicitation and Cognitive Debriefing Assessment (Content Validity).

Literature Review

A literature review was conducted to search for relevant medical and social science literature. The PubMed (NLM) and EMBASE (ProQuest) databases were searched using terms and variant spellings of terms and appropriate subject headings/subheadings when possible. The key search terms included: preference, patient preference, rating scale, method, survey, measure, questionnaire and/or growth hormone deficiency. The literature review was used to inform the development of the semi-structured interview guides for the CE interviews.

Participant Recruitment and Eligibility Criteria

Recruitment

Caregiver and child participants were recruited from private practice pediatric endocrinologists or from clinical sites participating in a Phase 2 trial of an investigative oral treatment for PGHD (LUM-201-01 Trial; NCT04614337; Clinical Trials.gov; https://www.clinicaltrials.gov/; registered on 02 November 2020) and by a professional recruitment agency using their proprietary databases (patient panels) as well as via clinicians and advocacy groups. Confirmation of growth hormone deficiency (GHD) diagnosis in the form of a GH medication label, or a letter/clinic note from a healthcare provider confirming the child’s diagnosis, was required for any participants not referred by a physician.

Eligibility Criteria

Clinical expert participants were pediatric endocrinology physicians (MD) or nurse practitioners (NP) with at least 5 years of experience in the specialty and caring for a minimum of 25 children with GHD in a clinical setting. Experts were interviewed to gain a deeper understanding, from the clinical perspective, of the preferences, likes, and dislikes that caregivers and children have for PGHD treatment.

Eligibility criteria for children and caregivers were the same for all recruitment sources for both the CE interviews and the independent sample for the CD assessment interviews. Eligible caregivers were the caregiver of a child with a diagnosis of idiopathic GHD aged ≥ 3 years and aged ≤ 11 years (girls) and ≤ 12 years (boys). Caregivers were required to be currently living with the child, actively involved in the child’s day-to-day care and GHD treatment, willing to provide informed consent to participate in the interview, and able to read and communicate in English. Eligible children had a diagnosis of idiopathic GHD, were aged ≥ 10 years and aged ≤ 11 years (girls) and ≤ aged 12 years (boys), before bone growth plate closure had occurred, and were able to read and communicate in English.

Concept Elicitation Interview Data Collection

Concept elicitation data were collected via individual telephone interviews or virtually by video conference (cameras off), using semi-structured interview guides. The interview guides used open-ended questions to elicit experiences related to the child’s PGHD treatment. Questions were framed to query what attributes of treatment were preferred (liked vs disliked) in terms of the three pillars of treatment satisfaction (convenience, efficacy, and side effects)²¹ as well as interference in daily life, impacts on emotional well-being, and treatment compliance. Interviews were iterative where information from interviews was used to inform subsequent interviews. Interviews lasted approximately 60 minutes and were audio recorded and transcribed verbatim. Interviews were conducted by two experienced qualitative researchers who participated in a training to ensure similarity in how questions and probes in the interview guide were asked. Additionally, they met regularly to compare interview notes and any revisions needed.

Concept Elicitation Interview Qualitative Data Analysis

Data from the CE interviews were analyzed based on an adapted grounded theory approach.^22,23 Adapted grounded theory is a methodology in which concepts and theory are developed in a manner that is “grounded” in the qualitative data analysis based on participants’ words and meanings, while acknowledging existing clinical and expert knowledge. All interview transcripts were analyzed for content by theme using the Dedoose qualitative and mixed methods analysis software program (Dedoose© Version 9.0.90 2023). A preliminary code list was constructed and then emerging concepts that arose during the coding process were added, and previously coded transcripts were evaluated for new concepts. Throughout the coding process, concepts/codes were organized into categories encompassing larger themes and sub-themes. Thematic saturation analysis was conducted separately for child and caregiver interviews to ensure that all important and relevant concepts were covered and considered reached when 95% of concepts were covered. All coding and analysis were conducted by the same person.

Item Generation and Translatability Assessment

An item generation meeting was then held with the entire research team to review the analysis of the CE data and generate draft items for the measures. These drafts then underwent a translatability assessment to determine if the words used could be easily translated into non-English languages.

Cognitive Debriefing Assessment Interview Data Collection

The draft measures then underwent cognitive debriefing assessment. Individual interviews were conducted with caregivers and children by telephone or virtually by video conference (cameras off) and lasted approximately 60 minutes. Participants’ thoughts on the meaning of all instructions and items were elicited and compared to the intended meanings outlined in the draft item definition table. A “think aloud” method, as well as verbal probing, was used to ask respondents questions regarding the relevance and importance of the questionnaires’ items and the meaning of items and instructions.

Assessment interviews were conducted in blocks of 3 participants. After a block was completed, the findings were reviewed and suggested modifications were identified. An updated version of the questionnaire was then created for use in the following block. This process repeated until it was determined that the readability and relevance were acceptable based on consensus agreements among respondents in an entire block. The decision to change an instruction, item, or response option wording on the questionnaire was typically made when 2 participants had similar comments or if the change was viewed as an improvement.

Results

Expert Interviews

Five clinical experts (3 MDs and 2 NPs) participated in individual interviews. The experts had a mean of 16.4 years’ experience practicing in their current specialty (SD, 7.5; range, 7–28). Three (60.0%) worked in private practice, and two (40.0%) worked in a hospital setting. On average, experts spent most of their time in clinical practice (66.0%, SD, 31.3), followed by research (23.0%, SD, 18.6), teaching (7.0%, SD, 13.0), and other tasks (4.0%, SD, 8.9). The estimated percentage of their patients who were diagnosed with idiopathic PGHD averaged 74.4% (SD, 30.0; range, 40–97).

Expert Interview Findings

Experts reported on their own and their patients’ treatment preferences and issues. Table 1 provides a summary of results.

Table 1 Expert Interview Findings

Caregiver Concept Elicitation Interviews

Sample Description

Table 2 presents the demographic characteristics of 15 caregivers of children with GHD who completed a CE interview and the demographic and general health characteristics of the children of caregiver interview participants.

Table 2 Demographic Characteristics and General Health Characteristics

Caregiver Interview Findings

Thematic saturation was assessed for the 15 caregiver interviews and identified 171 different concepts related to their general experiences with and preferences for GHD treatment for their children. Following the 14th caregiver interview, 165 concepts (96.5%) were mentioned, and thematic saturation was considered reached.

Based on the caregiver transcripts’ analysis, a wide range of treatment-related preferences, experiences, and impacts were identified. Table 3 presents a summary of the findings for themes and exemplary quotes.

Table 3 Caregiver Interview Findings

Child Concept Elicitation Interviews

Sample Description

Table 4 presents the demographic characteristics of 15 children with GHD who participated in a CE interview and the caregiver-reported general health characteristics of the child interview participants.

Table 4 Demographic and General Health Characteristics: Child Concept Elicitation Participants

Table 5 presents the demographic characteristics of the caregivers of the children who completed a CE interview.

Table 5 Demographic Characteristics: Caregivers of Child Concept Elicitation Participants

Child Interview Findings

Thematic saturation for the 15 child interviews identified 104 different concepts related to their GHD treatment preferences and experiences. Following the 12th interview, saturation was considered reached with 99 concepts mentioned (95.2%).

From the child interview transcripts’ analysis, treatment preference and impact themes were identified. Table 6 presents a summary of findings for themes, with exemplary quotes.

Table 6 Child Interview Findings

Questionnaire Development

Based on the analysis of the CE interview transcripts, two questionnaires were developed: the GHD-Preference Measure (GHD-PRM) and the GHD-Attribute Measure (GHD-ATM). Two versions of each of the questionnaires were generated; one to be completed by children with GHD aged ≥ 10 years to ≤ 12 years and one to be completed by the caregivers of children with GHD aged ≥ 3 years to ≤ 12 years.

The GHD-PRM was intended to be used when the respondent had experienced 2 different treatment options, whereas the GHD-ATM was designed to be relevant when a respondent had not experienced the 2 treatment options being investigated.

The items for the questionnaire were based on the major subthemes/issues identified in the analysis, using caregiver and child words as much as possible. The criteria for identifying whether concepts were considered major included:

• Endorsement percentages of at least 10% by both child and caregiver participants,
• The concept had to be applicable without respect to treatment type, and
• The concept had to be applicable to subjects participating in a clinical trial.

First, the GHD-PRM, which asks the respondents to indicate their preference for one of two different treatments that they have experienced and to identify the attributes which underpin their preference, was generated. The GHD-PRM assesses: 1) which treatment is preferred; 2) factors that affected their treatment preference; 3) selection of the most important factor (child version) or a ranking of three most important factors (caregiver version) for the treatment preferred; 4) which treatment they would prefer to continue after clinical trial completion; and 5) which treatment they would recommend to others. The caregiver version has two additional stems with items asking for the caregiver’s personal experience with their child’s GH medication.

The GHD-ATM was developed after the GHD-PRM. This questionnaire leveraged what was learned from the interviews in terms of what were the major attributes underpinning the choice/preference for a treatment by mirroring the attributes of the GHD-PRM, but rather than asking the respondent to make any comparisons, the respondent is asked to simply rate the degree or “presence” of each attribute in their current treatment. The attribute questionnaire contains a 5-point, Likert-type response scale. Items measuring ease/difficulty had response scales ranging from “Not at all easy” to “Extremely easy”. Items measuring like/dislike had response scales ranging from “Not at all” to “Extremely”. Items measuring “how often” had response scales ranging from “Never” to “Always”.

The GHD-PRM is intended to be used in study designs such as a cross-over or switch study when a respondent has had the opportunity to experience different treatments, whereas the GHD-ATM is intended to be used in designs such as a clinical trial or in clinical practice when the respondent has not experienced a comparator treatment.

These questionnaires are meant to be completed as self-reported questionnaires, except the caregiver versions, which include two items asking about the child’s emotional state. These questions were considered as observer-reported outcome (ObsRO) questions and included instructions to complete the items based upon what the caregiver had seen or been told, and not on their opinion. These items have an additional response option for “Don’t know” to allow caregivers to indicate when they do not have enough information based on their observations to answer the item.

Cognitive Debriefing Assessment Results

Translatability assessment identified only minor formatting and wording issues which were incorporated into the draft questionnaires used for the cognitive debriefing assessment interviews. The debriefing assessment interviews were conducted in an independent sample of 22 respondents (12 caregivers, 10 children). Four blocks of caregivers and three blocks of children were needed to refine the questionnaires, items, and instructions in terms of comprehension, formatting, readability, and relevance.

Final Measures

The child version of the GHD-PRM has 20 items, and the caregiver version has 31 items. Both versions share 20 conceptually equivalent items. Examples of shared items from the GHD-PRM child version are shown in Figure 2.

Figure 2 Examples of shared items from the GHD-PRM Child version.

Ten additional items in the caregiver version ask questions about the caregiver and why they prefer the GH medication they selected. Figure 3 presents examples of these items. One additional question asks the caregiver to rank the three most important reasons they prefer the medication they chose.

Figure 3 Example items asked of caregivers regarding themselves about their treatment preference from the GHD-PRM Caregiver version.

Both versions of the GHD-ATM have 16 items, which are conceptually equivalent, asked from either the child or caregiver perspective. Examples of items from the GHD-ATM are shown in Figure 4.

Figure 4 Example items from the GHD-ATM Child version.

Scoring

The GHD-PRM can be scored and/or interpreted in 3 different ways:

1. Simple count of the stated preference of which treatment is preferred for the sample under study. For example, x number of people prefer treatment Y over treatment Z.
2. Summary count of the number of attributes for the preferred treatment as an indication of the strength of the preference for the preferred treatment. For example, there are x number of attributes (explanations for) why treatment Y is preferred over treatment Z.
3. Rank ordering of the individual attributes of the preferred treatment to better understand the “why” of treatment preference.

The GHD-ATM is scored as one total transformed score with reverse coding as needed so that a higher score indicates a stronger positive treatment attribute presence.

Discussion

Although there is FDA guidance on the importance of PED and PPI evidence, as well as proposed guidelines and frameworks on how to use and evaluate this type of data,^12,24,25 the actual methodology to develop assessments, apart from conjoint analysis experiments, is less common. The processes used to develop these novel PGHD treatment preference and attribute questionnaires, which do not require the patient to perform a complex risk/benefit analysis regarding their preferences, are meant to help fill that gap. The novel measures fall under the category of a clinical outcome assessment but are not strictly speaking a PRO questionnaire as they assess preferences regarding treatment attributes rather than outcomes. Further, PRO questionnaires may provide a snapshot of a patient’s own assessment of various outcomes at a given point in time; however, they do not convey how much the patient values one specified outcome or therapy when compared to other potential outcomes and therapies.³ Nor would they be considered an ObsRO questionnaire when the patient is a child and the caregiver is completing the questionnaire; as it is most often the caregiver’s preference, even if based on the child’s experience, that would be most appropriate to assess. By drawing from best practices for developing PRO and ObsRO measures, treatment preference and attribute measures can be developed with scientific rigor and validity. Further, combining methodology drawn from PRO/ObsRO measure development and discrete choice concepts allows these questionnaires to be relevant to all aspects of drug development as well as clinical practice and research as they go beyond the simple “which do you prefer” and assess the “why” by understanding the attributes which underpin a preference as revealed (actual) rather than stated (hypothetical). Lastly, these measures are practical to develop and easily interpretable, which should allow the research community to actively embrace them. By basing the development of preference and attribute questionnaires on a range of best practices across available methodologies, we can ensure that these patient-centric questionnaires provide credible and meaningful evidence when evaluating treatment options. Although GHD is used here as the disease model for the development of these measures, we believe this methodology is applicable across disease states and can serve as a prototype for future preference and attribute tool development.

Preference questionnaires present their own unique set of methodological challenges for development and interpretation, especially when the treatment is for a child, yet it is the caregiver who is the decision maker for preferences. In this case, some preferences may be experienced by the caregiver while others may be based on how the child feels or reacts. By incorporating best practices for both PRO and ObsRO measures’ development, we believe the methodology exists to meet this challenge by clearly delineating which preferences are caregiver based and thus, a caregiver is able to assess regarding their own experience or using best practices for ObsRO measure development to report on child experiences. By providing clear instructions to the caregiver to only select the response that best matches what they have seen or been told by their child as well as including a “Do not know” response option, this challenge can be addressed.

Assessing preference is not a marketing message or simply a question of, “Which drug do you prefer?” but rather also, an understanding of why one drug is preferred over another and the strength of that preference. Treatment preference questionnaires, when used in a trial such as a short-term cross-over design which would limit recall bias or with an extension arm where patients on treatment A are given the chance to continue on treatment B, can provide real-world evidence of preferences if the assessments are done within a reasonable time frame of the switch. However, their utility may be limited when a patient has not had the opportunity to experience more than one treatment option on which to base their preference and can only provide hypothetical preferences. This is the case in a treatment efficacy trial where patients are randomized to either treatment A or B but do not experience both. In this situation, we propose that the scientific evidence used for the development of a preference “choice” questionnaire can be leveraged by using the data to also develop a treatment attribute questionnaire. The attribute questionnaire does not ask for a comparison between treatments but rather asks the respondent to rate the presence or strength of the attributes which underly preference and are key factors contributing to preference choices. An attribute questionnaire of this type makes it possible to provide evidence that drug A (experienced by one treatment arm) has more or less of the necessary attributes which would suggest that the respondent would prefer the treatment. By basing the preference and attribute questionnaires’ development on best practices of PRO and ObsRO measures’ development, we can ensure that these questionnaires provide credible and meaningful evidence when evaluating treatment options.

Scoring is another methodological challenge as a preference questionnaire does not necessarily contain “domains” or clusters of concepts and as such is scored as a simple count of number of preferred attributes that make up a preference for one drug versus another. However, it should be noted that, as reported by clinicians and caregivers, efficacy and safety are generally the key drivers of preference. The simple count of number of attributes is unweighted for these key preferences and thus, when using the measure, the interpreter may wish to examine attributes with or without these key attributes depending upon the question being examined. For example, if the question is, “What attributes beyond safety and efficacy are important?” then a score without these attributes may be preferred.

Standard of care therapy for PGHD with daily SQ injections poses challenges in terms of treatment burden, satisfaction, and adherence. Non-adherence with daily GH therapy is common. Kaplowitz et al²⁶ reported suboptimal adherence rates, with only 32% of commercial and 18% of Medicaid patients reporting rates exceeding 80%. In a national study of GH adherence in New Zealand children,²⁷ two-thirds of patients missed more than one dose per week. Predictably, with greater non-adherence there was a progressive decline in annualized height velocity (growth). Despite years of treatment with daily GH, the full genetic height potential may not be reached, as reported in a meta-analysis of registry data of >4,500 patients.²⁸ The current landscape of PGHD treatment includes traditional daily GH injections, recently approved once-weekly long-acting GH injections, and an investigational oral secretagogue in clinical trials. Clinical tools such as PPI can aid in shared decision making between the provider and the caregiver/child to help optimize treatment success.

Limitations

It should be noted that respondents for the CE interviews came from a variety of sources, including the general population of children with GHD as well as from those participating in a clinical trial, and there was wide variation in some respondent characteristics. Although this heterogeneity of respondents provides greater generalizability of findings and a broader range of experiences,²⁹ it should be taken into consideration when interpreting findings. Additionally, this study was based in the US. Consequently, there may be cultural factors relevant for other countries that were not adequately captured.

Development of the preference attributes based on interviews with patients who may not have experienced all available treatment options, as was the case in this study, poses an additional challenge. Unfortunately, the oral treatment sample was small due to the reality that there are no oral treatments currently available outside the clinical trial setting. Therefore, it was not possible to assess saturation of concepts by treatment type. However, the sample size, when combining those on both injectables and oral treatments was adequate for capturing broad treatment experiences,^30,31 and 95% saturation of relevant concepts was achieved. It would be valuable to further study experiences with oral GH treatments once they are more readily available. When all treatment options are not available, hypothetical scenarios for CE interviews and the CD assessments can be considered. Lastly, larger quantitative studies, incorporating these questionnaires as outcome measures, may also provide the opportunity to psychometrically examine some of their measurement characteristics and structure, such as inter-item correlations and test-retest reliability.

Currently, there are limited disease-specific preference measures for GHD treatment. The GHD-PRM and GHD-ATM could prove to be valuable assets in evaluating and comparing factors influencing patients’ treatment satisfaction across diverse settings, administration routes and mechanisms of action, including newer routes of administration such as oral. By actively participating in the dimensions of patient care captured by the GHD-PRM and GHD-ATM, the importance of understanding patient preferences in the clinical setting can be reinforced. This understanding holds the potential to improve treatment compliance, enhance the social and emotional well-being of children with PGHD and their caregivers, and positively impact clinical outcomes. Additionally, greater understanding of these factors among clinicians will facilitate health care provider–patient communication and help clinicians better tailor treatment plans to patients’ needs. Lastly, the ability of regulatory, payer and clinical audiences to better understand the patient experience should not be underestimated.

Conclusion

In summary, the GHD-PRM and the GHD-ATM can be considered rigorously developed and valid preference and attribute questionnaires specific to PGHD and other conditions treated with recombinant human GH injections. These preference and attribute measures can be incorporated into clinical trials and clinical practice and may also inform future research regarding the assessment of treatment preferences in other conditions.