Development and Content Validation of the Patient’s Qualitative Assessment of Treatment – Real-World (PQAT-RW): An Instrument to Evaluate Benefits and Disadvantages of Treatments in Real-World Settings

Introduction

Patient-reported outcome measures (PROMs) and patient-reported experience measures (PREMs) collect information about health outcomes and experiences of care directly from patients in a standardized manner.¹ Patient-reported data are gaining recognition as important evidence to help support patient-centric drug development and improve clinical care. In particular, patient perceptions of the benefits and risks/harms of a treatment can be of great value for informing decision-making around investment in drug development and prescribing in real-world clinical practice, as well as increasing our understanding of factors that might influence treatment adherence.^2–5

Many PROMs and PREMs can be used to assess patient-perceived benefits or risks/harms of treatment (eg patient experience of symptoms or of side effects of treatment). However, most measures do not evaluate benefits and risks/harms within the same instrument or assess the extent to which patients weigh benefits and risks/harms against one another; they therefore fail to provide a holistic picture of patients’ evaluation of treatment.

Furthermore, most instruments are designed to generate data that will be analyzed at a population level, allowing decision-makers to ascertain whether, on average, observed benefits outweigh observed risks/harms for a given treatment. However, the relative importance individuals assign to the benefits and risks/harms associated with a treatment varies greatly, and an average score of this relative importance may poorly represent priorities or the perceived benefits/burden of treatment for individual patients.⁶

At the time of this publication, three generic (ie not disease-specific) instruments were identified that assess both patient-perceived benefits and risk/harms of treatment within the same instrument at the individual level: the Treatment Satisfaction Questionnaire for Medication (TSQM),^7–9 the Patient Global Impression of Benefit–Risk (PGI-BR)¹⁰ and the Patient’s Qualitative Assessment of Treatment (PQAT).¹¹ As with most PROM and PREM instruments, the TSQM and PGI-BR use predefined response scales on which patients provide quantitative information about their outcomes or experiences. The PQAT¹¹ is unique in that it allows for both free-text response options and a predefined response scale. The free-text response options allow patients to provide information in their own words and can help to identify previously unknown aspects of a treatment. Qualitative data from free-text responses can aid interpretation of quantitative data by providing additional context for ratings on response scales and indicating the relevance of different outcomes or experiences to patients.^6,12–15 Free-text responses are especially informative when evaluating new treatments, or when investigating outcomes of existing treatments in heterogenous populations, for example in a real-world setting, where varied outcomes or experiences are expected, and valued differently, by individual patients.

The PQAT asks patients to describe and rate their experience of benefits and disadvantages of treatment (as well as their willingness to continue treatment). This wording allows for broader concepts to be captured that might not necessarily be considered as risks (eg the probability of experiencing unfavorable outcomes from a treatment) or harms (eg adverse outcomes experienced by the patient, such as side effects), but may nonetheless negatively affect how a patient perceives a treatment (such as inconvenience or frequency of administration).

The original PQAT (also referred to as PQATv1) was developed for use in clinical trials and was cognitively tested before being used in a Phase 2 clinical trial for a new injectable treatment for type 2 diabetes mellitus (DM).¹¹ Findings from this initial development and validation study demonstrated that patients found the PQAT simple and easy to understand, and that it provided a good understanding of the benefits and disadvantages of a treatment from the patient’s perspective (including unexpected effects), as well as how patients balance these benefits and disadvantages when deciding whether or not to continue with treatment.¹¹

Since its initial development and validation, the PQAT has been updated to include two additional numeric rating scales to assess the magnitude of patient-experienced benefits and disadvantages of treatment. The updated version, known as the PQATv2 (Figure 1 and Table 1), has undergone content validation in panels of patients with type 1 and type 2 DM (S1 Table). The development and results of PQATv2 content validation are further described in S1 Supporting information.

Table 1 Comparison of Items Included in the Different Versions of the PQAT Instrument (PQATv1 versus PQATv2 versus PQAT-RW)

Figure 1 Development and main features of different versions of the PQAT instrument. Abbreviations: PQAT, Patient’s Qualitative Assessment of Treatment; PQATv2, Patient’s Qualitative Assessment of Treatment version 2; PQAT-RW, Patient’s Qualitative Assessment of Treatment – Real-World. The PQATv2 was developed as an update to the original PQAT.11 The PQAT-RW is an adaptation of the PQATv2 for use in real-world settings.

Qualitative and quantitative data for patient-experienced benefits and disadvantages in real-world settings are valuable to supplement evidence from clinical trials for appraisals of new treatments, to inform regulatory and payer decision-making (eg treatment approvals, access and pricing negotiations) and to guide decision-making in routine clinical practice.^2,16–22 It is important to note that data from the PQAT can complement, but is not designed to replace traditional safety reports. As is the case for any PRO instrument capturing aspects of drug toxicity, data from the PQAT do not involve clinical judgement.²³

However, if intended for use in real-world settings, an instrument must ensure specificity for the treatment and condition being assessed. This is especially important for populations of patients with comorbidities or requiring multiple medications. As the PQATv2 asks participants about a specific drug taken during a clinical study and has only been tested in patients with type 1 and type 2 DM, adaptation and additional evidence to ensure its applicability in real-world settings were required.

This paper describes the adaptation of the PQATv2 for use in real-world settings (PQAT-RW), and the content validation of the PQAT-RW in a patient population with diverse chronic conditions receiving treatment in a real-world setting.

Materials and Methods

Development of the PQAT-RW

The PQAT-RW was developed by adapting the items of the PQATv2 instrument (Figure 1) for use in real-world settings. Like the PQATv2, the PQAT-RW is comprised of six items, three qualitative (including two free-text response items) and three quantitative, as described in Table 1.

To adapt the PQATv2 items for real-world use, all references to “the drug you received during this trial” in the PQATv2 were removed and replaced with placeholders for the drug name and patient’s condition to be filled in by the healthcare professional (ie “[drug], the drug that you are currently taking to treat your [condition]”). This change ensures that the instrument is relevant to different patient conditions and medication regimens without changing its intent or structure, or impacting its understandability among patients with comorbidities or multiple medications. To ensure consistent entry of drug and condition names, healthcare professional instructions were also added to the start of the instrument, such as reminders to include both the brand and generic names of medications (eg “Lantus [insulin]”), and to use the common name of conditions rather than scientific terminology (eg “breast cancer” rather than “ductal carcinoma”).

The PQAT-RW, developed for use in patients with chronic health conditions in real-world settings, is copyrighted by Sanofi and available through the Mapi Research Trust (https://eprovide.mapi-trust.org/instruments/patient-s-qualitative-assessment-of-treatment-real-world).

Content Validation of the PQAT-RW

Design

The PQAT-RW was tested as part of a non-interventional study in which qualitative, semi-structured interviews were conducted with participants from the USA with a range of chronic health conditions. The interviews were conducted in an iterative process to allow for modifications to the instrument and the interview guide between interviews, when necessary, so that the updated instrument could be used for debriefing with the remaining participants.

Participants

A recruitment agency, Global Perspectives, assisted in the recruitment of participants across two sites in the USA. A purposive sampling approach was employed, in which quotas were set to ensure the sample of patients included a range of different ages, genders, races, education levels, and types of chronic conditions. Participants were eligible for inclusion if they were at least 18 years of age at the time of recruitment, had a diagnosis of a chronic condition for which they were currently receiving treatment via a medicinal product or medical device for at least one month, and were verbally fluent and literate in English. Participants were excluded if they had any difficulty hearing, reading, or speaking, or if they had severe physical, neurological or cognitive deficits that might prohibit their ability to participate in an interview.

A sample size of 16 was deemed appropriate to meet the sampling quotas to represent a range of chronic conditions in the sample of patients, as well as to meet the number of interviews (7–10) that is suggested to explore the comprehensiveness of clinical outcome assessments, to confirm patient understandability of an item, and to identify potential problems with an item.^24–26 Furthermore, this sample size allowed modifications to be made to the instrument following initial patient feedback, which could then be debriefed in a sufficient number of remaining participants to ensure relevance and understandability. Due to the simplicity of the items included in the tool, it was expected that this number of interviews would be sufficient; indeed, the feedback received was relatively homogenous. Participants were financially compensated for their time and involvement in the study.

Interviews

Cognitive debriefing interviews were conducted over the telephone and lasted approximately 45 minutes each. All interviews were conducted by trained and experienced qualitative interviewers, and a semi-structured interview guide was used to ensure all topics of interest were discussed (S2 Supporting information).

Before each interview, the interviewer filled in the placeholders for drug name and condition for each participant. At the start of each interview, the interviewer and the patient engaged in a short conversation about the patient’s condition and related medication. Participants were then instructed to focus on one specific condition and medication while completing the PQAT-RW. Patients with comorbidities and/or who were using multiple medications were asked to focus on only one condition and one medication (as determined based on consideration of the sampling quotas) when responding to the items.

The interview aimed to assess respondents’ understanding and their ability to complete the PQAT-RW. Specifically, participants were asked to read each instruction and item out loud and to describe how they would respond to the item. Further probes were then asked about the participant’s understanding and comprehension of each instruction/item and response option. General probes were also asked to assess the overall appropriateness/acceptability of the whole instrument and if anything was missing from the patient’s perspective.

Data Analysis

Interviews were audio-recorded and transcribed verbatim. A thematic analysis method was used to code transcripts, which consisted of tagging codes (ie labels that are assigned to each concept or sub-concept) to segments of textual data within each transcript, to group information and facilitate comprehension of the themes within and between each transcript. Transcripts were coded using the qualitative software package ATLAS.ti (version 8) by two coders. The lead coder reviewed the code list, coded the first transcript, and quality checked the coding of all remaining transcripts (analyzed by the second coder) to ensure consistency. For each item, a frequency count was provided for each participant’s understanding of all instructions and items (understood, not understood, not clear if understood) and ease of response options (easy, difficult, unclear).²⁷ Frequency counts were also provided for any other aspects of interest, such as the number of participants who suggested a change to an item or thought that an additional item should be added.

Participants’ definitions and interpretations of terms within items and response options were coded, and any similarities or differences between interpretations were highlighted. Responses to the instrument items were also coded to reflect the type of concepts that participants considered when answering each item (eg effectiveness of the drug, side effects, etc). The concepts were spontaneously elicited from participants (ie not specifically probed by the interviewer). This exploratory analysis was carried out to identify any overlap between the concepts discussed as benefits or disadvantages, or any items missing from the instrument.

Results

Results of the PQAT-RW Cognitive Debriefing Interview

Study Sample

A total of 16 participants with a range of chronic conditions across the sample were recruited across two sites in the USA (Table 2). The majority of the participants were female (n = 11, 68.8%) and White/Caucasian (n = 13, 81.3%). Most were educated to university level or higher (n = 11, 68.8%). Ages ranged from 21 to 70 years and the mean age was 45 years old. The sample consisted of participants with a wide range of chronic conditions, including diabetes, cancer, multiple sclerosis or chronic obstructive pulmonary disease (Table 3). Participants were taking a variety of different medications for each condition, and the majority of participants were taking multiple medications to treat their condition. Five participants (n = 5/16, 31.3%) experienced comorbid conditions.

Table 2 Sample Demographic Characteristics of the Patient Cohort

Table 3 Clinical Characteristics of the Patient Cohort

Participants’ Overall Ability to Understand the Instructions and PQAT-RW Items

During the interviews, participants were presented with the PQAT-RW and asked to describe their overall understanding of the instructions and items. Overall, all participants (N = 16/16, 100%) reported that they had a good general understanding of all instructions and items, and results from the general feedback demonstrated that the PQAT-RW was easy to respond to. Participants’ understanding of the instructions and individual items, including sample quotes, is included in the supporting information (S3 Supporting information and S4 Supporting information).

During the content validation of the PQATv2 (see S1 Supporting information), it was noted that the word “disadvantageous” (item 4) might be considered a cognitively complex word. Therefore, in an attempt to increase clarity and use common terminology, alternative wording for items 2 and 4 was presented to participants, replacing “how disadvantageous” and “how beneficial” with “how would you rate the overall disadvantages” and “how would you rate the overall benefits”, respectively. Both options (original and alternative wordings) were provided for participants to comment on during the cognitive debriefing interviews. The majority of participants reported that they preferred the alternative wording for items 2 and 4 (“how would you rate the overall benefits/disadvantages”) over the original wording (“how beneficial/disadvantageous was the drug you received during this trial”; n = 10/16, 62.5% and n = 14/16, 87.5%, respectively). This was particularly apparent for item 4, in which the majority of participants (n = 13/16, 81.3%) felt that “disadvantageous” was difficult to understand and 12 (75.0%) participants recommended to change the word “disadvantageous” in the original version to an easier term. Nonetheless, all participants were able to demonstrate understanding of the word “disadvantageous” in the context of the instrument.

Participants’ Overall Ability to Respond to the PQAT-RW Items

All participants were able to respond to all items. Only a minor modification to the response format of item 6 was deemed necessary. Participants’ understanding of the modified format was confirmed during the final five interviews.

When asked to report whether they found each individual item easy or difficult to respond to, the majority of participants (n = 9/16, 56.3%) stated that they found all items easy to answer and respond to (Figure 2). The remaining participants (7/16, 43.8%) reported that one (n = 4/7) or two (n = 3/7) of the items were difficult to respond to. However, all participants were able to select a response. When participants were asked to explain why they found some items less easy to respond to, one participant (n = 1/16, 6.3%) stated that she/he found item 2 (rate overall benefits) difficult to answer, citing that her/his response would vary day to day. Some participants (n = 5/16, 31.3%) found item 4 (rate overall disadvantages) hard to answer. Of these, three participants (n = 3/5, 60.0%) reported that this was because they felt it was relatively harder to rate the disadvantages compared to the benefits, rather than because of an issue with the item itself, and two participants (n = 2/5, 40.0%) initially misinterpreted the scale direction. However, all five participants (n = 5/5, 100%) were able to answer the question, and the participants who misinterpreted the direction of the scale were able to correct their error without interference from the interviewer.

Figure 2 Participants ease of response for items on the PQAT-RW. Abbreviation: PQAT-RW, Patient’s Qualitative Assessment of Treatment – Real-World. *The format of the response options for item 6 was changed from vertical to horizontal for the final five interviews and none of the final five participants expressed difficulty responding to this item.

Another notable reason some participants reported that some of the items were difficult to respond to was participants’ understanding of the format of the response options for item 6. This item had initially been developed in line with the electronic version of the PQATv2 in which the numerical response options were displayed vertically. During feedback from the first 11 interviews, four participants (n = 4/11, 36.4%) expressed difficulty with this format and felt that the response options should be displayed horizontally. Based on this, the format of the response options was updated to be displayed horizontally. This new format was debriefed in the final five interviews, and none of the five participants expressed difficulty with the new horizontal format.

Subgroup Differences

No major group differences were observed among age, education level or disease condition for general understanding or ability to respond to items. For example, of the participants who found some items difficult to answer, there was no clear pattern suggesting education level might affect ease of response; one had some high school education, two had some college or university education, three were educated to university degree level, and one was educated to graduate or professional degree level.

However, a difference in preference for drug names was observed when comparing disease conditions, with all oncology participants (n = 2/2, 100%), participants with systemic lupus erythematosus (n = 1/1, 100%) and with pseudotumor cerebri (n = 1/1, 100%) stating that they preferred the generic name for the drug. Conversely, all other disease conditions in the sample showed an overall preference for the brand name.

General Feedback and Suggested Changes

In terms of comprehensiveness, the majority of participants (n = 11/16, 68.8%) felt that the items in the PQAT-RW covered all aspects of their experience with their treatment that they would like to communicate. During the first 13 interviews, a few participants (n = 4/13, 33.3%) suggested the addition of a financial item. Following this feedback, probe questions were added to the interview guide for the final three participants to further explore whether practical and financial aspects would be more readily considered with the addition of further items. These questions and further exploratory analysis revealed that all remaining participants (n = 3/3) would already consider financial aspects in their response to existing items, if they felt these aspects were an important issue in their experience (see Figure 3). All participants (N = 16/16, 100%) stated that they found it easy to focus on one treatment only when answering to the items. In terms of relevance, no participants recommended removing any of the items.

Figure 3 Global Concepts considered across all items in the PQAT-RW. Abbreviation: PQAT-RW, Patient’s Qualitative Assessment of Treatment – Real-World.

Concepts Considered by Participants When Answering the PQAT-RW

Concepts Considered by Participants

Analysis of the answers to open-ended items identified a total of eight global concepts relating to the benefits and/or disadvantages of the treatment that were considered by participants when responding to the items of the PQAT-RW: treatment effectiveness, side effects, method of administration, frequency of administration, financial considerations, storage, packaging and drug preparation (Figure 3).

The primary concept, considered by all participants (N = 16/16, 100%), was the effectiveness of the drug in improving their condition. This demonstrates the high importance of treatment effectiveness to all participants. Other concepts that were frequently considered by participants were side effects (n = 11/16, 68.8%) and the method of administration (n = 10/16, 62.5%). Other concepts that were discussed in a smaller proportion of the sample were frequency of treatment administration (n = 5/16, 31.3%), treatment storage (n = 3/16, 18.8%), financial (n = 4/16, 25%), packaging (n = 2/16, 12.5%) and drug preparation (n = 1/16, 6.3%). Further exploratory analysis of these concepts revealed that side effects, method of administration, frequency of administration and storage were considered both as an advantage and a disadvantage (eg presence vs lack of side effects), while financial considerations, storage and drug preparation were considered only as a disadvantage.

Further to assessing the concepts participants considered more frequently across the overall instrument, the analysis also assessed the different concepts considered by the participants within each specific item, as evident in free-text responses or during the debriefing process (Figure 4). When debriefing with patients which benefits of the drug they considered when answering item 1 (main benefits) and item 2 (rate beneficial/overall benefits), the most frequently considered benefit was treatment effectiveness in improving their condition (N = 16/16, 100% for both items), followed by lack of side effects (item 1: n = 2/16, 12.5%; item 2: n = 3/16, 18.8%), frequency of administration (item 1: n = 2/16, 12.5%; item 2: n = 1/16, 6.3%), method of administration (item 1: n = 1/16, 6.3%; item 2: n = 1/16, 6.3%) and storage (item 2: n = 1/16, 6.3%). When debriefing with patients which disadvantages of the drug they considered when answering item 3 (main disadvantages) and item 4 (rate disadvantageous/overall disadvantages), participants considered a wider range of concepts, with every one of the eight global concepts identified by the analysis being discussed by at least one participant (Figure 3). The most frequently considered concepts when discussing disadvantages were presence of side effects (n = 10/16, 62.5% [item 3], n = 9/16, 56.3% [item 4]) and method of administration (n = 8/16, 50.0% [item 3], n = 7/16, 43.8% [item 4]). However, lack of treatment effectiveness in improving their condition was also considered in a sizable proportion of participants (n = 6/16, 37.5% [item 3], n = 5/16, 31.3% [item 4]). When discussing if participants were willing to continue the treatment (item 5), effectiveness was again the most frequently discussed concept (n = 14/16, 87.5%), followed by side effects (n = 6/16, 37.5%). Finally, when discussing the participants’ reasons in weighing the benefits against the disadvantages of treatment (item 6), the three main concepts that participants considered were treatment effectiveness (n = 12/16, 75.0%), side effects (n = 5/16, 31.3%) and method of administration (n = 4/16, 25.0%).

Figure 4 Concepts considered for each item of the PQAT-RW. Abbreviation: PQAT-RW, Patient’s Qualitative Assessment of Treatment Real World.

Subgroup Differences

There were some small differences in the range of concepts considered by participants with different conditions. More specifically, the groups that considered relatively few concepts, compared with other patient groups, were oncology patients (3/8 concepts; treatment effectiveness, side effects and method of administration), the participant with pseudotumor cerebri (2/8 concepts; treatment effectiveness and side effects) and the participant with depression (1/8 concepts; treatment effectiveness). Packaging was considered only by two patients (n = 2/16, 12.5%), both of which had chronic obstructive pulmonary disease, and drug preparation was only considered by the patient with systemic lupus erythematosus (n = 1/16, 6.3%). The latter patient was the respondent who considered the largest number of concepts (7/8 concepts), with packaging being the only concept not considered.

Discussion

Findings from content validation of the PQAT-RW revealed that it was well understood across patients with different chronic conditions and using different medications. The study also revealed that the PQAT-RW was well suited to gather individualized data about patients’ perceived benefits and disadvantages of a specific treatment used to treat a specific condition in a real-world setting, as well as to evaluate patients’ perception of the balance between benefits and disadvantages when considering continuation (or discontinuation) of treatment.

The majority of participants reported that no items needed to be removed or added to the instrument, because they felt the existing items allowed them to consider all aspects, they would want to communicate to give a full account of their experience of treatment. For example, even participants who suggested that the addition of a financial item might benefit the instrument felt that the wording of the existing open-ended items would already elicit consideration of financial aspects in patients who are concerned about the cost of treatment. Importantly, after beginning the interview with a short conversation about their condition and treatment with the interviewer, participants felt that it was easy to respond to the instrument with one specific condition and treatment in mind. This finding is of particular value because it indicates that the PQAT-RW is effective in isolating and assessing an individual treatment and condition, even in patients who have multiple comorbidities and who are receiving multiple treatments, as is often the case in real-world settings. The lack of subgroup differences and the relative homogeneity of results obtained across participants who had different chronic conditions and who were using different treatments with varying modes of administration (including oral, injectable, inhaler, infusion, patch) support the suitability of the PQAT-RW as a generic instrument for assessing patient experience of benefits and disadvantages of treatment, without the need to tailor the structure or content of the instrument to a specific disease or drug. The only subgroup difference that was noted concerned the preference for the generic versus brand name of the drug across different conditions. These responses resulted in a decision to retain both generic and brand names in the final version of the instrument.

Participants considered a range of concepts when responding to the items of the instrument. The effectiveness of treatment was the primary concern for all participants. After treatment effectiveness, side effects and method of administration were consistently noted as important considerations across all items. These findings indicate that these three concepts represent patients’ core considerations when evaluating the benefit–disadvantage ratio of the treatment that they are taking, regardless of their condition and medication. Importantly, all core concepts were considered by patients as both benefits (eg treatment effectiveness, lack of side effects, ease of administration) and disadvantages (eg lack of treatment effectiveness, presence of side effects, difficulty of administration), depending on the question. While financial considerations (eg cost of the drug) were not seen as a main disadvantage, they were reported as a key concept for many participants; specifically, when making decisions about whether to continue treatment. However, this might reflect a bias due to the origin of the sample (ie the USA); financial considerations might not be a key concept for patients from countries with universal healthcare systems. Furthermore, the PQAT-RW was able to identify additional global concepts concerning more practical aspects of treatment that patients felt were important to communicate, such as packaging, drug preparation and storage. The fact that only minor differences in the range of concepts considered by participants with different conditions were observed is further evidence that the PQAT-RW captures details of a variety of patient experiences.

The use of generic terms, such as “benefits” and “disadvantages”, and the opportunity to expand on answers in free-text responses, allow patients to consider freely which concepts are most important in their own personal experience, as well as when to consider them as benefits or disadvantages (or both). Responses to qualitative items can contextualize and provide valuable additional insight into the patient’s quantitative responses. Therefore, an instrument using generic terms and free-text responses alongside quantitative scales might provide a richer and more realistic picture of patient experience compared with instruments that only ask patients to rate predetermined concepts on quantitative scales.

Indeed, the mixed-method PQAT-RW identified a larger number of global concepts than the TSQM^7–9 and the recently developed PGI-BR instrument,¹⁰ both of which use predetermined quantitative response scales only. The four concepts evaluated by the TSQM (effectiveness, side effects, convenience and global satisfaction) and identified by the PGI-BR (effectiveness of treatment, side effects, convenience of use and overall acceptance/satisfaction with treatment) are similar to the eight concepts identified from participant responses to the PQAT-RW in this study (ie effectiveness, side effects, method of administration, frequency of administration, financial considerations, storage, packaging and drug preparation). This overlap demonstrates that the heterogeneity of responses to generic and open-ended items does not come at the expense of capturing important concepts covered by other, quantitative-focused instruments. Additionally, the free-text responses of the PQAT-RW allowed identification of further concepts by teasing apart considerations that might be grouped under the same concept in the PGI-BR. For example, a patient responding to a quantitative item on “convenience of use” in the PGI-BR could consider several concepts, such as method and/or frequency of administration, packaging or drug preparation, which were identified by the PQAT-RW. These four concepts represent very different issues that would be addressed or prioritized differently in clinical practice, based on the specific needs of individual patients.

Interestingly, the TSQM item evaluating global satisfaction (“How satisfied are you that the good things about this medication outweigh the bad things?”) is closely related to the PQAT-RW’s assessment of patient-perceived balance between benefits and disadvantages, although the TSQM item is focused on treatment satisfaction and is close-ended, whereas the PQAT-RW allows participants to consider concepts relevant to their own treatment experience.

The PQAT-RW is a generic instrument that has potential applications in a variety of real-world settings, with a relatively low response burden to participants and healthcare professionals owing to its concise nature. For example, it could be used to facilitate incorporation of patient perspectives into regulatory and payer decisions regarding a treatment’s benefit–risk profile as well as to identify unsuspected benefits or disadvantages of novel therapies. In clinical practice, the instrument could be used to improve communication between patients and clinicians about benefits and disadvantages of treatments and facilitate shared decision-making. Most importantly, the PQAT-RW’s ability to extract a detailed picture of the relative importance patients give to different benefits and disadvantages can provide actionable insights about factors that influence treatment adherence.

The present study helps to address some of the limitations of the PQAT instrument that were highlighted by Eek and colleagues.¹⁰ Specifically, unlike the PQAT, the PQAT-RW was tested in a heterogeneous patient population and updated, as required, based on patient feedback. However, some limitations of our study should be noted. The sampling quotas were intended to ensure heterogeneity across diverse therapy areas but not across specific conditions. Therefore, the study sample comprised multiple participants with more common chronic conditions (eg DM or chronic obstructive pulmonary disease), and the eight different conditions covered do not fully represent the variety of conditions observed in the real world. Additionally, information on the setting in which participants received their care was not collected. Future research could seek to include a more balanced and broader range of participants with different conditions and receiving different types of treatment (for example, although treatments with various modes of administration were assessed, the sample did not include participants receiving topical treatments). Furthermore, the greatest proportion of participants were educated at high school level or higher. This resulted in a sample with a relatively higher educational level, and therefore probably a more advanced vocabulary and general understanding of the items, than might be reflected in a typical real-world population. Nonetheless, all participants in this study, including participants who had not completed high school were able to respond to both the original (“how beneficial/disadvantageous was the drug you received during this trial”) and alternative (“how would you rate the overall benefits/disadvantages”) versions of items 2 and 4. Therefore, it can be inferred that both versions should be well understood when applied more widely in a real-world population. However, because patients reported a preference for the alternative wording, the alternative version of both items was retained in the final version of the PQAT-RW. While this alteration means that the PQAT-RW differs slightly from the PQATv2 used in clinical trials (see Table 1), there are also slight differences in the linguistic adaptation of “disadvantageous” in some English and non-English versions of the PQATv2²⁸ developed during linguistic validation (see S1 supporting information). The alteration is therefore unlikely to cause significant differences in interpretation by real-world respondents.

Another note on the sample of this study concerns the fact that it included exclusively participants from the USA, which might limit the conclusions that can be drawn about the generalizability of the PQAT-RW to non-English-speaking countries. However the PQATv2 has been linguistically validated, including cognitive debriefings with patients with diverse chronic diseases, in over 20 languages.²⁸ This demonstrates that the PQAT-RW can potentially be applied in varied cultural and linguistic contexts. Furthermore, while the PQAT-RW has now been developed in electronic format, it was administered in paper format in this study. The findings from the current study and from the testing and validation of the PQAT instrument¹¹ show that these instruments can be administered successfully both in paper and electronic format.

The purpose of this qualitative study was to demonstrate the content validity of the PQAT-RW in a range of chronic medical conditions. Since psychometric evaluation was not intended as part of this study, next steps in the validation of this new instrument could include a large-scale study to generate quantitative data that can be used to establish the most appropriate way to evaluate the outcomes. Further, testing the PQAT-RW in the context of longitudinal studies, to compare responses at different time points, could provide useful information about how participants’ perspective on the benefits and disadvantages of treatment might change over time, especially when starting a new treatment. The PQAT-RW is included in an ongoing prospective interventional study exploring the patient experience during treatment with isatuximab for patients with relapsed/refractory multiple myeloma.²⁹

Conclusion

Findings from this study demonstrate the value and uniqueness of the PQAT-RW as a generic instrument to assess individualized patient perceptions of the benefits and disadvantages of treatments, the relative importance individuals attribute to each, the patient-perceived balance of benefits and disadvantages and its impact on patients’ willingness to continue their treatment, across a varied range of conditions and medications.