Evaluation of the Content Validity of the COVID-19 Symptoms Daily Diary

Introduction

In the winter of 2019–2020, when severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) infections led to the worldwide coronavirus disease 2019 (COVID-19) pandemic, clinical and preclinical investigators, government agencies, and public health groups reacted with a concerted effort to address the global need for risk mitigations and treatments. Scientific publications and clinical trials on COVID-19 were accelerated to meet these demands. From the time of the first wave of infections, the need for patient-reported outcome (PRO) measures for COVID-19 patients became apparent.¹ At the height of the COVID-19 pandemic, on the basis of hallmark symptoms identified by the Centers for Disease Control and Prevention (CDC)² and in alignment with guidance on assessing COVID-19–related symptoms in adults and adolescents from the United States (US) Food and Drug Administration (FDA),³ a 15-item COVID-19 Symptoms Daily Diary (CSDD) was developed to support clinical investigations. Though the landscape shifted as the world entered into a “post-pandemic” world, COVID-19 infections remain prevalent and patients at high risk for severe outcomes require adequate treatment.⁴ Symptom profiles may continue to change with new variants, and a method for measuring the impact of different symptom profiles on patients remains necessary to understand how therapeutics may impact and alleviate them. Additionally, as mortality decreases but infection rates remain high, symptom evaluations may play a larger role in the evaluations of new therapeutics for COVID-19 than hospitalization or mortality rates.

Newly developed PRO measures must undergo rigorous validation of both their content and construct. Qualitative interviews with impacted individuals (ie, patients) are used to assess the content validity of PRO measures.⁵ In qualitative interviews, participants are asked open-ended questions, probing their experience with the disease in question, in order to elicit feedback on relevant concepts for inclusion in the measure (ie, concept elicitation).^5,6 To be in full alignment with FDA PRO guidance⁷ and patient-focused drug development guidance documents,⁸ this study, composed of both a targeted literature review and 30 qualitative patient interviews, aimed to evaluate the content validity of the CSDD and to determine if it is fit for the purpose of supporting efficacy endpoints in planned clinical trials of treatments for COVID-19. In addition, the qualitative interviews provided the opportunity to obtain evidence to facilitate the interpretation of CSDD scores and meaningful change thresholds via the debriefing of patient global impression items (ie, impression of severity and return to usual activities and health). To assist in the development of clinical trial endpoints, the appropriateness of a newly developed Pre–COVID-19 Symptoms Questionnaire was also evaluated.

Methods

Study Design

This single-visit, cross-sectional, observational study involved the completion of 3 iterative rounds (10 participants per round; N = 30) of in-depth interviews. A semistructured guide was developed for these interviews, informed by a targeted literature review aimed at identifying the most salient and most frequently reported signs and symptoms of COVID-19 (Supplemental Table 1). The literature search included qualitative studies, survey studies, or literature reviews published from 1 January 2020 to 5 June 2023, to reflect the existing body of research, including literature added since the development of the CSDD. For qualitative or survey studies that included a literature review component, data were only extracted from the qualitative research component.

Interviews were composed of both concept-elicitation and cognitive-debriefing components and were conducted via Zoom with English-speaking adults residing in the US who self-reported a recent (ie, within 4 weeks before screening) COVID-19 diagnosis. Diagnoses were confirmed via documentation (eg, physician verification, provision of MyChart screenshot/clinic documentation of a positive polymerase chain reaction test, confirmation/photo of a positive rapid test result in the defined window) prior to participant inclusion. Study procedures were reviewed by RTI International’s institutional review board before participant recruitment began, and the study was deemed exempt from oversight.

The CSDD was developed to assess the severity or frequency of 15 core COVID-19 symptoms from the patient perspective. The CSDD was designed to be answered every 24 hours, with respondents identifying the symptoms they have experienced from a checklist and then rating the severity or frequency of only those symptoms experienced in the past 24 hours.

Participants in COVID-19 clinical trials may have comorbid medical conditions with symptoms that overlap with those relevant to COVID-19 (eg, asthma, allergies). To accurately assess the impact of treatment on COVID-19–related symptoms over the course of a clinical trial, there is a need to evaluate the baseline symptoms experienced by participants that may overlap with COVID-19 symptoms but are attributable to other medical conditions. The instructions and recall period of the CSDD were therefore modified to develop a Pre–COVID-19 Symptom Questionnaire. This questionnaire asks participants to identify those CSDD symptoms they experienced in the 7 days before COVID-19 symptom onset and then to rate the severity (or frequency) of their endorsed symptom at its worst during the 7-day period before their COVID-19 symptoms started. Participants reviewed and provided feedback on the CSDD and, time permitting, other items (ie, Pre–COVID-19 Symptoms Questionnaire, Patient Global Impression of Severity [PGIS], and Patient Global Impression of Return to Usual Health [PGI-RUH] and Usual Activities [PGI-RUA] items).

Participants and Recruitment

A convenience sample of participants for this study was recruited by L&E Research (Raleigh, NC) from their proprietary database of individuals who had expressed interest in qualitative research participation. Interested individuals who met the inclusion criteria (Table 1) were included in the interview process. Recruitment targets for this study allowed up to 15 participants with previous nirmatrelvir/ritonavir treatment. Emphasis was placed on recruiting participants at increased risk for progression to severe COVID-19, as defined in Table 1.

Table 1 Inclusion and Exclusion Criteria

Interview Methods

Interviews were conducted by 2 experienced qualitative interviewers; one led the interview while the second took detailed notes and ensured that no content within the semistructured interview guide was missed. The interviews were targeted to be completed within 2 weeks of screening and typically lasted between 75 and 90 minutes. Immediately prior to the initiation of each interview, participants were provided a brief overview of the study, including reminding participants of the purpose of the study as well as the voluntary nature of participation and that the information they provided would be confidential. Verbal informed consent to participate was obtained and documented. Interviews were recorded. The concept-elicitation phase of the interviews began with open-ended questions to explore participant experiences with COVID-19 symptoms. These open-ended queries were followed by more targeted probes, to ensure that a comprehensive list of COVID-19 symptoms experienced by each participant was obtained. To better understand the relative importance of symptoms, participants were also asked to describe their most bothersome symptoms and the symptoms most important to treat.

Following discussion of their experiences with COVID-19 symptoms, participants began the cognitive-debriefing portion of the interview, starting with the debriefing of the CSDD. During cognitive debriefing, participants were asked to think aloud, describing their thought processes as they responded to each item. Interviewers then posed follow-up questions designed to gain further insight into the ease with which participants understood, interpreted, and selected a response to each item. Participants were also probed on the degree of symptom improvement that would constitute a meaningful change in the context of each response scale. Upon completion of the debriefing, participants were asked about the comprehensiveness of the CSDD measure and whether any important COVID-19 symptoms were missing from the questionnaire. Time permitting, participants were also asked to provide feedback on the newly developed Pre–COVID-19 Symptoms Questionnaire as well as on items assessing patient global impression including PGIS, PGI-RUH, and PGI-RUA.

Analysis

Both qualitative and quantitative analysis methods were used to review interview data. Descriptive analyses were conducted to describe participant demographics and included computation of frequencies and percentages for categorical variables (eg, race and ethnicity, education level) and computation of means and ranges for continuous variables (ie, age). Between each round of interviews, qualitative field notes were reviewed to inform item revisions. Specifically, participant feedback pertaining to each item (eg, question wording, response choices) was carefully considered to determine whether modifications were needed to facilitate comprehension and/or response. Interviewers discussed participant feedback carefully to align on the need for revisions. The instructions, questions, and response scales were then modified as appropriate based on the results of each round of interviews, for further testing in the subsequent round of interviews.

Upon completion of all 3 rounds of interviews, verbatim transcripts and field notes were analyzed using qualitative content and thematic analysis methods.^9–12 With these analyses, concepts/symptoms and dominant trends were identified in each interview and compared across interviews¹³ to enable the assessment of patterns in participants’ responses and to document the frequency of reported symptoms and concepts. Similarly, concept saturation (ie, the point in the research process when additional information is no longer necessary to generate themes and qualitative data collection can be considered complete) was assessed across 3 rounds of iterative interviews.^14–16 Additionally, potential problems with content or comprehension of the PRO items were identified based on patient input in each interview and then compared with the results of other interviews to document the frequency with which participants reported such issues. The transcript coding and analysis processes were facilitated using qualitative analysis software (ATLAS-ti 9.0) and aided by Excel (Microsoft Corporation; Redmond, WA), as appropriate.

Results

Literature Review

A total of 81 articles were initially identified. After removal of duplicates and abstract screening, 11 articles underwent full-text review; 1 article was removed at this stage because it did not focus on patient-reported COVID-19 symptoms. Two additional articles including PRO measures were added from desktop research that explored additional literature; thus, 12 total articles—10 from the original literature search and 2 from desktop research—were included in the review, extraction, and summarization (7 qualitative studies, 4 surveys, and 1 literature review).^17–28

Of the more than 100 symptoms of COVID-19 identified in the targeted literature review, the most frequently identified (ie, identified in each of the 12 publications reviewed) patient-reported or patient-endorsed symptoms of COVID-19 included cough, sore throat, shortness of breath or difficulty breathing, muscle or body aches or joint pain, feeling hot or feverish, and loss of smell/taste. Each of the 15 symptoms (eg, cough, sore throat) included in the CSDD were well-endorsed across the reviewed articles (ie, all symptoms were reported or endorsed by at least 9 of the 12 studies, and many symptoms were endorsed by all 12 studies). Additionally, symptoms were frequently reported by more than half of study participants and were characterized as most important or bothersome to patients.

Participant Characteristics

A total of 30 adults recently diagnosed with COVID-19 (ie, within 4 weeks of screening) participated in the qualitative interviews, which were completed between 7 August and 26 September 2023. As shown in Table 2 (demographic and clinical characteristics collected at screening), the median age of participants was 38 years (interquartile range, 14.5), and 22 of the 30 participants (73%) were female. Participants most frequently described their worst COVID-19 symptoms as “moderate” (n = 17; 57%). Participants also reported symptoms spanning multiple systems, with almost all (n = 29; 97%) participants reporting respiratory symptoms (eg, cough, sore throat, nasal congestion, runny nose, shortness of breath) and a large majority (n = 23, 77%) reporting systemic symptoms (ie, muscle or body aches, fatigue, feeling hot or feverish, chills, headache). Twenty-three participants self-reported conditions consistent with the high-risk categorizations; however, only 1 of these participants (a 71-year-old) provided documentation of diagnosis (stage IV chronic kidney disease) and could be confirmed to be at high risk.

Table 2 Demographic and Clinical Characteristics Reported at Screening

Concept Elicitation

Participants were asked at the beginning of each interview to describe the symptoms they experienced during their COVID-19 infection, specifically their most recent COVID-19 infection if they had been diagnosed with COVID-19 more than once. Experience with the symptoms assessed in the CSDD as well as other symptoms of interest (ie, chest pain, heart palpitations, and dizziness) were probed if not spontaneously reported. Once a comprehensive list of symptoms was obtained, participants were asked to identify the COVID-19–related symptoms that were most bothersome to them and those symptoms they would most want to see improved with treatment.

The CSDD was inclusive of those symptoms most commonly reported (either spontaneously or when probed) by participants (Table 3). The most frequently reported symptoms (endorsed by n ≥ 24, or at least 80% of participants) included fatigue/tiredness (n = 30; 100%), cough (n = 26; 87%), headache (n = 26; 87%), nasal congestion (n = 25; 83%), and muscle/body aches (n = 24; 80%). Additionally, at least half of the study participants reported chills (n = 22; 73%), feeling hot/feverish (n = 22; 73%), shortness of breath (n = 22; 73%), sore throat (n = 21; 70%), runny nose (n = 21; 70%), dizziness (n = 19; 63%), and change of smell (n = 15; 50%). Dizziness was the only non-CSDD symptom reported by at least 50% of the sample (n = 19; 63%); however, most participants (n = 16; 53%) only reported dizziness when probed, and several of these participants noted that this experience happened rarely (perhaps only once) and was an impact of a change in physical position (eg, moving from sitting to standing).

Table 3 COVID-19 Symptoms Reported by Participants (n ≥ 5) During Concept Elicitation

Most Bothersome Symptoms

As shown in Figure 1, the symptoms most frequently reported as among the most bothersome were headache (n = 10; 33%), fatigue (n = 10; 33%), and cough (n = 9; 30%). The top 10 most bothersome symptoms were all included in the CSDD. The most bothersome symptoms of COVID-19 were those participants described as the most severe or most disruptive to their lives. Participants consistently described the negative impact these symptoms had on their ability to complete daily activities and/or an uncomfortable/distressing experience associated with these symptoms. A few participants also noted that some of these symptoms (eg, headache, coughing, fever) were the most persistent.

Figure 1 Most bothersome and important-to-treat symptoms. Abbreviations: COVID-19, coronavirus disease 2019; CSDD, COVID-19 Symptoms Daily Diary. Notes: Participants often reported more than 1 most bothersome symptom. Bolded items are included in the CSDD. Most bothersome symptoms of COVID-19 reported once included earache, chest pain, chest tightness, cold sweats, digestive issues/cramping, dizziness, jaw pain, runny nose, sore throat, sweating, throat ulcers, vomiting, and weakness.

Participants also described which symptoms were most important to treat (Figure 1); the most common were muscle/body aches (n = 10; 33%), cough (n = 8; 27%), and fatigue (n = 6; 20%). Participants generally indicated that, as with the most bothersome symptoms, these important-to-treat symptoms had a significant negative impact on their physical and emotional functioning.

Concept Saturation

To evaluate concept saturation, the amount of new information being reported in each round of the interviews was assessed. Across all 3 rounds of interviews, 29 symptoms in total were reported by 2 or more participants. Twenty-seven of the 29 symptoms were reported during the first round of interviews, 1 new symptom (high blood pressure [self-reported]) was reported during the second round, and 1 new symptom (sweating, potentially distinct from fever or chills) was reported during the third round (Table 4). Given that high blood pressure is not a symptomatic disease, and that it was self-reported by only 2 participants, this symptom was not included in the CSDD.

In light of the fact that the patient experience across all 3 rounds was largely similar—in that the majority of symptom concepts emerged in the first set of interviews—these findings provide strong evidence that concept saturation was achieved across the total sample and the symptom concepts were fully explored and understood through this research.

Table 4 Evidence of Concept Saturation (Symptoms Reported by 2 or More Participants)

Cognitive Debriefing

COVID-19 Symptoms Daily Diary

Upon reviewing the CSDD, participants described the instructions and the symptom checklist as clear and easy to understand (Figure 2). When asked to describe in their own words what the instructions were asking them to do, participants confirmed understanding that they were to select all the symptoms that they had experienced in the past 24 hours or select “I did not experience any of these …” if they did not experience any of the listed symptoms. Most participants noted that they did not have any difficulty thinking back over the past 24 hours when asked to recall their symptoms. All 30 interview participants stated that the CSDD instructions were clear and easy to understand and did not provide any recommendations for improvement. All participants were able to select a response to each of the CSDD items using the appropriate response scale (ie, either the severity or frequency response scale). Specifically, when probed regarding the response options, the majority of participants in rounds 2 and 3 indicated that the response options were clear and easy to use, were distinct, and covered the full range of symptom severity/frequency as relevant.

Figure 2 CSDD cognitive debriefing. Abbreviation: CSDD, COVID-19 Symptoms Daily Diary.

Each item of the CSDD was cognitively debriefed with participants who reported experiencing the symptom matching that item. For each item participants reviewed, they were asked to describe what amount of change (eg, severe to mild, moderate to mild) would constitute a meaningful improvement. Due to time constraints, participants commonly were only able to explore 1 meaningful change scenario (by item); as such, the denominator (or number of participants asked a specific question) varies when reporting meaningful change results. In light of this, for each scenario (eg, change from severe to mild) the number of participants probed is provided. Overall, all change increments for scale-based items were found to be meaningful by a majority of the participants who evaluated them (Figure 3). For frequency items, a larger magnitude of change was needed for the majority of participants to find the change meaningful. However, a change from a frequency of 1 or 2 bouts of vomiting or diarrhea to none was meaningful to all participants probed. Based on participant feedback that these items were easy to understand and respond to, the items for cough, sore throat, nasal congestion, runny nose, shortness of breath, muscle or body aches, fatigue/tiredness, feeling hot/feverish, chills, headache, nausea, change in sense of smell, and change in sense of taste were retained without change for the final version of the CSDD.

Figure 3 Percentage of participants reporting a change in symptom severity or frequency would be meaningful on the CSDD. (A) Symptom severity. (B) Symptom frequency. Abbreviation: CSDD, COVID-19 Symptoms Daily Diary.

Two items, vomiting and diarrhea, underwent revisions over the course of the 3 rounds of interviews due to participant feedback. In round 1, participants described differing ways of determining what constituted or “counted” as distinct diarrhea or vomiting events. To facilitate consistent interpretation and reporting, definitions of how to count a diarrhea and vomiting event were added to each respective item after round 1. These definitions tested well in subsequent interview rounds, with only minor changes to the diarrhea definition resulting from participant feedback in round 2. The versions tested in the third interview round were ultimately recommended for inclusion in the final CSDD following participant feedback. The vomiting item was revised to include

Each time you emptied or partially emptied your stomach (threw up) counts as 1 vomit. If you vomited multiple times within a short time, even within a few seconds, each individual vomit counts as 1.

The diarrhea item was revised to include

Each toilet visit during which you passed any loose/watery stool counts as 1 diarrhea event. Even if little time has passed since you last left the toilet, if you returned and passed any loose/watery stool, count this as a separate event. If you passed any amount of loose/watery stool before making it to the toilet (had an accident), count this as a separate event.

All participants initially noted that no items should be removed from the CSDD, and most participants (n = 25; 83%) reported that the questionnaire was comprehensive and that no important COVID-19 symptoms were missing. When further probed, 5 participants (17%) reported concepts as potentially missing: 4 participants each reported 1 (unique) symptom, and 1 participant reported 3 symptoms. The potential missing symptoms included brain fog (n = 2), cold sweats (n = 1), dizziness (n = 1), ear pain/pressure (n = 1), decreased appetite (n = 1), and stomach ache (n = 1). Because these symptoms were only reported by 1 or 2 participants each, they were not included in the CSDD.

Pre–COVID-19 Symptoms Questionnaire

To explore participants’ ability to recall the 7 days before their COVID-19 symptoms started and to confirm the appropriateness of the question structure included in the Pre–COVID-19 Symptoms Questionnaire, 2 items were debriefed across the 3 rounds of interviews, when time permitted. The 2 items were a severity-based question and a frequency-based item, depending on what symptoms the participant reported. The first item assessed symptom severity at its worst during the 7 days before having COVID-19, and the second assessed symptom frequency at its worst during the same reference period. Both the severity and frequency items underwent revisions after the first round of interviews. While most participants understood these items, they reported differing ways of interpreting the reference period (“think about the 7 days before you had COVID-19”). Specifically, some participants reported thinking about the period before their COVID-19 symptoms started, whereas others thought about the 7 days before their COVID-19 diagnosis. These periods may overlap but are not necessarily the same (ie, participants commonly reported experiencing symptoms, for varying durations, prior to diagnosis). On the basis of this feedback, both items were revised to specify the recall period to “the 7 days before COVID-19 symptoms started.” These revisions improved the consistency of participants’ interpretation of the reference period and were retained without change through the next 2 rounds.

PGIS

Consistent with recommendations in the FDA COVID-19 guidance document (2020),³ 2 PGI items (Table 5) assessing return to usual health and return to usual activities—via a dichotomous response scale (yes/no)—were tested across all 3 rounds of interviews, as time permitted. When probed, participants understood the concept of “usual activities” to refer to the things they wanted to or needed to do in their everyday lives, including but not limited to activities of daily living, working, and physical and social activities. The majority of participants who were debriefed on these items (n = 12 of 14 for the PGI-RUH and n = 13 of 13 for the PGI-RUA) were able to understand the questions and select a response with relative ease and had no difficulty recalling the past 7 days. As such, no changes were recommended to the PGI-RUH and PGI-RUA.

Table 5 Patient Global Impression Items of Return to Usual Health and Activities Items

A COVID-19 PGIS item (Table 5) was also tested across all 3 rounds of interviews. All 10 participants debriefed on this item reported that it was clear, easy to understand, and easy to answer. When probed, all participants understood the term “overall” to refer to all their COVID-19 symptoms in general. When probed on the degree of change that would constitute a meaningful improvement in their COVID-19 symptoms, all participants, regardless of starting point, indicated that a 1-category change would correspond to a meaningful improvement.

Discussion

The CSDD is a rigorously developed PRO measure, designed to evaluate changes in core COVID-19 symptoms in adult patients with COVID-19. A robust evaluation of the content validity and appropriateness of the CSDD, consistent with regulatory standards,^3,7,8 was completed as a part of this study. The results of both the literature review and the 30 hybrid concept-elicitation and cognitive-debriefing interviews support the content validity of the CSDD in adult patients with COVID-19, as well as its fitness for the purpose of assessing change in COVID-19 symptoms among this patient population.

Across the 12 studies identified from the literature review, all symptoms assessed in the CSDD were the most frequently identified symptoms (ie, reported or endorsed by at least 9 of 12 studies) associated with COVID-19.^17–28 The following CSDD symptoms were identified as among the most bothersome COVID-19 symptoms across the 2 published studies^20,26 that reported these data: cough (n = 2), loss of taste/smell (n = 2), fatigue/tiredness (n = 2), feeling hot or feverish (n = 1), and headache (n = 1). The findings of these studies align with the 5 symptoms most frequently reported as “most bothersome” by this study’s participants during interview rounds: headache (n = 11), cough (n = 9), fatigue (n = 9), fever (n = 8), and muscle/body aches (n = 8), which are all symptoms included in the CSDD. While other symptoms were reported or endorsed by the reviewed literature, these symptoms were characterized in some instances as less frequently occurring in patients with COVID-19 and/or were not endorsed as being the most bothersome or most important to patients. That said, of the symptoms not assessed by the CSDD, “chest pain or tightness” was among the most frequently identified across reviewed articles (n = 10), with 2 studies reporting 50% of participants experiencing this symptom.^19,20 Our findings show that 36% of participants reported experiencing “chest pain” (Table 3); however, this symptom was not reported to be “most bothersome” or “most important to treat” by the participants (Figure 1).

Additionally, the symptoms included in the CSDD were the COVID-19 symptoms most frequently endorsed and most frequently reported as the most bothersome and most important to be treated by the 30 interview participants. During cognitive debriefing, participants consistently understood and interpreted the final CSDD items and reported ease in response selection. Participants likewise reported that the response scale was easy to use, the response options were distinct, and the items reflected the spectrum of their experiences with COVID-19 symptoms. The majority of patients found that each change in assessed severity represented meaningful change in their symptoms. Participants reported similar feedback for the Pre–COVID-19 Symptoms Questionnaire, PGI-RUH and PGI-RUA, and PGIS item. The refinements made between interview rounds were reported to be beneficial to item interpretation and response selection.

When probed, almost all participants (n = 25) reported that the CSDD was a comprehensive assessment of their COVID-19 symptoms and that no symptoms of importance were missing. Five participants indicated that some symptom(s) were potentially missing from the measure; however, several of these symptoms (such as loss of appetite, cold sweats, and dizziness) were described by participants as being highly related to or consequences of other COVID-19 symptoms (change in taste or smell; chills). Brain fog was reported by several participants; however, only 2 participants indicated they thought brain fog was potentially missing as a CSDD symptom. When participants described brain fog, they described problems with focus and memory that were caused by or co-occurred with other symptoms, such as body aches, fatigue, dizziness, and fever. However, participant descriptions of brain fog were vague, and some participants were unsure of whether it was directly related to COVID-19. Therefore, these symptoms were found not to be independent and not to merit inclusion. Finally, data from the concept-elicitation component of the interviews were highly consistent, and concept saturation was reached. These data strongly support the comprehensive nature of the CSDD.

The CSDD fills an assessment gap in the COVID-19 treatment landscape. As the pandemic (including the development of vaccines and treatments) has evolved, the need for rigorously developed PRO assessments for informing COVID-19 treatment efficacy has become critical. A preliminary version of this diary was developed during the height of the pandemic in accordance with guidance documents from the CDC² and the FDA.³ This preliminary version of the paper version of CSDD was used in the MOVe-OUT trial (NCT04575597)²⁹ for molnupiravir, an oral antiviral currently registered or authorized for treatment of mild-to-moderate COVID-19 in adults who are at high risk for progression to severe COVID-19 (the specific wording of the indication varies by country). CSDD completion compliance was high in the Phase 3 MOVe-OUT trial: the completion rate throughout the 29-day study period was over 90%. In the MOVe-OUT trial, patients reported the presence/absence and severity of symptoms over the course of their COVID-19 infection. In a post hoc analysis of MOVe-OUT, several symptoms identified in this current study as the most bothersome and important to treat (ie, cough, fatigue) were less frequently experienced or more quickly resolved for patients treated with molnupiravir.²⁹ With the improvements to and content validation of the current version of CSDD provided by this study, the current version will be useful in supporting key endpoints for future trials studying treatments for COVID-19.

As the incidence of hospitalizations attributable to COVID-19 declines due to the transition from a pandemic to endemic state, the ability to track meaningful changes in symptom severity across a wide range of common symptoms becomes increasingly crucial for evaluating future treatments.³⁰ Although global lockdowns have lifted, acute COVID-19 burden remains high for patients at high risk for severe outcomes and healthcare systems.³¹ The long-term impacts of COVID-19 infections remain in question, and treatments are still being evaluated to mitigate the impacts and burdens of symptoms.^32,33 Accurate tracking of patient symptoms can play a critical role in both the development and evaluation of COVID-19 treatments, as evident by the success of coupling PROs with clinical responses for other health conditions to increase diagnostic accuracy, reduce symptom burden, improve health-related quality of life, and reduce acute care needs.^34–36 The CSDD may be helpful in the evaluation of protracted disease (“long COVID”) and post-acute sequelae of COVID-19 (PASC).

The FDA guidance for measuring COVID-19 symptoms in patients has been updated to address new findings as our knowledge of COVID-19 has grown. The CSDD remains in line with FDA guidelines, but there are some differences between the examples used in the guidance document and the final form of the CSDD.³ For instance, “stuffy or runny nose” is a single item in the example PRO measure used in the FDA guidance, but the CSDD separates these out into “nasal congestion” and “runny nose.” In the qualitative interviews conducted in our study, participants confirmed that all symptoms were distinct, indicating an ability to distinguish between nasal congestion and runny nose and finding it meaningful to treat them as separate entities. Furthermore, the FDA guidance suggests using global impression items like the PGI-RUA or PGI-RUH with a 24-hour recall period, but also states that differing recall periods, including a 7-day recall period, may be appropriate based on the intent of the assessment.³

In recent years, several PRO measures have been developed to evaluate symptoms and impacts experienced by patients with COVID-19. Some studies have relied on previously developed and validated general PRO measures like the RAND-36, which has been applied to respiratory infections, including Middle East Respiratory Syndrome and SARS-CoV-1.^37,38 The RAND-36 is a widely used tool worldwide to assess PROs, but it does not include specific COVID-19 symptoms or burdens. COVID-19 specific measures, like the ACTIV-2 COVID-19 Symptom Diary (ACSD),²⁶ the Symptoms of Infection with Coronavirus-19 (SIC),³⁹ the FLU-PRO+,⁴⁰ and the COVID‐19 Yorkshire Rehabilitation Scale (C19-YRS), also have been developed.⁴¹ Development and validation of the SIC found similar symptom impacts and burdens found in the CSDD content validation.³⁹ The C19-YRS and the FLU-PRO+ have also been applied to monitor recovery from “long COVID.”^40,42 The C19-YRS is a telephone screening questionnaire with 19 items in a mix of yes/no questions and 0–10 scale rankings. In contrast to the CSDD, the C19-YRS is intended as a way for primary care providers to monitor recovery in patients with COVID-19 in a community setting, although it has since undergone validation for clinical trial settings.⁴³ The focus of the C19-YRS differs from the CSDD; the CSDD was developed to assess the severity of a wide range of clinically relevant symptoms of COVID-19, whereas the C19-YRS was developed to assess larger impacts of the disease, such as development of post-traumatic stress disorder and detriments to daily and vocational activities.⁴¹ The content validation of the CSDD also took place in the late pandemic/post-pandemic era, which may result in more up-to-date interpretation of symptom burden. Each has its role in assessing patients in the endemic stage of COVID-19. This rigorously developed and implemented qualitative study follows regulatory and industry guidelines for developing and evaluating the content validity of clinical outcome assessments.^3,7,8,44,45 In addition to the robust methodology, strengths of this study include the qualitative design (as this allowed for capturing direct participant feedback), the richness of data shared by participants, and the reduction of bias (as researchers shared varied perspectives when reviewing the data to reach consensus). However, as with any study, there are some potential limitations. These include the potential for recall bias, as study participants were describing experiences that happened a few weeks before the interview, and selection bias, as participants were interested in participating in research/in contact with a recruitment firm. Patients were specifically asked about their most recent COVID-19 infection. Some patients may have had COVID-19 more than once, potentially creating bias to report no or milder symptoms compared with their first infection. Furthermore, participants were recruited in the US only and were limited to an English-speaking population; therefore, the findings may not be generalizable to other cultures and countries. However, a cultural adaptation and rigorous translation process (per international guidelines) conducted prior to clinical trial implementation in other countries can mitigate any potential bias.^46,47 Based upon the nature of the disease and findings from the scientific literature, it is unlikely that the symptoms experienced by patients in the US differ from those experienced by individuals in other countries. In addition, every effort was made to ensure that the qualitative study population was aligned with a population with risk factors for severe COVID-19 outcomes. Finally, interviews took place via Zoom. The use of a Zoom methodology restricts participation to those individuals who are able to access phones and internet-enabled devices.

Conclusion

The results of both the literature review and qualitative interviews support the CSDD’s ability to assess the COVID-19 symptoms most commonly experienced by adults; the results also indicate that the CSDD includes those symptoms participants identified as important to see treated. Specifically, the symptoms included in the CSDD represent those that were most frequently reported as being the most bothersome and important to improve with an efficacious treatment. Triangulation of the concept-elicitation and cognitive-debriefing data with data from the literature review support the content validity of the measure as well as the measure’s fitness for the purpose of evaluating COVID-19 symptoms in adult patients.