Performance of Four Screening Tools for Identifying Obstructive Sleep Apnea Among Patients with Insomnia

Introduction

Obstructive sleep apnea (OSA) is a clinical condition caused by repeated episodes of upper airway collapses during sleep. The resultant sleep disruption is frequently associated with excessive daytime sleepiness. Untreated OSA may increase the risk of cardiovascular disorders, stroke, diabetes mellitus, cognitive impairment, traffic accidents, and ultimately damage the quality of life and life expectancy.¹ It has been estimated that over 900 million individuals are affected by OSA globally, creating a massive socioeconomic burden.²

To date, polysomnography (PSG) and home sleep apnea test (HSAT) are the standard medical test for the diagnosis of OSA.³ However, the time-consuming and technology-dependent PSG test often hinders the assessment.⁴ While the HSAT provides a more feasible technique, the resources for sleep services are still inadequate in vast regions where health resource is limited, leaving a large proportion of OSA cases unrecognized.⁵ To cope with this situation, numerous screening tools have been developed.⁶ These tools rapidly identify individuals at high risk for OSA and guide subsequent diagnostic studies in general clinical settings. The adaptability of these tools for specific at-risk OSA populations demands validation.

Co-morbid insomnia and sleep apnea (COMISA) has attracted much attention as it has been linked to higher levels of psychological distress, greater cardiovascular and metabolic risk, and increased all-cause mortality.^7–9 The pooled prevalence of insomnia among patients with OSA was 38%, and up to 35% of insomnia patients fulfilled the diagnostic criteria for OSA.¹⁰ This higher-than-expected prevalence of the comorbidity, together with other experimental and clinical evidence, has suggested an interplay between the two conditions, leading to a cycle of poor sleep quality and impaired daytime function.⁷

The scale of excessive daytime sleepiness has been incorporated in some of the most frequently used OSA screening questionnaires, including STOP-Bang,¹¹ Berlin,¹² and Epworth sleepiness scale (ESS).¹³ However, nearly half of insomnia sufferers may also experience excessive daytime sleepiness, questioning the performance of such tools in this population.¹⁴ To date, experience of OSA screening among individuals with insomnia remains scarce.¹⁵ The current study was designed to evaluate the performance of STOP-Bang, Berlin, ESS, as well as NoSAS (a newly proposed screening tool without items associated with daytime sleepiness) in a sample of consecutive adult patients with or without insomnia.

Methods

Participants

This cross-sectional study was conducted at a sleep breathing disorder clinic in northern China.

Individuals were included if they had been suspected of OSA and referred to in-laboratory PSG. Individuals were excluded for any of the following reasons: 1) age <18 years, 2) received treatment for sleep apnea prior to enrollment, 3) unable to fill out or did not complete the electronic questionnaire; 5) technically inadequate PSG. All participants provided written consent. The study was approved by the Research Ethics Committee of Peking Union Medical College Hospital (Approval No. K24C2001) and performed in accordance with the Declaration of Helsinki.

As the study was designed to compare primarily the sensitivity of screening tools, the sample size required for participants with insomnia was calculated through the equation where n is the sample size, and are the reported sensitivity of two close performing tools (STOP-Bang and NoSAS) for identifying moderate-severe OSA, and Ψ is the probability of disagreement between the two tools.^15,16 Here we applied an of 5%, of 10%, and maximum Ψ of . Assuming one-fourth of patients with suspected OSA has comorbid insomnia, the final enrollment size was estimated to be at least 828. Considering the annual workload of our center, we enrolled participants from January 2022 to December 2023.

A total of 1428 individuals with suspected OSA were referred for in-lab PSG during the study period, among which 162 individuals (11%) were excluded according to the criteria. The number of individuals excluded due to each exclusion criteria was shown in Figure 1.

Figure 1 STROBE flowchart of the study. Abbreviation: ISI, Insomnia Severity Index.

Before performing PSG, patient characteristics including age, sex, marital status, smoking and drinking status, and use of sleep aid were collected. Information on underlying clinical conditions was collected both from patient’s self-reports as well as the electronic medical record system. On the night of PSG, an integrated electronic form that consisted: 1) Insomnia Severity Index (ISI); 2) STOP-Bang questionnaire; 3) Berlin questionnaire; 4) Epworth Sleepiness Scale (ESS); 5) NoSAS score was then filled out by each participant. Assistance, if needed, was provided by a sleep physician during the survey. The height, body mass, and neck circumference (NC) were then measured. The body-mass index (BMI) was calculated by dividing the weight by the square of the height.

Insomnia Assessment

The ISI is a validated and globally employed questionnaire assessing the nature, severity, and impact of insomnia.¹⁷ The self-reported instrument is composed of 7 items that evaluate 1) the severity of sleep-onset, 2) sleep maintenance, 3) early morning awakening, 4) satisfaction with current sleep pattern, 5) interference with daily functioning, 6) noticeability of impairment attributed to the sleep problem, 7) level of distress caused by the sleep problem during the last two weeks. Each of these items is rated on a 5-point Likert scale (eg, 0 = no problem, 4 = very severe problem), yielding a total score from 0 to 28. A total score ≥15 had been proposed by the developers as “clinically significant insomnia”.¹⁷ This cutoff was adopted by the current study for the grouping of participants. For simplicity, the group with ISI ≥15 was referred to as “clinical insomnia”, while the group with ISI <15 was referred to as “non-insomnia” in the text. This only reflects the severity and impact of insomnia and does not imply a clinical diagnosis.

OSA Screening

The STOP-Bang questionnaire includes 8 yes or no items: “S” for loud snoring, “T” for tiredness during daytime, “O” for observed apnea, “P” for high blood pressure, “B” for BMI >35 kg/m², “a” for age >50 years, “n” for NC > 40 cm, and “g” for male gender.¹¹ One point is given for each question answered with yes, and a total score of 3 or more sensitively predicts OSA in both the original study and a validation study among the Chinese population.^11,18

The Berlin questionnaire consists of 10 questions with additional information on height and weight. These items were arranged into 3 categories: 1) snoring and cessation of breathing (5 questions); 2) excessive daytime sleepiness (3 questions); 3) BMI and hypertension (1 question plus BMI). Positive scores in 2 or more categories suggest a high risk for OSA.¹²

The ESS measures the level of daytime sleepiness by the chances of falling asleep in 8 daily conditions. Each item is rated from 0 (would never doze) to 3 (high chance of dozing), totaling a score from 0 to 24.¹³ ESS score above 10 has been widely recognized as excessive daytime sleepiness which motivates further OSA evaluation in patients with snoring.¹⁹

The NoSAS is a clinical score that allocates 4 points for NC > 40 cm, 3 points for 25 kg/m² < BMI < 30 kg/m² or 5 points for BMI > 30 kg/m², 2 points for snoring, 4 points for age > 55 years, and 2 points for being male.²⁰ A threshold of 8 points was chosen in the original study to identify individuals at risk for clinically significant OSA. The utility of NoSAS has been validated in a sleep clinic-based study in China.²¹

Among the above tools, daytime function is weighted as follows: 8/8 items in ESS, 1/3 categories in Berlin questionnaire, 1/8 items in STOP-Bang questionnaire, and none of the items in NoSAS.

Polysomnography

All participants underwent full-night PSG (Embla N7000; Natus Medical Incorporated, Orlando, USA; or SOMNO HD; SOMNO Medics AG, Randersacker, Germany) from 10 PM to 6 AM. Electroencephalogram, electrooculogram, chin electromyogram, electrocardiogram, airflow, respiratory effort, pulse oxygen saturation (SpO₂), snoring, and body position were continuously recorded. Data were recorded on software that accompanies the device (RemLogic for Embla N7000; DOMINO for SOMNO HD) and scored by two experienced sleep technicians following the AASM Manual for the Scoring of Sleep and Associated Events (version 2.6). The technicians were blinded for insomnia status as well as OSA screening results. The documented Cohen’s kappa in sleep staging between the two technicians was 0.80 [95% CI 0.75, 0.84].

Apnea was scored as a drop of airflow ≥90% from the pre-event baseline lasting at least 10 seconds. Hypopnea was defined as a drop of airflow ≥30% lasting at least 10 seconds with an associated ≥3% decrease in SpO₂ or arousal response of more than 3 seconds. AHI was defined as the number of apnea and hypopnea events per hour of sleep. Oxygen desaturation index (ODI) was defined as the number of SpO₂ drops of 3% or more per hour of sleep. Total sleep time (TST), sleep latency, sleep efficiency, wake after sleep onset (WASO), the percentage of each sleep stage, the lowest oxygen saturation by pulse oximetry (LSpO₂), and the percentage of sleep time with SpO₂ below 90% (T90) were also collected.

Data Analysis

Data were analyzed with SPSS v. 22.0 software (IBM, Armonk, NY, United States). The normality of variables was tested by the Kolmogorov–Smirnov test. Demographic, clinical, and PSG data were summarized either as means and standard deviation (quantitative data in normal distribution), median and interquartile (quantitative data not in normal distribution), or exact counts and percentages (qualitative data). Comparisons between the insomnia and non-insomnia groups were made using Student’s t-test or Mann–Whitney U-test for continuous variables, and chi-square test for categorical variables. Correlations between scores of the screening tools and AHI were examined by Pearson’s (STOP-Bang, ESS, and NoSAS) or Spearman’s (Berlin) correlation analysis. Using PSG data as the gold standard, the sensitivities, specificities, positive predictive value (PPV), and negative predictive value (NPV) of each screening tool were calculated. The diagnostic performance of screening tools was further assessed through the net reclassification improvement (NRI), area under the receiver operating characteristic (ROC) curves and Youden’s index (YI). The NRI and its significance testing were calculated by the method proposed by Pencina et al.²² The areas under the ROC curves (AUCs) were compared with the DeLong method using the MedCalc software (MedCalc Software Ltd, Ostend, Belgium).²³ All statistical tests were 2-sided unless specified, and P values less than 0.05 were considered statistically significant.

Results

Study Population

Of the 1266 eligible patients, 313 were classified as clinical insomnia. Characteristics of the insomnia and non-insomnia groups are shown in Table 1. Demographic and clinical characteristics were comparable between the two groups except for a higher proportion of females and use of sleep aids among participants with insomnia.

Table 1 Description of the Study Population

No significant difference was identified regarding the means of AHI and ODI between the two groups. The prevalence of AHI≥5/h, AHI≥15/h, AHI≥30/h was 72%, 48%, 28% for the insomnia group, and 76%, 52%, 34% for the non-insomnia group, respectively. There was no significant difference in the frequency between insomnia- and non-insomnia patients (P>0.05 in all comparisons).

Screening Scores in Different Insomnia Severity Groups

The STOP-Bang and NoSAS scores did not differ significantly between the insomnia and non-insomnia groups (Table 1). The ESS and number of positive categories in the Berlin questionnaire were greater in the clinical insomnia group.

The contribution of items related to daytime function in the STOP-Bang and Berlin questionnaire was then investigated. The positive rate for “tired” in the STOP-Bang was 52% in the insomnia group and 49% in the non-insomnia group (P>0.05). Notably, the positive rate for category 2 (excessive daytime sleepiness) in Berlin questionnaire was 50% in the insomnia group, compared to only 17% in the non-insomnia group (P<0.0001). The total score and positivity of category 1 (snoring and cessation of breathing) and category 3 (BMI and hypertension) of Berlin questionnaire were comparable between the insomnia and non-insomnia groups (P>0.05 in all comparisons).

Using conventional cutoffs, high risk for OSA was present in 81% of insomnia and 75% of non-insomnia participants defined by STOP-Bang (P=0.02), 65% of insomnia and 46% of non-insomnia participants defined by ESS (P<0.0001), 63% of insomnia and 43% of non-insomnia participants defined by Berlin questionnaire (P<0.0001), and 48% of insomnia and 52% of non-insomnia participants defined by NoSAS (P>0.05) (Table 1).

Correlation Between Screening Scores and Polysomnographic Data

The correlation between screening scores (positive categories in the case of the Berlin questionnaire) and PSG parameters that reflect the severity of OSA (ie, AHI, ODI, LSpO₂, T90) was evaluated among participants with and without insomnia (Table 2). The STOP-Bang and NoSAS scores demonstrated low but significant correlation with AHI, ODI, and LSpO₂ with and without the presence of clinical insomnia. The correlation between ESS or Berlin questionnaire with PSG data was relatively weak, as in both groups r or Spearman rho were <0.3 or around 0.3 for all variables.

Table 2 Correlation Analysis Between Screening Scores and Polysomnographic Data

Predictive Value of Screening Tools

The predictive performance of the screening tools was presented in Table 3. Compared with participants without insomnia, the STOP-Bang, ESS, and Berlin questionnaires, demonstrated clear trends toward higher sensitivity and lower specificity in screening for all levels of OSA among participants with insomnia. The difference in the predictive value of NoSAS was relatively small between the insomnia and non-insomnia groups.

Table 3 Predictive Value of OSA Screening Tools Among Patients with or Without Insomnia

Among participants with clinical insomnia, the STOP-Bang demonstrated the highest sensitivity [87.4% (95% CI 82.2–91.4) for AHI≥5/h, 93.2% (95% CI 87.6–96.5) for AHI≥15/h, 94.4% (95% CI 86.8–97.9) for AHI≥30/h], while other screening tools had sensitivities less than 80% for all AHI cutoffs. In terms of specificity, the NoSAS was the highest among the four instruments (72.7% (95% CI 62.0–81.4) for AHI≥5/h, 67.7% (95% CI 59.9–74.6) for AHI≥15/h, 62.9% (95% CI 56.2–69.2) for AHI≥30/h). Compared to STOP-Bang, NoSAS brought a net reclassification improvement of 6.8% for AHI≥5/h (P>0.05), 9.7% for AHI≥15/h (P>0.05), and 18.8% for AHI≥30/h (P=0.0017).

The STOP-Bang and NoSAS displayed AUCs higher than 0.7 at all levels of OSA severity (Figure 2). In paired analyses using the DeLong method, the AUCs of STOP-Bang and NoSAS were significantly higher than either the ESS or Berlin questionnaire (p<0.05 in all cases). The comparisons between AUSs of STOP-Bang and NoSAS did not yield significant difference in either AHI≥5/h [0.74 (95% CI, 0.68–0.79) vs 0.73 (95% CI, 0.67–0.80)], AHI≥15/h [0.71 (95% CI, 0.68–0.79) vs 0.72 (95% CI, 0.66–0.77)], or AHI≥30/h [0.73 (95% CI, 0.67–0.79) vs 0.75 (95% CI, 0.70 −0.81)] conditions (P>0.05 in all comparisons).

Figure 2 Receiver operating curves of common sleep apnea screening tools in participants with clinical insomnia. Obstructive sleep apnea severity was classified by the AHI. The STOP-Bang questionnaire and NoSAS score had similar AUCs, both performing higher discriminative capacity than the ESS and Berlin questionnaire at all OSA severity.

As the STOP-Bang and NoSAS have an overall advantage among participants with insomnia, the performance of the two tools to predict moderate-severe OSA under each cutoff was determined (Figure 3). The maximum YI for STOP-Bang (0.302, 95% CI 0.190–0.385) was reached at score 4 with sensitivity of 73.7% (95% CI, 65.8–80.5%) and specificity of 56.5% (95% CI, 48.5–64.3%). The maximum YI for NoSAS was reached at score 7 (0.339, 95% CI 0.239–0.416) with sensitivity of 77.2% (95% CI, 69.6–83.7%) and specificity of 56.7% (95% CI, 48.8–64.4%).

Figure 3 Performance of STOP-Bang and NoSAS to predict moderate-severe obstructive sleep apnea among participants with insomnia at each cutoff. The maximum YI was reached at score 4 for STOP-Bang, and score 7 for NoSAS, respectively. Error bars stand for 95% confidence interval. Data were not shown for NoSAS scores at 1, 14, and 16 as these scores are not reachable by the definition of the tool. Abbreviations: Se, sensitivity; Sp, specificity; YI, Youden’s index.

Discussion

The current study evaluated on the performance of four commonly used OSA screening tools in the specific population of insomnia. It was noted that the sensitivity and specificity of STOP-Bang, ESS, and Berlin questionnaires had considerable difference between the insomnia and non-insomnia population. Within the insomnia population, STOP-Bang and NoSAS demonstrated an advantage over the ESS or Berlin questionnaire in terms of discriminative power. The conventional cutoff of STOP-Bang score ≥3 provided excellent sensitivity for screening, while analysis of the YI favors a NoSAS cutoff score lower than the originally proposed one.

The clinical application of screening questionnaires focuses on selecting patients who are at high risk of OSA. An essential aspect of an ideal screening tool is the adaptability to both general and specific population.⁶ Numerous studies have validated the performance of STOP-Bang, ESS, and Berlin questionnaire in patients referred to sleep clinics for suspected OSA.^24–26 However, when the targeted sample was extended to patients referred to a non-respiratory-based sleep clinic with various sleep disorders, the Berlin questionnaire yielded lower-than-expected performance,²⁷ suggesting a possibility that other common sleep disorders and/or medications may interfere. The current study demonstrated how the presence of insomnia significantly increases the ESS score and the score of the daytime sleepiness category of Berlin questionnaire. This resulted in a notable change to the performance of ESS and Berlin questionnaire. By contrast, the NoSAS tool, without evaluation of daytime sleepiness, had a similar performance between the insomnia and non-insomnia groups. It should be noted that excessive daytime sleepiness is a highly prevalent condition with various underlying conditions including insufficient sleep, circadian rhythm disorders, central disorders of hypersomnolence, and the use of medications (eg, alcohol, benzodiazepines, opiates).²⁸ The results from the current study imply that OSA screening tools incorporating measures of daytime function should be used with caution not only in the presence of comorbid insomnia but also in other conditions that increase the likelihood of sleepiness.

In the present study, participants completed all the four screening tools before undergoing PSG, providing a chance to compare the performance of these tools. The results support the use of STOP-Bang and NoSAS, over ESS or Berlin, for superior capability measured by AUCs in the insomnia population. Previous comparative studies conducted in the general population have favored the use of STOP-Bang over Berlin or ESS for higher discriminative capability.^29,30 Likewise, the more recently developed NoSAS, has outperformed Berlin and ESS in a number of reports.^15,20,21 Similar AUCs were observed for STOP-Bang and NoSAS in the current study, which is consistent with two previous sleep clinic-based studies.^15,31 It has also been confirmed that the established correlation between STOP-Bang and NoSAS scores with the severity of OSA^32,33 may extend to individuals with insomnia. Both the STOP-Bang and NoSAS can be easily calculated based on self-reports and simple physical examination, enabling convenient assessment in outpatient settings.

In practice, individuals at risk are selected by screening tools through certain cutoffs. A STOP-Bang score of 3 or more provides high sensitivity with acceptable specificity in most conditions and has been adopted in various populations (eg, general, surgical, sleep clinic, commercial drivers) across different geographic regions.^34,35 However, when the demographic features of a certain population directly influence the STOP-Bang score, as in the case of obesity, a higher cutoff might improve the balance of sensitivity and specificity.³⁶ In the current study, although STOP-Bang score cutoff of 4 maximized the YI among the insomnia population, the sensitivity for detecting moderate-severe OSA was below 80%. Considering the application scenario of the questionnaire, we recommend the conventional cutoff score of 3, which retained high sensitivity across the entire spectrum of OSA severity. For NoSAS, the presence of insomnia had little effect on its performance. However, the proposed cutoff of 8 was associated with poor sensitivity and NPV (both below 70%) for AHI≥15/h in the current study, and the cutoff of 7 reached a higher YI with a sensitivity of 77.2%. Notably, the cutoff of 8 provided sensitivity of 79% and NPV of 90% for AHI≥20/h in the original population-based study conducted in Switzerland.²⁰ The age distribution might be the main factor contributing to this discrepancy, as only 24% of participants in the current study were over 55 years old (the single age cutoff assigned with 4 points), compared to a mean age of 59 years in the derivation study. By contrast, the performance metrics of NoSAS in the current study is more comparable to a population-based cohort study in Singapore with a similar age distribution.³⁷ It should also be pointed out that Asian OSA patients are reported with lower BMI,^38,39 and whether ethnic-specific BMI cutoff should be adapted has been debated in the field.⁴⁰ Moreover, the study setting may have an impact as the cutoff of 7 was proposed by another sleep clinic-based study to achieve optimal performance.³³ In general, the present study opened up discussion on the issue of appropriate cutoff selection for OSA screening tools in the presence of insomnia. External validation of the proposed cutoffs in population samples with ethnic diversity would be highly valuable.

The current study represents an attempt to evaluate the performance of commonly used OSA screening tools among the insomnia population. The findings provide guidance on the selection of instruments, choice of cutoffs, and interpretation of screening results. The study is limited by the selection strategy and single-center design. The inclusion of referred patients increased the pre-test probability of OSA, and the sample consisted mainly of middle-aged overweight males. Validations in population-based samples with a broader age range and a more balanced sex ratio should be sought in the future.

Conclusion

The performance profile of OSA screening tools consisting daytime function measurement may be altered by the presence of insomnia. Among individuals with insomnia who were referred for sleep studies, the STOP-Bang and NoSAS provided overall superior discrimination performance compared with Berlin questionnaire and ESS. A STOP-Bang score ≥3 may sensitively select individuals at risk. The appropriate cutoff score for NoSAS needs further validation.