Design and Validation of Sleep Apnea Risk Assessment (SARA): A Screening Tool for Moderate-to-Severe Obstructive Sleep Apnea

Introduction

Obstructive sleep apnea (OSA) is the most common sleep-related breathing disorder (SBD), and moderate and severe OSA increase the risk of morbidity and mortality.^1,2 Approximately 936 million adults worldwide have mild to severe OSA, and 425 million adults aged 30–69 years have moderate to severe OSA. In some countries, the prevalence of OSA exceeds 50%.³ The prevalence of OSA is as high as 24% in men and 9% in women if accompanied by excessive daytime sleepiness.⁴ Obstructive sleep apnea in adults is characterized by ≥ 10 seconds of repetitive episodes of complete (apnea) or partial (hypopnea) upper airway obstruction during sleep. The apnea-hypopnea index (AHI), calculated as the average number of apneas and hypopneas per hour, was used to classify the OSA severity. Normal breathing in adults is AHI<5, mild OSA of AHI 5 < 15, moderate OSA of AHI 15 < 30, and severe OSA of AHI ≥ 30.⁵ Patients with variable amounts of central apneas or hypopneas are scored for OSA if obstructive episodes are predominant.⁴ The heart rate and peripheral blood oxygen saturation (SpO₂) are monitored besides each other.

Efficient diagnostic and treatment methods are essential to reduce the harmful health effects of moderate and severe OSA if undetected⁶ and maximize cost-effectiveness. Polysomnography (PSG) is the gold standard for diagnosing OSA, but the long waiting list complicates this examination. It is also expensive and time-consuming; thus, it is not suitable for the routine screening of OSA. The American Academy of Sleep Medicine Guide for Sleep Medicine Terminology recommends portable cardiorespiratory polygraphy (PG) monitoring and home sleep apnea testing using a scoring respiratory event index.⁷

Various screening questionnaires that accompany PG may provide sufficient diagnostic and inexpensive results. One of the most commonly used questionnaires is the STOP-Bang (SBQ).⁸ It has a sensitivity range of 88–93% and a specificity range of 28–42%.^9–12 The sensitivity and specificity of the Berlin Questionnaire (BQ)¹³ are 73–83% and 22–59%, respectively.^11,14–16 Silva et al¹⁷ compared sleep questionnaires: the Epworth Sleepiness Scale (ESS),¹⁸ SBQ, and STOP questionnaire¹⁹ and found that the sensitivity for moderate and severe OSA was 39.0%, 87.0%, and 62.0% and specificity was 71.4%, 43.3%, and 56.3%, respectively.

Sleep questionnaires accurately identify individuals with obstructive sleep apnea (OSA) (high sensitivity), but they often fail to rule out OSA-negative cases (low specificity). The high number of false-positive diagnoses causes unnecessary patient anxiety, additional testing, and increases healthcare costs. The problem of low specificity is consistent regardless of OSA severity.²⁰

Furthermore, the time required to complete questionnaires eg STOP-Bang, Berlin questionnaire, or PSQI is substantial, and their scoring systems often assign equal weight to diverse factors (hypertension is weighted the same as witnessed apnea in the STOP-Bang), potentially obscuring the relative importance of different symptoms in OSA diagnosis. A further limitation of named sleep questionnaires is their inability to stratify OSA severity. Consequently, questionnaire results cannot reliably differentiate between clinically significant (moderate-to-severe) and mild OSA.

This study aimed to develop a concise screening tool for identifying clinically significant obstructive sleep apnea (OSA), specifically moderate and severe grades, which are associated with the most substantial health risks and necessitate timely diagnosis and treatment. The target tool was designed to achieve at least equivalent sensitivity and improved specificity compared to existing questionnaires. The instrument was created for self-administration, requiring no ancillary equipment, and its scoring system was structured for rapid evaluation to facilitate widespread implementation without imposing an undue burden on healthcare providers.

We named this the Sleep Apnea Risk Assessment (SARA). We presented the original Czech version in Supplementary Material No 1 and professional transcriptions in Polish in Supplementary Material No 2 and English in Supplementary Material No 3. Although we cannot exclude possible cultural bias, we believe SARA has a potential to be tested and accepted across different populations.

Materials and Methods

We conducted a multicenter two-phase study to (1) design a new survey to screen for moderate and severe OSA and compare it to five sleep questionnaires (cohort A) and (2) validate it in an independent group of patients (cohort B). The Questionnaires (ESS, BQ, PSQI, and the SBQ) were used in accordance with its licensing agreement, ensuring compliance with all copyright and ethical guidelines. The appropriate license for translated versions of the scales was obtained from Mapi Research Trust. The STOP questionnaire comprises the first part of the STOP-Bang questionnaire.

Participants

Cohort A

From September 1, 2018, to June 30, 2020, we prospectively screened consecutive adult patients registered in an outpatient clinic for snoring and sleep disorders at the Ear-Throat-Nose (ENT) Department in the Czech Republic. Participants were enrolled in Cohort A if they met the following inclusion criteria: (1) age ≥18 years; (2) PG in a sleep laboratory; (3) ability to respond to the ESS, BQ, PSQI and SBQ questionnaires (with separate analysis of the STOP component) and (4) had no history of significant cardiorespiratory disease, neuromuscular respiratory weakness, hypoventilation (awake or sleep-related), chronic opioid use, stroke, or severe insomnia. Participants were excluded from the final analysis if they had (1) central sleep apnea (CSA), (2) incomplete PG data, or (3) incomplete data for at least three of the five questionnaires.

We collected demographic data and patients’ body mass index (BMI) in kg/m². The local Ethics Committee approved the study protocol. Written informed consent was obtained from all the participants.

Cohort B

We retrospectively collected data from consecutive screened subjects for OSA from hospital databases from January 1, 2022, through October 31, 2023, at Sleep Laboratories in Ostrava (Czech Republic) and Bielsko-Biała (Poland). The inclusion criteria were: (1) age≥18 years and (2) PG in a sleep laboratory. The exclusion criteria were (1) CSA, (2) incomplete PG data, and (3) missing information from the patients’ reports. To ensure data integrity and minimize bias, we included participants in Cohort B only if complete data were available for all variables. All registered patients in the health care institution provided written consent with data collection for research and academic purposes. The local ethics committees of both institutions approved this retrospective analysis protocol.

Methods

Phase 1-Cardiorespiratory Polygraphy, Questionnaires, and the SARA Design

Patients of cohort A underwent PG, Alice NightOne Respironics Inc., Murrysville, PA, USA; Hersching Germany, SN 2008270, under the technician’s supervision in the sleep laboratory. Oronasal airflow, thoracoabdominal breathing effort, pulse oximetry monitoring of heart rate and oxygen saturation, snoring, and body position were recorded. A somnologist scored the records manually following the American Academy of Sleep Medicine Scoring Manual.²¹ Moderate OSA was evaluated if AHI was 15 < 30 and severe if AHI was ≥ 30. All enrolled participants answered five defined OSA screening questionnaires: (1) The ESS, (2) the BQ, (3) the Pittsburgh Sleep Quality Index (PSQI),²² (4) the SBQ, and (5) the STOP Questionnaire. The most relevant variables for moderate and severe OSA were selected based on the Sleep Apnea Risk Assessment (SARA).

Phase 2-Validation

We validated the new screening questionnaire SARA retrospectively in an independent cohort B.

Statistical Analysis

To determine the individual risk factors for moderate and severe OSA (AHI≥15), binary logistic regression (crude OR: odds ratio) was conducted on data from five sleep questionnaires (ESS, BQ, PSQI, SBQ, and STOP) collected in cohort A. The factors that showed the best relationship with OSA occurrence were selected. The sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) of the parameter “Sleep apnea observed by the partner” were evaluated to determine its quality. Weights for individual SARA parameters were determined using fully adjusted odds ratios (ORadj). We subsequently selected the best cut-off value using the Youden index to optimize sensitivity and specificity, alongside consideration of the AUC. These results led to the development of SARA, a new screening questionnaire.

Descriptive statistics, such as frequency tables, mean, standard deviation, and minimum and maximum values, were used to describe the cohorts. For the comparison of demographic characteristics between cohorts A and B, a t-test and chi-square test were used, depending on the type of data. To evaluate the quality of the diagnostic tests, sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) were calculated and supplemented with 95% confidence intervals (CI). The area under the ROC curve (AUC) and likelihood ratio for the positive and negative tests were also assessed. A chi-square test was used to compare true and false results between the SARA questionnaire and the individual questionnaires from the five tested. All statistical tests were performed at a significance level of 5%. Stata software (version 16) was used for data processing.

Results

Cohort A

In total, 269 patients were screened. PG was incomplete in 19 (7.1%) patients, 26 (9.7%) patients did not reliably answer at least three of the five questionnaires, and three had CSA. We enrolled 221 (82%) patients aged 50.5±13.0; 153 (69.2%) were men, and 107 (48.4%) had a BMI≥30. and eighty-two patients answered five questionnaires, 25 answered four questionnaires, and 14 answered three questionnaires. Snoring was confirmed in 197 (89.1%) participants and sleep apnea was observed by a partner in 98 patients (44.3%).

Cohort B

Of 301 screened subjects, 253 eligible patients (84%) were enrolled. Among 117 patients screened in Bielsko-Biała (Poland), 96 (82%) were eligible. Of the 184 patients screened in Ostrava (Czech Republic), 157 (85%) were enrolled in the study. We excluded 14 (4.7%) patients with incomplete PG and 32 (10. 6%), with incomplete medical reports. Two patients (0.7%) had CSA. The mean age of enrolled patients was 48.0±13.4 years; 191 (75.5%) were men, and 122 (48.2%) had a BMI≥30. Two hundred and twenty-six (89.3%) participants reported snoring, and a partner observed sleep apnea in 164 (64.8%) participants.

The demographic data of cohort A and B we present in Table 1.

Table 1 Demographic Data and AHI Distribution in Cohort A and B

We present the inclusion/exclusion process in Figure 1.

Figure 1 Flow chart—the inclusion/exclusion process.

Phase 1

Cardiorespiratory Polygraphy, Questionnaires, and SARA Design

Cohort A included 221 patients who had completed the study protocol. Moderate-to-severe OSA (AHI≥15) was observed in 149 patients (67.4%). Of the five selected questionnaires (ESS, BQ, PSQI, SBQ, and STOP), we collected the most relevant demographic and clinical factors (age, sex, BMI, neck circumference, waist-to-hip ratio, height, weight, snoring, hypertension, and daytime fatigue). Based on the diagnostic ORs, the variables with the highest accuracy in detecting moderate and severe OSA (AHI≥15) were selected and designed for the new screening SARA questionnaire as follows: sleep apnea observed by a partner, snoring, male sex, age ≥ 50 years, daytime fatigue, and BMI≥30 kg/m². When determining the points, the logistic regression model for AHI > 15 and the clinical severity of the factors were used (Table 2). Six factors were assessed with the OR value for “Sleep apnea observed by a bedroom partner” being 4.75, and for the other factors, except for snoring, the OR values ranged between 2–3. The OR for snoring was high because of the minimal number of people without snoring. The total score was set at 24 (6 × 4 points), based on the number of factors and OR values. For the parameters of sex, age, daytime fatigue, and BMI, points were assigned according to the results of the models with rounded to whole values, also considering clinical significance and the distribution of the feature in the population (the incidence of obesity in the general Czech population is relatively high (52%), with 17% having BMI >30, so the point value was reduced to 2 from the OR value of 2.69). For the factors “Sleep apnea observed by a bedroom partner” and snoring, clinical severity and the calculated PPV and NPV were evaluated. For “Sleep apnea observed by a bedroom partner”, the PPV was 90% (95% CI 82.4–95.1), and NPV was 51.2% (95% CI 42.0–60.4), and for snoring, the PPV was 74.7% (95% CI 68.1–80.6) and NPV was 95.7% (95% CI: 78.1–99.9%). The factors “Sleep apnea observed by a bedroom partner” and snoring are clinically significant, but for individuals without a partner, it is impossible to determine; this alternative was also considered when assigning points. Therefore, the algorithm assumes two major risk factors: “Sleep apnea observed by a bedroom partner” and at least one other risk factor, or “Snoring” and two or more risk factors. Alternatively, the occurrence of all risk factors, except for the two main ones. Determination of the cut-off value confirmed our assumption. Based on Youden’s index, the cut-off value was set at 11, corresponding to the sum of all factors, except for the two main risk factors. The score allocates 8 points if a partner observes and reports sleep apnea and 5 points if snoring is reported. Other factors were scored according to ORs as follows: 3 points for men, 3 points for age≥50 years, 3 points for daytime fatigue, and 2 points for BMI≥30 kg/m². Zero points were counted when the answer was “negative” or “unknown”. The total SARA score was calculated as the sum of the points for each question, and ranged from 0 to 24. The data are presented in Table 2.

Table 2 The Most Relevant Variables to Design the SARA in Cohort A

Testing and Validation of SARA’s Cut-off Value in Cohort A

In the next step, we tested the cut-off values for the total SARA questionnaire scores. The most suitable cut-off value for cohort A, with a sample size of 221 patients, was 11. This cut-off value had a sensitivity and specificity of 87.2% (95% CI: 80.8–92.1) and 66.7% (95% CI: 54.6–77.3), respectively. The ROC curve was 0.77 (95% CI: 0.71–0.83). The PPV was 84.4% (95% CI: 77.7–89.8), and the NPV was 71.6% (95% CI: 59.3–82.0). The data are presented in Table 3.

Table 3 The Evaluation of the Performance of SARA for Cut-off Values of 11–15 Points in Cohort A

The SARA questionnaire showed the highest sensitivity and ROC area compared with most questionnaires used in sleep medicine. The confidence intervals of all questionnaires did not contain the ROC curve of SARA (0.77), thus proving the reliability of the results. There was a statistically significant difference in the proportion of true/false results between the SARA and ESS, PSQI, and STOP questionnaires for persons with moderate-to-severe OSA. A statistically significant difference was found between the SARA questionnaire and all questionnaires except the ESS for persons with no or mild OSA. The SARA yielded 24 false positives (33.3%) and 19 false negatives (12.7%). In all cases where a statistically significant difference was found, the proportion of true results was higher for the SARA questionnaire.

The results are presented in Table 4.

Table 4 Five Questionnaires and SARA Comparison in Cohort A: Sensitivity, Specificity, and ROC Area

The ROC and AUC for cohort A suggested a positive correlation between moderate and severe OSA risk and the total SARA questionnaire score. Figure 2.

Figure 2 The ROC and AUC for SARA in comparison with SBQ, BQ, and STOP questionnaires in cohort A and for SARA in cohort B. Abbreviation: AUC, Area under curve.

Phase 2

Validation of SARA

To validate SARA, we retrospectively tested and created an independent group of 253 patients (Cohort B). Moderate and severe OSA (AHI≥15) were observed in 153 patients (60.5%). Similarly, we confirmed the most suitable cut-off value of 11 points for SARA for moderate and severe OSA. The new questionnaire’s sensitivity at a cut-off value of 11 was 98% (95% CI: 89.2–95.4), the specificity was 59% (95% CI: 54.5–69.5) and the ROC area was 0.79 (95% CI: 0.74–0.84) in this cohort and is a consistent result with the outcome of the cohort A. Figure 2.

Discussion

We created the Sleep Apnea Risk Assessment (SARA), a quick and user-friendly questionnaire with a maximum of 24 points and a cut-off of≥11 points to screen for the risk of moderate and severe OSA with AHI≥15. The SARA sensitivity of 87.2%, specificity of 66.7%, and ROC area of 0.77 seems to have a high impact on effective screening for moderate and severe OSA compared to the efficiency of five commonly used questionnaires in sleep medicine (ESS, BQ, PSQI, SBQ, and STOP Questionnaire).

These questionnaires are used worldwide; however, the sensitivity and specificity of those published across the scientific community may differ according to OSA severity and geographic area.¹⁰ Almost 50% of our patients in both cohorts were obese with a BMI≥30 kg/m². Despite factors that may contribute to OSA among different racial and ethnic populations, obesity remains a risk factor for OSA across ethnic groups.⁵ Thus, we believe SARA can potentially give reliable results out of the Central European region. Completing and evaluating SARA is simple and comparable to the STOP and ESS questionnaires. The SBQ is more demanding because more parameters need to be measured or calculated.⁸ The PSQI and BQ contain more items, and their assessment is more complex and time-consuming than the SARA questionnaire. The efficient design of SARA and ease of use offer several promising clinical applications beyond routine screening. Pre-operative screening using SARA could identify surgical patients at higher risk of respiratory complications (eg, obese or elderly individuals) who may benefit from pre-operative sleep studies, optimizing post-operative respiratory management. Its efficiency also applies to large-scale population screening, facilitating identifying at-risk groups and enabling targeted preventive interventions. Furthermore, SARA can streamline referrals to sleep specialists by acting as a filter, prioritizing patients with a high likelihood of significant OSA and thereby reducing the burden on specialized sleep centers. Finally, SARA’s brevity makes it particularly well-suited for integration into the workflows of busy primary care practices, potentially enabling earlier identification and management of moderate-to-severe OSA.

SARA comprises six questions concerning the most significant factors according to the odds ratios. These are easy to identify, making SARA a straightforward and precise screening tool. The question of witnessed apneas is crucial as it has a very high PPV (90%). This question was missing in some questionnaires, such as the ESS. We focused primarily on excessive daytime sleepiness. The STOP-Bang, STOP, and Berlin questionnaires contained questions about the apneic events observed by a partner. However, the evaluation of a positive answer was rated similarly to that of the less significant factors.

In the SARA, we assigned points according to the significance of the factors. The highest OR group included questions concerning snoring. Logistic regression analysis showed that snoring was a more significant parameter than sleep apnea observed by a partner for detecting moderate and severe OSA. In our practice, most patients with OSA were treated in an outpatient clinic for snoring, and sleep disorders reported snoring, whereas only some reported sleep apnea observed by a partner. On the basis of this experience, the parameter of sleep apnea observed by a partner was tested separately as the most crucial factor. A high PPV (90%) for witnessed apnea by a partner was also confirmed by Costa et al,²³ suggesting that sleep apnea observed by a partner is an important factor. Compared with snoring, the lower odds ratio of this parameter can be explained by some patients’ lack of awareness of sleep apnea. Snoring is loud and easier to detect than sleep apnea, particularly by partners. Additionally, some patients do not have partners and therefore do not have information about their sleep. We decided to allocate 8 points if “Sleep apnea observed by a partner” was reported and 5 points if snoring was reported.

Daytime fatigue in patient reports is also a significant factor, but it cannot be used separately as in ESS. In cohort A, the sensitivity and specificity of the ESS were only 36.3% and 78.4%, respectively. El-Sayed et al²⁴ presented higher sensitivity for moderate OSA at 75.7% and 79.73% for severe OSA but lower specificity for moderate and severe OSA (48.2% and 46.4%, respectively). Silva et al¹⁷ presented a sensitivity of ESS for moderate OSA of 39%, specificity of 71.4%, sensitivity of 46.1%, and specificity of 70.4% for severe OSA, which is consistent with our results. Hesselbacher et al²⁵ presented a higher sensitivity in moderate OSA, but no data are available for severe OSA.

Other screening tools, such as the BQ or PSQI, are more complicated and their evaluation is more time-consuming than that of SARA. The SBQ includes a question regarding neck circumference, which we did not include in the SARA, as it often correlates with a BMI≥30kg/m².^26,27 Most patients know their weight, but very little is known about their neck circumference. Measurement of neck circumference requires extra engagement from the medical staff and the use of a measuring tape.

We designed a questionnaire to detect moderate and severe OSA (AHI≥15). OSA severity was intentional, as patients with moderate and severe OSA were at a greater risk of health problems than those with mild OSA. For instance, Peppard et al²⁸ and Nieto et al²⁹ reported that patients with AHI≥15 had a significantly higher risk of arterial hypertension than patients with AHI<15. Moderate and severe OSA increase the risk of stroke and chronic heart failure.^30–32

SARA had the highest sensitivity (87.2%) and ROC area (0.77) for detecting clinically significant OSA in our study compared with the other five questionnaires in cohort A. The sensitivity of the SBQ was the second highest after SARA (82.6%), and the specificity was only 46.7% compared with 66.7% for SARA. Silva et al¹⁷ reported similar results, with a sensitivity of 87% and specificity of 43.3%. Cohort A had the third-best BQ score with a sensitivity of 79.9% and a specificity of 43.5%. The BQ was studied frequently, and its sensitivity and specificity were achieved in other presented studies 73–83% and 22–61.9%, respectively^{11,14–16,33} consistent with our results. We prioritized SARA for its best sensitivity and ROC area for screening moderate and severe OSA compared with the other questionnaires. We screened SARA with≥11 points in 154 (70%) of 221 patients in cohort A, of whom 24 were false-positive and had normal breathing or mild OSA. SARA’s false positive rate (33.3%) is significantly lower than STOP-Bang’s 53.3%, reducing patient anxiety and healthcare costs associated with unnecessary further investigations. SARA yielded fewer false-negative results (12.7%) than SBQ and BQ (17.4% and 20.1%, respectively). SBQ and BQ include patients with mild OSA, and compared to more sensitive and specified SARA for moderate and severe OSA, they might extend the waiting list to diagnose and treat patients with higher risk. Compared to the three questionnaires with the highest sensitivity in cohort A (SBQ, BQ, and STOP), the specificity of SARA was 20% higher (66.7% vs 46.7%). SARA also had a lower occurrence of false-positive results for moderate and severe OSA (33.3%) than SBQ, BQ, and STOP (53.3%, 56.5%, and 54.3%, respectively). SARA and cardiorespiratory polygraphy can reduce the financial costs of unnecessary PSG examinations. We are aware of the risk of insufficient information about patients’ sleep and do not monitor the total sleep time and sleep stages; thus, we strongly support investigating patients in PSG laboratories if the clinical complaint is conclusive. A validation study in primary care settings should precede the widespread implementation of SARA.

Limitations and Strengths

While this study demonstrates SARA’s effectiveness, limitations include the use of a sleep-lab referral population (high pretest probability), potentially introducing selection bias. However, data from independent international centers showed no significant differences, suggesting the results may be generalizable across central Europe. Future work will involve prospective, observational primary care validation (screening during routine check-ups with positive cases referred for diagnostic testing). External validation in diverse populations, including healthy controls, is also needed to further enhance the test’s validity. PSG remains the gold standard for OSA diagnosis; however, SARA offers a cost-effective, accessible first-line screening option for moderate-to-severe OSA. We recommend PSG only if SARA indicates a high risk and PG results are inconclusive.

Conclusions

Sleep Apnea Risk Assessment (SARA) is a promising tool with high sensitivity and specificity for detecting moderate-to-severe OSA. Completing the questionnaire was simple and fast, making it a practical screening option. A validation study in primary care settings should precede the widespread implementation of SARA.