Establishment and Validation of a Nomogram for Identifying False Positives in Xpert MTB/RIF Rifampicin Resistance Test

Introduction

Tuberculosis, caused by the Mycobacterium tuberculosis (MTB) complex, remains a major public health concern worldwide. According to the 2023 World Health Organization report, approximately 410,000 people worldwide had developed rifampicin-resistant tuberculosis (MDR/RR-TB) in 2022, and only 175,650 of these were admitted for treatment.¹ A timely and accurate diagnosis of MDR/RR-TB is vital to global tuberculosis control. The traditional phenotypic drug susceptibility test (pDST) is the gold standard for diagnosing rifampicin resistance. However, as this method is complex to operate, with a longer detection cycle and long waiting times, there is often a delay in diagnosis. This in turn increases the risk of continued transmission of tuberculosis in the community.² In the past decade, with the continuous advancement of molecular diagnostic technology, especially the emergence and widespread promotion of Xpert detection technology, the detection speed of RIF-R has significantly improved. Xpert is a molecular detection technology based on semi-nested real-time fluorescence quantitative PCR which can simultaneously diagnose tuberculosis and RIF-R by detecting mutations within the rifampicin resistance-determining region (RRDR).^3,4 Xpert has been widely promoted worldwide because of its rapid (approximately 2 hours), accurate, ability to simultaneously detect tuberculosis and RIF-R, and independence from laboratories.^5,6 However, owing to the influence of certain factors, 9.00%–17.31% of patients experience false positives in Xpert for detection of RIF-R.^7–9 This situation seriously interferes with doctors’ accurate diagnosis of drug-resistant tuberculosis, especially in primary hospitals that do not carry out DST. These hospitals rely heavily on the Xpert technology to determine whether a patient has RIF-R tuberculosis. With a false-positive result for RIF-R in the Xpert test, the patient can easily be misdiagnosed as having RIF-R tuberculosis, leading to unnecessary initiation of second-line drug treatment.¹⁰ This delays the effective treatment time and increases the patient’s financial burden. In addition, the cure rate of patients with RIF-R tuberculosis is much lower than that of patients with rifampicin-sensitive (RIF-S) tuberculosis (66% vs 88%).¹ Misdiagnosis as RIF-R tuberculosis undoubtedly brings a heavy psychological burden on the patients, further aggravating their condition along with the difficulty of treatment. Therefore, close attention must be paid to the phenomenon of false positives in the Xpert for detection of RIF-R. Some studies have attributed this false-positive phenomenon to factors such as probe mutation (probe B, OR = 5.67), probe delay (ΔCT 4–4.9, OR = 4.09), and bacterial load (extremely low, OR = 63.6).^8,11 However, most existing studies have focused on identifying these factors and have not established an effective strategy to help doctors make appropriate decisions.

Based on a retrospective analysis of the clinical data of inpatients at our hospital, we constructed a nomogram diagnostic prediction model. The model incorporates common key variables and conducts an in-depth analysis and deconstruction of important variables. This study aimed to assist clinicians, especially at primary medical institutions, in accurately identifying false positives of Xpert RIF-R and reduce the misdiagnosis of multidrug-resistant tuberculosis.

Material and Methods

We retrospectively analyzed the clinical data of patients hospitalized in Hangzhou Red Cross Hospital between September 2019 and February 2023. All included patients had complete Xpert and tuberculosis DST data. The diagnostic criterion for a true positive is that the initial Xpert test and DST are RIF-R. A diagnostic criterion is that an initial Xpert test result as RIF-R and the DST as RIF-S is considered as a false-positive result. Study variables included age, sex, resistance probe, specimen type, ΔCT in probe delay, Xpert quantitative data, type of tuberculosis, tuberculosis treatment history, diabetes, chronic liver disease, chronic nephrosis, hypertension, autoimmune disease, malignancy, and cardiovascular disease. As there was only one HIV case, HIV was not studied as a separate variable. All data were obtained from our hospital’s electronic medical record system. Definition of variables: chronic kidney disease, a disease that results in a progressive loss of kidney function; chronic cardiovascular disease, a disease that requires long-term use of cardiac medications such as coronary artery and valvular disease, cardiomyopathy, and arrhythmias; liver disease, a disease that results in progressive destruction and regeneration of the liver parenchyma, such as chronic viral hepatitis, alcoholic liver disease, and hepatic insufficiency.

This was a retrospective study and did not involve personal or commercial interests. The Ethics Committee of the Hangzhou Red Cross Hospital approved this study, with an exemption from the informed consent requirement (Ethical Application Ref: 2024YS031). This study complied with the Declaration of Helsinki. The research team was committed to maintaining the confidentiality of all patient information stored in the electronic medical record database.

Patient Selection

The inclusion criteria were as follows: (1) patients having detailed basic characteristics of the population, laboratory tests, and other clinical information; (2) those with valid Xpert test results for rifampicin resistance; and (3) those with a positive culture for MTB culture along with a DST profile. The exclusion criteria were as follows: (1) patients with negative, RIF-S, or invalid Xpert test results and (2) those with a negative culture for MTB or a positive culture but lacking rifampicin DST results. If the patient underwent repeat testing during hospitalization, the results of the first Xpert test were considered and included in the analysis.

MTB Culture and pDST

MTB was cultured in a liquid culture medium using the BACTEC MGIT 960 Mycobacteria Culture System (BD Diagnostic Systems, New Jersey, USA) according to the manufacturer’s instructions.

Phenotype drug susceptibility test (pDST) involved culturing MTB in the presence of a known concentration of the test drug (RIF: 1.0 µg/mL). The growth of MTB was observed and compared with that of a control tube without any drug. If both control and drug culture tubes exhibited growth, the bacterial strain was considered resistant to the drug. In contrast, if growth was observed only in the control tube, the bacterial strain was considered sensitive to the drug.

Xpert MTB/RIF

Xpert testing was performed in strict accordance with the manufacturer’s instructions. On completion of the experiment, the system automatically records the MTB and RIF-R results. This technology uses five molecular beacon-overlapping probes to simultaneously detect mutations in the RRDR gene (including codons 507–533), namely, A (codons 507–511), B (codons 512–518), C (codons 518–523), D (codons 523–529) and E (codons 529–533). The detection of RIF resistance was based on the absence or delay in the binding of the five probes. MTB-specific beacon, ΔCT, is the difference between the first (early CT) and the last (late CT) and is the basis for detecting RIF-R.

Statistical Analysis

The patients included in the study were randomly divided into training and validation groups at a ratio of 7:3. The training group was used for modeling, and the validation group was used to test the performance of the model. For skewed distribution measurement data, the median (M) and upper and lower quartiles (Q1, Q3) were used to describe the central tendency and dispersion of the data, and the Mann–Whitney U-test or Kruskal–Wallis test was used for comparison between the groups. Count data were described using the number of cases (n) and composition ratio (%), and the χ² test or Fisher’s exact probability test was used. LASSO regression was employed to filter the variables and reduce data dimensionality, and the final model selection was performed using a backward step-down selection process with the Akaike information criterion. Model performance was evaluated using receiver operating characteristic curves, calibration plots, and decision curve analysis (DCA). The comparegroups package was used for baseline description and difference analysis, the glmnet package was used for LASSO regression, the glm package was used for multifactor logistic regression, the ggROC package was used for discrimination analysis, the rms package was used for calibration, the resource selection package was used for Hosmer–Lemeshow testing, the rmda package was used for DCA curves, and the rms package was used for the nomogram. Statistical analysis was performed using the R software package (version 4.2.1) and DCPM (V4.01, Jingding Medical Technology Co., Ltd).

Results

Between September 2019 and February 2023, 21,998 Xpert assays were performed at our hospital. A total of 21,328 cases were excluded based on the standards, including 14,456 cases with negative Xpert, 6821 testing RIF-S, and 51 with invalid test results. After screening, 670 cases with RIF-R were initially included in this study. Subsequently, 206 cases without DST results and 80 with repeat test results were excluded. Finally, 384 patients with RIF-R, initially detected using Xpert, were included in this study. At a ratio of 7:3, the study population was randomly divided into training (n = 268) and validation (n = 116) groups (Figure 1). The incidence rates of false-positives in the training and validation groups were 21.64% (58/268) and 23.28% (27/116), respectively.

Figure 1 The entry rules of study population. Abbreviations: pDST, phenotypic drug susceptibility testing; RIF-S, rifampin-sensitive; RIF-R, rifampin resistance. Note: * excluded specimens.

Comparison of Basic Characteristics of the Study Population

Basic and clinical characteristics and laboratory test results of patients in the true-positive and false-positive groups were compared. The results revealed that there were differences in the resistance probe, ΔCT in the probe delay, and history of TB treatment between the two groups, and the differences were statistically significant (P<0.05, Table 1).

Table 1 Comparison of Basic Characteristics of the Study Population

Variable Screening and Model Building

LASSO regression was used to screen 15 feature variables of the training cohort, and based on the Lambda.min principle, 6 were finally selected (Xpert quantitative result, resistance probe, ΔCT in probe delay, tuberculosis treatment history, hypertension, cardiovascular diseases) (Figures 2A and B). The variables screened by LASSO regression were incorporated for multivariate logistic regression, using a backward step-down selection process with the Akaike information criterion. We constructed a clinical prediction model containing two variables (resistance probe and ΔCT in probe delay) and plotted a diagnosis nomogram (Figures 3 and 4).

Figure 2 Selection of features using the LASSO binary logistic regression model: (A) regression coefficient path diagram and (B) cross-validation plot.

Figure 3 Forest plot of multivariate analysis of false positives for RIF-R detected by Xpert.

Figure 4 Diagnostic nomogram for identifying false positives RIF-R in Xpert.

Validation of Diagnostic Nomogram

There was no statistically significant difference in the general information between the training and validation cohorts (P=0.826, Table 2). The area under the curve of the training and validation cohorts were 0.847 (95% CI: 0.794–0.901) and 0.850 (95% CI: 0.770–0.931), respectively (Figures 5A and B). The results demonstrate that the nomogram had a good ability to distinguish false positives from Xpert detection of RIF-R. There was a good agreement between the predicted and observed values, as seen by the calibration curve for the prediction of R-R false-positives by Xpert (Figures 6A and B). Simultaneously, the Hosmer–Lemeshow test was used to evaluate the calibration of the model. The test revealed that the model’s predicted risk was consistent with the actual risk (training cohort: χ² = 0.331 and P=0.847; validation cohort: χ²= 4.822 and P=0.185). The clinical applicability of the model was evaluated using clinical decision curve analysis. The DCA demonstrated that the model achieved the greatest net benefit when the threshold probability was set between 6% and 71% in the training cohort and 6% and 70% in the validation cohort (Figures 7A and B). The patients were more likely to benefit from clinical interventions within the threshold probability range.

Table 2 Comparison of Baseline Characteristics Between Training and Validation Cohorts

Figure 5 Receiver operating curve for the nomogram as measured by bootstrapping for 500 repetitions: (A) derivation cohort and (B) validation cohort.

Figure 6 Calibration curve using bootstraps sampling 500 for predicted probability of the nomogram: (A) derivation cohort and (B) validation cohort.

Figure 7 Decision curve analysis using bootstraps sampling 500 for the prediction model: (A) derivation cohort and (B) validation cohort.

Discussion

Xpert technology significantly reduces the time to diagnose tuberculosis and RIF-R tuberculosis.¹² Since its recommendation by the World Health Organization, Xpert MTB/RIF has been widely used in many countries with a high TB burden. However, false-positive results for RIF-R by the Xpert greatly hinder the development of effective antituberculosis treatment regimens.¹⁰ This phenomenon has attracted widespread attention, particularly in primary medical institutions that cannot conduct DST. The doctors are unable to identify false-positive results for RIF-R in the Xpert tests, which can easily lead to misdiagnosis. Even in medical units that can perform tuberculosis DST, results are available only 2–3 months after the patient visits. During the transition period between Xpert and DST, patients are often unnecessarily treated with second-line drugs as false-positive Xpert results are interpreted as RIF-R.

Previous studies have reported that false positives in Xpert detection of RIF-R are mainly related to factors such as silent mutations detected in the rpoB gene, delayed binding of specific Xpert probes, low bacterial load, and some controversial resistance site mutations.^13–15 Although the factors affecting the occurrence of false positives for RIF-R in Xpert tests are known, intuitive predicting the risk of false positives for RIF-R in individuals is not possible and detailed information for decision-making regarding disease management and treatment options cannot be provided. As an intuitive clinical prediction model, the nomogram has gradually emerged in the medical field since Puppo and Perachino first used it to predict lymph node metastasis of prostate cancer in 1997.¹⁶ With their unique advantages to provide a scientific basis for clinical decision-making, after years of usage and development, nomograms are being used by more medical researchers.¹⁷ This study aimed to establish a clinical prediction model based on demographic characteristics, clinical manifestations, and laboratory tests to help clinicians distinguish false-positive RIF-R diagnoses by Xpert this avoiding unnecessary misdiagnosis.

Of the 15 variables screened in this study, we finally obtained two independent influencing factors: the type of Xpert resistance probe and the ΔCT in the probe delay. Resistance probes are known to be influencing factors. Although many studies believe that probe mutation is an influencing factor in the occurrence of false positives in the detection of RIF-R by Xpert, slight differences between the study results exist. For example, Berhanu et al reported that the B probe mutation is an influencing factor in the occurrence of false positives for RIF-R,⁸ which is inconsistent with our results. In our study, probe D, probes A or C, and multiple probes were factors affecting the occurrence of false positives in Xpert for detection of RIF-R, but probe B was not. The distribution and frequency of the mutation sites may be related to regional differences. However, we did not have detailed sequencing information and could not undertake a detailed analysis of the mutation sites. In our study, probe delay was an influencing factor, consistent with the conclusions of Berhanu et al. Their study also reported that high inconsistency rates mainly occurred when the probe had delayed ΔCT (4–4.9) relative to ΔC T ≥ 5. A low bacterial load is associated with false positives in Xpert for detection of RIF-R.^11,18 However, in our study, the difference in low bacterial load was not statistically significant.

LASSO regression, first proposed by Tibshirani in 1996, has been reported to have significant advantages in variable screening, high-dimensional data processing, and solving multicollinearity problems.^19,20 This study first screened variables through LASSO regression and then applied multi-factor logistic (backward) to build and visualize a clinical prediction model containing two variables. The model was validated using data from the training and validation cohorts, and the results confirmed that it had good discrimination and calibration capabilities. When evaluating clinical prediction models, discrimination and calibration are the most commonly used indicators; however, these two indicators cannot provide a reference for clinical decision-making. DCA is used to evaluate the clinical practicality of a model, and it reflects its actual application effect by calculating the net benefit under different thresholds. This provides an important reference for clinical decision-making.²¹ DCA analysis showed that the use of this nomogram could benefit patients when the threshold probability was set between 6% and 71%. This study confirmed that our model is suitable for distinguishing false positives in the Xpert for detection of RIF-R. This prediction model selects common clinical characteristics as predictors, which can assist doctors in identifying false positives for RIF-R in Xpert tests; moreover, these are especially suitable for promotion in areas where tuberculosis DST is not carried out.

Our study has several limitations. First, this was a retrospective study and the research was limited by existing data and information. Second, the enrollment criteria of this study were relatively strict; therefore, the number of patients included was limited. Third, this study lacks validation using external data. In the future, if prospective studies can be conducted at multiple centers, the effectiveness of the model can be further verified. Finally, there were some differences between the results of our study and those of others; further research is needed to determine whether low bacterial loads are associated with false-positive results for RIF-R using the Xpert.

Conclusion

In this study, we developed a risk prediction nomogram containing two variables based on basic demographic characteristics, clinical manifestations, and laboratory test results. This nomogram can accurately predict the occurrence of false-positives in Xpert test for RIF-R and assist doctors provide a reference for disease management and treatment plan formulation, especially in areas where tuberculosis pDST has not been performed.