A Nomogram for Predicting the Effectiveness of Consultations on Multi-Drug Resistant Infections: An Exploration for Clinical Pharmacy Services

Introduction

The increasing frequency of antimicrobial resistance (AMR) poses a serious threat to global public health security and is a huge challenge facing the world.¹ The occurrence of AMR can make antibiotics ineffective or inefficient. Subsequently, higher doses of antibiotics are needed or patients have longer treatment courses, even die. According to statistics, there were approximately 4.95 million AMR-related deaths.² Figures from the UK government suggest that AMR will cause estimated 10 million deaths per year by 2050, with cumulative losses of approximately £100 trillion.³ Multi-Drug resistance (MDR) is a more serious condition in AMR and refers to the resistance to insensitivity (both resistant and mediated) to three or more classes of antimicrobial drugs within the antimicrobial spectrum, which is difficult to treat, has limited drug options and is more likely to result in patient death. The incidence of MDR has been on a rapid global rise in recent years and is a pressing social issue worldwide.⁴

The occurrence of AMR is a natural process and the mechanism of bacterial survival. However, the irrational use of antibiotics can accelerate the process.⁵ The appropriate use of antibiotics can effectively slow down the onset of AMR, shorten the length of hospital stay, reduce the treatment cost and maximize the cure rate of patients, which is the key to solve AMR. To combat AMR, antimicrobial stewardship (AMS) programmes have been introduced in several countries around the world to monitor antibiotic use and promote the rational use of antibiotics to avoid AMR.⁶ AMS programmes are a multidisciplinary mode of care consisting of professional clinical pharmacists, infectious disease experts and clinical microbiologists.^7,8 This mode of care is well suited to the treatment of MDR and is effective in improving the cure rate of MDR. In particular, AMS programmes emphasize the importance of clinical pharmacists. Studies have shown that clinical pharmacists can improve the outcome of multi-drug resistant organisms (MDRO) infections by participating in the anti-infection process through consultations and making individualized recommendations for drug therapy.^9–11

In China, the shift of pharmacists towards clinical services began to be reinforced in 2011 with the Regulations on the Administration of Pharmacy in Medical Institutions, stating that the duties of clinical pharmacists include pharmacy visits, pharmacological supervision, participation in the treatment process of doctors, and medication education for patients. These responsibilities can be collectively referred to as clinical pharmacy services. Most studies have demonstrated the positive effects of clinical pharmacy services in reducing adverse drug reactions, improving medication adherence and reducing treatment costs.^12,13 The greatest advantage of the clinical pharmacist is the ability to customize specific pharmacy services based on the patient’s own state or to a particular disease, maximizing the patient’s cure rate. Therefore, it is very important to improve the level of pharmacy services provided by clinical pharmacists.

Participating in disease consultations and making pharmacy recommendations is part of the clinical pharmacy service. A clear overview of the factors influencing the effectiveness of consultations and the probability of success in advance can help clinical pharmacists to make more accurate and rational drug recommendations, which is helpful to improve the quality of pharmacy services. The nomogram is a predictive model based on the influencing factors of the disease, which can convert complex mathematical formulas into visualized graphs, making it more convenient for doctors and clinical pharmacists to predict the probability of the occurrence, cure or recurrence of the disease. It is often used in clinical practice and helps to promote personalized medicine.^14,15

Based on the global focus of MDR, this study analyzed the factors affecting the efficacy of MDR consultation, and constructed a nomogram prediction model for consultation failure, in order to provide references for clinical pharmacists participating in MDR treatment and improve the effectiveness of consultation. In addition, based on this prediction model, how to better serve clinical pharmacists was discussed. To the best of my knowledge, this study established the prediction model of MDR consultation failure for the first time, which is of great research necessity.

Methods

Study Design

The study aimed to develop a risk calculator to predict the outcome of consultation and explore ways to improve clinical pharmacy services. In the study, a retrospective research approach was employed to screen all patients diagnosed with MDRO infection and consulted by clinical pharmacists in the Affiliated Wuxi People’s Hospital of Nanjing Medical University from January 2021 to May 2023. Clinical data of septic patients were collected through the Hospital Information System (HIS). Univariate analysis and multivariate logistic regression analysis were used to identify independent risk factors for MDRO infection consultation effectiveness, and then a nomogram was constructed and validated.

Patients and Data Collection

Patients diagnosed with MDRO infection and consulted by clinical pharmacists in the Affiliated Wuxi People’s Hospital of Nanjing Medical University from January 2021 to May 2023 were included in this retrospective study. The exclusion criteria were: (1) patients were not treated with antimicrobial drugs, (2) The purpose of the consultations was not related to anti-infection, and (3) patients who died or were discharged within 3 days were excluded due to insufficient time for efficacy evaluation.

The patient’s medical record was accessed through the HIS, and the data was extracted as independent variables: (1) basic information of patients: hospitalization number, gender, age, underlying disease, surgery, (2) infection-related indicators: body temperature, infection site, white blood cell count (WBC), fast C-reactive protein (CRP), serum albumin (ALB), alanine aminotransferase (ALT), glomerular filtration rate (GFR), (3) pathogenic bacteria situation: the category and number of pathogenic bacteria, drug resistance, and (4) consultation situation: consultation pharmacist, drug plan, consultation suggestion, recommendation adoption, consultation effectiveness.

Ethical Considerations

The study was approved by the Ethical Committees of the affiliated Wuxi People’s Hospital of Nanjing Medical University (KY23102) and conducted in accordance with the 1964 Helsinki Declaration and its later amendments or comparable ethical standards. The data of the patients used in the study were anonymized, and no samples from human or animals were specially collected for this research. Informed consent was waived by our Institutional Review Board because of the retrospective nature of our study.

Consultation Intervention

The process of clinical pharmacists participating in consultation was as follows: (1) Physicians submitted the anti-infection consultation requirement through HIS, (2) After receiving the application for consultation, clinical pharmacists formulated the reasonable drug treatment plans according to the age, underlying diseases and symptoms of patients based on the consultation demands of physicians, such as the selection or adjustment of antibacterial drugs, the adjustment of antibacterial doses, and the necessity of combination drugs, etc., (3) finally, the physicians decided whether to adopt or partially adopt the clinical pharmacist’s recommendation, and the data can be obtained from HIS and included in the study.

Study Groups and Definitions

All patients in this study were consulted by clinical pharmacists, and data of those who received multiple consultations were collected based on the first consultation. Simple random sampling method was used to divide the research objects into a training set and a verification set in a ratio of 7:3, with the training set served to build the nomogram prediction model, and the validation set helped to verify the discrimination and calibration of the model. The operation process of simple random sampling method was as follows: (1) The random number for each patient was generated using the RAND function in MS EXCEL. (2) The random numbers were arranged in ascending order. (3) The training set and the validation set were obtained successively in a ratio of 7:3. The patients in the training set were divided into two groups according to the improvement of their symptoms after the consultation intervention. The clinical symptoms, vital signs, WBC, CRP and other laboratory indicators 3 days after consultation was evaluated. Patients who meet one of the following criteria were included in the effective group: (1) Body temperature decreased from hyperthermia (39.1 °C −41 °C) to below 38.5 °C, or recovered from a low fever state (37.5 °C −38.5 °C) to below 37 °C. (2) The WBC and CRP values were reduced by 30% or to the normal range. Other patients were categorized as ineffective group.

Statistical Analysis

Statistical Package for the Social Sciences (SPSS) 22.0 (IBM, Chicago, IL, USA) was used for statistical analysis of the data, and the R Programming Language (R) software (version 4.3.0, http://www.Rproject.org) was used for the establishment and verification of the nomogram. The comparison of basic information of patients in the training set and validation set was shown in Table 1. Independent sample t-test was used to compare continuous variables, which are shown in the form of means and standard deviations (SD), while categorical variables were displayed in the form of numbers and percentages by chi-square test. P<0.05 was considered statistically significant.

Table 1 Comparison of Basic Information Between the Training and Validation Set

The following steps were conducted to establish and validate a nomogram to predict the effectiveness of consultation for MDRO infections. Firstly, univariate analysis was used to initially identify the factors influencing the effectiveness of MDRO infections consultations. Then, multivariate Logistic regression analysis was used to determine the independent factors to develop the nomogram. The results were presented with odds ratios (OR) and 95% confidence intervals (CI). Next, a nomogram was constructed to predict the effectiveness of consultations with the help of R software. Finally, the obtained nomogram model was validated. Bootstrap resampling method was adopted for internal verification and external verification was carried out with Calibration method based on the verification set. Receiver operating characteristic (ROC) curve was drawn and the area under the ROC curve (AUC) and the consistency index (C-index) were used to evaluate the differentiation of the model, and the calibration curve was drawn to evaluate the accuracy of the model.

Results

Patient Characteristics

After excluding 2 patients who did not receive anti-infection treatment, 5 patients whose purpose of the consultation was not related to anti-infection, and 27 patients whose efficacy could not be assessed, 198 patients were finally included in this study (Figure 1). These patients included 130 males and 68 females, with a mean age of (66.20 ± 18.12) years, 169 (85.4%) with comorbid underlying disease and 102 (51.5%) who had received surgical treatment prior to infection. The infection sites of patients were as follows: 91 cases in lung, 34 cases in abdominal cavity, 13 cases in urinary tract, 10 cases in surgical site, 3 cases in intracranial, 7 cases in blood flow and 40 cases in multiple sites. The pathogenic bacteria of the patients were as follows: multi-drug resistant G⁺ bacteria in 78 cases (39.4%), multi-drug resistant G⁻ bacteria in 148 cases (74.7%), combined fungal infections in 25 cases (12.6%) and sepsis in 24 cases (12.1%). These patients were divided into a training set (140 cases, 98 in the effective group and 42 in the ineffective group) and a verification set (58 cases), and the differences in patient Characteristics between the two groups were compared. The results revealed that there was no significant difference in other factors except the carbapenem-resistant G⁻bacteria infection rate (Table 1), indicating the randomness and rationality of the grouping.

Figure 1 The flowchart of patient inclusion, exclusion and grouping for this study.

Univariate and Multivariate Analysis

Univariate analysis was applied to compare the differences between the effective and ineffective groups in the training set to determine the potential prognostic factors for the outcome of consultation for MDRO infections, with factors at P<0.05 being considered to be associated with consultation effectiveness. These results were presented in Table 2. The number of underlying diseases, whether surgery was performed prior to infection, body temperature, ALB level, CRP level, number of infection sites, number of pathogenic bacteria, title of pharmacist, whether advice was taken, and number of antimicrobials used were significantly different between the two groups (P<0.05) and were included in the multivariate Logistic regression analysis.

Table 2 Univariate Analysis of Patient Characteristics in Effective and Ineffective Group

The results of the multivariate Logistic regression analysis were illustrated in Table 3. The number of underlying diseases (OR=1.720, 95% CI: 1.260–2.348), whether surgery was performed prior to infection (OR=8.853, 95% CI: 2.668–29.373), ALB level (OR=0.885, 95% CI: 0.805~0.974), pharmacist title (OR=3.463, 95% CI: 1.277~9.396) and whether the recommendation was taken up (OR=0.117, 95% CI: 0.030~0.462) were identified as independent influences on the effectiveness of the consultation. Among them, to some extent, more underlying diseases, having undergone surgery before infection, the low clinical pharmacist title will increase the failure rate of consultation. Whereas a higher ALB level and adoption of the clinical pharmacist’s consultation recommendations reduced the failure rate of the consultation.

Table 3 Multivariate Logistic Regression Analysis of Influencing Factors of Consultation Effectiveness

The Construction and Validation of a Nomogram Model

Based on the results of the Logistic regression analysis, the above five independent factors were used to construct a nomogram model to predict the probability of unsuccessful consultation. All processes were performed by R software using the “rms” package. As shown in Figure 2, the risk of clinical pharmacists participating in MDR consultation failure was up to 0.9. Each risk factor in the nomogram model was assigned points by plotting a vertical line from the corresponding factor to the point axis. The total point was the sum of all points from all factors. The probability of consultation failure can be evaluated by drawing a vertical line from the total points axis to the consultation failure axis. The number of underlying diseases and the level of ALB had the greatest impact on the outcome of the consultation, with a higher number of underlying diseases and a lower level of ALB increasing the risk of consultation failure. Secondly, whether to undergo surgery before infection and whether to adopt the consultation advice of clinical pharmacists had similar effects on the efficacy of the consultation. The title of clinical pharmacist had the least effect on the effectiveness of the consultation.

Figure 2 A nomogram to predict the failure rate of MDRO infection consultation.

Internal validation and external validation were implemented to check the predictive power of the nomogram model. The nomogram was verified using the method of bootstrap resampling. The bootstrap method was applied to 1000 resamples. The ROC curves obtained were plotted as shown in the Figures 3A and 4A, and the area under the curve was calculated. The discrimination of the prediction model is determined according to the AUC. The AUC may range from 0.5 (no predictive ability) to 1.0 (complete discrimination). Specifically, the AUC value range of 0.5–0.6 is bad, 0.6–0.7 is poor, 0.7–0.8 is satisfactory, 0.8–0.9 is good, and 0.9–1.0 is excellent.¹⁶ The AUC of the training set (Figure 3A) and the verification set (Figure 4A) were 0.849 (95% CI: 0.780–0.917) and 0.761 (95% CI: 0.616–0.907) respectively, demonstrating that the model had good discrimination. The calibration of the nomogram was presented by the Hosmer–Lemeshow test. The calibration curves exhibited good overlap between the data predicted by the model and the actual data (Figure 3B and 4B), indicating that the model has good accuracy.

Figure 3 Internal validation to check the predictive power of the nomogram model in training set. (A) Receiver operating characteristic curve to evaluate the differentiation of the model. (B) The calibration curves to evaluate the accuracy of the model.

Figure 4 External validation to check the predictive power of the nomogram model in verification set. (A) Receiver operating characteristic curve to evaluate the differentiation of the model. (B) The calibration curves to evaluate the accuracy of the model.

Discussion

MDRO infections are difficult to treat clinically and are prone to epidemic in-hospital outbreaks that threaten patients’ lives. Clinical pharmacists, as part of the treatment team, play an active role in facilitating the process of patient care.^17–19 In this study, after screening by Logistic regression, we found that the number of patients’ underlying diseases, whether they had surgery prior to infection, serum albumin level, title of the consulting pharmacist and whether the recommendations were adopted were factors affecting the effectiveness of MDRO infection consultation. Further analysis of the constructed nomogram prediction model revealed that more underlying diseases, pre-infection surgery, senior pharmacist title and the failure to adopt pharmacy advice were positively associated with consultation ineffectiveness, and serum albumin level was negatively associated. The internal validation of the training set and the external validation of the validation set confirmed that the model had good accuracy, and the predicted failure risk of clinical pharmacists participating in MDRO infection consultation was in good agreement with the actual failure risk.

MDRO infection is associated with multiple factors, such as patient gender, number of underlying diseases, co-morbidities, APACHE II score, mechanical ventilation, length of stay in ICU, indentured catheter, number and duration of antimicrobial drug use were the most reported predictors of infection risk. Underlying disease ≥3, mechanical ventilation, APACHE II score ≥22, and infectious shock were key factors affecting patient prognosis.^20–23 This study found that there were significant differences in the number of underlying diseases, surgery, body temperature, ALB level, CRP level, number of infection sites, number of pathogens, titles of pharmacists, adoption of recommendations, and number of antimicrobials used between the effective group and the ineffective group, which was consistent with the risk factors of MDRO infection reported in the literature.

Patients with co-morbidities often have low body functions. Anemia patients, where the oxygen-carrying capacity of the blood is reduced and the blood volume is lowered in various degrees, resulting in reduced cellular immunity and leukocyte phagocytosis.²⁴ Patients with respiratory failure have varying degrees of pulmonary ventilation or air exchange dysfunction, and are unable to exchange gases effectively, resulting in the accumulation of respiratory secretions, which worsens the condition of patients.²⁵ The metabolism and excretion of antimicrobial drugs are affected to varying degrees in cases of liver and kidney function insufficiency, limiting the options and dosage of antibacterial drug, which leads to the poor anti-infective treatment effect.²⁶ Therefore, the more underlying diseases a patient has, the more complex the condition is after the occurrence of MDRO infection, making treatment more difficult and affecting the prognosis.

Protein binding rate has an important impact on the pharmacokinetics of antibiotics. The decrease of serum albumin level will affect the distribution and metabolism of antibiotics with high protein binding rate in the body, resulting in an increase of free drugs concentration and rapid drug clearance. As a result, the concentration of free drugs in serum is instead lower than that of normal albumin level, which affects the anti-infection efficacy.²⁷ In addition, the physiological barriers and protective mechanisms of the body are disrupted in surgical patients, predisposing them to an increased probability of MDR. Postoperative drainage tubes, arteriovenous catheters, and indwelling catheters are often placed, connecting the internal environment with the external environment, thus destroying the protective barrier of the body and hindering the effective penetration of some antibacterial drugs.²⁸ The nomogram model established in this study also confirmed that more underlying diseases, lower albumin level and pre-infection surgical history were more likely to lead to a failed MDR consultation.

The involvement of clinical pharmacists in the treatment of patients with MDRO infection can significantly improve the cure rate, shorten the course of anti-infection therapy, reduce the cost of antibacterial treatment per capita and the incidence of adverse drug reactions.^29–31 During the consultation, clinical pharmacists make individualized treatment plans by communicating with clinicians and patients, adjust the drug administration plan according to the PK-PD results, and monitor the adverse drug reactions, so as to optimize the treatment plan, improve the treatment outcome of patients and effectively utilize medical resources.³² Whether consultation advises are adopted or not is an important factor affecting the anti-infection treatment effect of patients. A multicenter cohort study based on a database of 17 hospitals in western China included 2663 infected patients who had been consulted by clinical pharmacists. The data showed that patients who adopted pharmacists’ advice had a higher effective rate in terms of their condition (81.34% vs 67.16%, P<0.001).³³ Another study of 50 clinical pharmacists’ anti-infection consultations in 17 provinces of China showed that the acceptance rate of clinical pharmacists’ consultation recommendations was 93.13%, and the effective rate of treating patients after accepting the pharmacists’ advices was 93.6%. Moreover, the adoption of the pharmacists’ recommendations could significantly improve the prognosis of patients with infectious diseases, with a OR of 2.08.³⁴ Anti-infective consultation by clinical pharmacists plays a positive role in improving the prognosis of patients with infectious diseases. This study also confirmed that failure to follow pharmaceutical recommendations was an independent risk factor for ineffective consultation.

In this study, the success rate of consultation by intermediate clinical pharmacists was significantly higher than that of senior clinical pharmacists, which may be due to the implementation of the grid-based pharmacy service model in our hospital. Clinical pharmacists with intermediate titles are also required to conduct consultations after completing standardized training. They have received systematic antibacterial training and have accumulated rich experience in anti-infection consultation work in practice. Clinical pharmacists with senior titles are more inclined to carry out consultation by directly going to clinical departments to discuss with bed doctors, while clinical pharmacists with intermediate titles are more inclined to browse patients’ medical records, sort out medication history, review relevant guidelines, and discuss treatment plans with superior pharmacists before consultation, and then discuss with bed doctors. Better preparation before consultation effectively improves treatment success rate.

Studies have shown that about 30–50% of antibiotics related prescriptions in hospitals can be further optimized,^35,36 suggesting the importance of clinical pharmacy services. The prediction model of this study can provide certain reference significance for improving clinical pharmacy services. The first is that clinical pharmacists should ensure their professional competence. Studies have shown that clinical pharmacists specializing in infectious diseases can provide more aggressive treatment plans in combating AMS compared to general clinical pharmacists.³⁷ This suggests that clinical pharmacists must have a deep knowledge base and rich clinical practice experience in order to make a more appropriate drug treatment plan. Second, clinical pharmacists must strengthen communication with clinicians. The results of this study show that the success rate of consultation can be improved when doctors follow the advice of clinical pharmacists. It has been reported that when clinical pharmacists participated more in the doctor’s diagnosis and treatment process and increased their communication with each other, physicians were more willing to adopt the clinical pharmacists’ advice. Third, the clinical pharmacist should be fully aware of the characteristics and changes in the patient’s condition and carefully review the relevant guidelines before the consultation. In this way, clinical pharmacy services are expected to be greatly improved based on the above three points.

Conclusion

In this study, a nomogram model was developed to predict the risk of consultation failure based on five factors, including the number of underlying diseases (OR=1.720, 95% CI: 1.260–2.348), pre-infection surgical history (OR=8.853, 95% CI: 2.668–29.373), serum ALB level (OR=0.885, 95% CI: 0.805~0.974), the title of clinical pharmacist (OR=3.463, 95% CI: 1.277~9.396) and whether the consultation opinion was adopted or not (OR=0.117, 95% CI: 0.030~0.462), and was shown to be good accuracy and prediction efficiency (the AUC of the training set and the verification set being of 0.849 and 0.761, which can provide a reasonable prediction of the effectiveness of MDRO infection consultations, and provide proactive interventions to improve outcomes for potentially treatment ineffective patients.

Clinical pharmacists can improve the cure rate and shorten the treatment course by providing pharmaceutical advice in the treatment of infectious patients. Many studies have demonstrated the positive role of clinical pharmacists. However, there are no studies that have elucidated the factors influencing the effectiveness of clinical pharmacists participating in MDRO infection consultation and constructed a nomogram model to predict this outcome. Using “(Nomogram) AND (Consultation) AND (Infection)” as search terms, 21 papers were retrieved in the PubMed database, but none were similar to this study. This study established for the first time a predictive model for the effectiveness of clinical pharmacists’ participation in MDRO infection consultations. According to the nomogram model, the risk predictors can be converted into specific risk scores. By calculating the sum of the scores of the risk factors for a certain patient, clinicians can accurately predict the probability of remission of the patient’s disease after receiving consultation advices. This enables them to make relevant interventions in advance, thus promoting the rational application of antibiotics and improving the prognosis of the patients. Due to the limitations of sample size and single- center research attributes, there are still some optimizations in this study. In the future, a platform for evaluating the efficacy of anti-infective consultations by clinical pharmacists and informative follow-up and a full process management mode for clinical pharmacists to participate in anti-infection consultations should be established, facilitating the collection of data and the conduct of the study, and promoting the improvement of the accuracy of the prediction model.