Deep Learning Models for Predicting the Recurrence of Idiopathic Granulomatous Mastitis

Introduction

Idiopathic granulomatous mastitis (IGM) is a rare chronic inflammatory breast disease that presents significant challenges in diagnosis and treatment. This condition is characterized by the presence of granulomas within the breast tissue, leading to severe discomfort and complications for patients.^1–3 The etiology of IGM remains poorly understood, with potential links to autoimmune reactions, infections, and hormonal factors being explored.^4,5 Despite its rarity, the recurrence of IGM poses a significant clinical problem, necessitating effective prediction methods to improve patient management and treatment outcomes.

Previous studies have employed various combined indicators or traditional statistical methods to predict the recurrence of IGM, with mixed results. Ciftci used the serum albumin to globulin ratio (AGR) as a predictive factor and developed an AGR-based scoring system, where a low AGR was identified as a risk factor for recurrence.⁶ Traditional methods, such as logistic regression, have provided foundational understanding but often fail to capture the complex, non-linear relationships within patient data. For instance, Lermi et al used logistic regression models to predict recurrence and found that surgical intervention and the presence of abscesses increased the recurrence rate in IGM treatment.⁷ However, the authors did not assess the model’s accuracy in prediction, highlighting the need for more sophisticated techniques.

In recent years, machine learning models such as random forests and neural networks have shown promise in medical predictive analytics due to their ability to handle large, complex datasets and uncover hidden patterns, thus offering superior predictive capabilities.^8–10 Jin et al¹¹ conducted a study applying random forest models to predict the recurrence of various breast diseases, demonstrating that their random forest model outperformed traditional logistic regression methods. However, the application of neural networks in this context has been relatively unexplored.

This study aims to fill this gap by evaluating and comparing the performance of logistic regression, random forest, and neural network models in predicting the recurrence of IGM using comprehensive patient data. By leveraging the strengths of these advanced machine learning models, we seek to provide a more accurate and reliable prediction tool for clinicians managing IGM, ultimately improving patient outcomes and reducing the burden of recurrent disease.

Materials and Methods

Study Design and Data Collection

A retrospective analysis was conducted on 212 patients diagnosed with idiopathic granulomatous mastitis (IGM) at Third Hospital of Mianyang between January 2020 and June 2022. The data collection process involved gathering comprehensive clinical information for each patient, including serological markers, basic demographic details, and treatment history. This comprehensive dataset was then used to develop and evaluate various predictive models for IGM recurrence. Inclusion criteria comprised (1) Pathologically confirmed diagnosis of idiopathic granulomatous mastitis, (2) Age over 18 years at the time of diagnosis, and (3) No diagnosis of malignant disease, while exclusion criteria included (1) Later diagnosed with tuberculous mastitis, (2) Later diagnosed with Wegener’s granulomatosis, (3) Male patients, (4) Insufficient follow-up information, and (5) Missing critical data. This study was conducted in accordance with the principles outlined in the Declaration of Helsinki and approved by the Institutional Review Board of the Ethics Committee of Third Hospital of Mianyang. Patient confidentiality and data privacy were strictly maintained throughout the study process.

Outcome Definition and Recurrence Monitoring

Definition of Recurrence

Recurrence of idiopathic granulomatous mastitis (IGM) was defined as the reappearance of clinical symptoms or imaging findings indicative of active disease following an initial resolution post-treatment. Recurrence was confirmed through a combination of clinical examination, imaging studies (including ultrasound and/or magnetic resonance imaging), and, if necessary, histopathological analysis of tissue biopsies.

Monitoring Protocol

All patients included in the study underwent a standardized follow-up protocol to monitor for recurrence. First 6 Months Post-Treatment: Patients were evaluated at 1-month intervals to closely monitor the early post-treatment phase. Each follow-up visit included a detailed clinical examination focusing on signs of inflammation (eg, erythema, tenderness, induration, or discharge), as well as a high-resolution breast ultrasound to detect subclinical disease. Laboratory tests, including inflammatory markers such as erythrocyte sedimentation rate (ESR) and C-reactive protein (CRP), were performed if clinical or imaging findings raised suspicion of recurrence; 6–12 Months Post-Treatment: Patients were evaluated every 3 months. Clinical examination and breast ultrasound remained the mainstay of monitoring during this period. For patients with a history of higher disease severity or complex treatment courses, additional imaging (eg, MRI) was performed at 6 months; Beyond 12 Months: Follow-up visits were conducted at 6-month intervals for up to 24 months post-treatment to assess long-term outcomes. Clinical evaluation and breast ultrasound were performed at each visit, with imaging tailored based on clinical findings and risk factors for recurrence.

Data Preprocessing

To ensure all numerical features contributed equally to the model and to avoid dominance by variables with larger numerical ranges, continuous variables such as serological markers and imaging measurements were normalized to a common scale. This normalization step was crucial for maintaining consistency across features and enhancing model performance. Missing data, which accounted for less than 5% of the dataset, were addressed using imputation methods to minimize data loss and maintain dataset integrity. For continuous variables, missing values were replaced with the mean value calculated from the training dataset, ensuring the imputed data represented the central tendency of the feature. For categorical variables, missing entries were filled with the most frequently occurring value (mode) to preserve the original distribution of the data. To manage class imbalance between recurrent and non-recurrent cases, which could bias the model toward the majority class, the synthetic minority oversampling technique (SMOTE) was applied. This method generated synthetic samples for the minority class (recurrent cases) by creating new samples based on interpolations between existing samples within the same class. Importantly, SMOTE was applied only to the training dataset to prevent information leakage and ensure the evaluation of model performance on the testing dataset remained unbiased. These preprocessing steps collectively ensured that the dataset was well-prepared, balanced, and consistent for training, validation, and testing, supporting the development of robust predictive models.

Dataset Splitting

The Dataset Was Split Into Three Subsets to Ensure a Robust Evaluation of Model Performance

Training Set (70%): This subset (148 patients) was used to train the models by fitting the data to the algorithms and optimizing their weights.

Validation Set (15%): This subset (32 patients) was used for hyperparameter tuning and to validate model performance during training, ensuring that the models did not overfit to the training data.

Testing Set (15%): This subset (32 patients) was reserved for final evaluation to assess the generalization and predictive accuracy of the models on unseen data.

Model Training

Logistic regression, random forest, and neural networks were used as predictive models in this study.^12,13 Each model was trained using the training dataset. During training, hyperparameter tuning was conducted using the validation set to optimize model parameters and ensure the models were well-calibrated. A grid search approach combined with cross-validation was employed to systematically explore the best combination of hyperparameters for each model, minimizing the risk of overfitting or underfitting.

Model Evaluation

After model training and tuning, the final performance of each model was evaluated using the testing dataset. This independent evaluation allowed for an unbiased assessment of the models’ ability to predict IGM recurrence. Performance metrics such as accuracy, sensitivity, specificity, F1-score, and area under the ROC curve (AUC) were calculated based on the predictions made on the testing set. The following metrics were used to assess the models’ ability to predict IGM recurrence:

Accuracy: The proportion of correctly predicted instances (both recurrent and non-recurrent) out of the total instances.

Sensitivity (Recall): The ability of the model to correctly identify patients with recurrence (true positive rate).

Specificity: The Ability of the Model to Correctly Identify Patients Without Recurrence (True Negative Rate)

Area Under the ROC Curve (AUC): A measure of the model’s ability to distinguish between recurrent and non-recurrent cases, with values closer to 1 indicating better performance.

Feature Importance Analysis

After training, feature importance analysis was conducted to identify which variables had the most significant impact on the prediction of IGM recurrence. This analysis provided insights into the underlying factors contributing to recurrence, helping to enhance the interpretability of the models.

By systematically evaluating and comparing the performance of logistic regression, random forest, and neural network models, this study aimed to identify the most effective approach for predicting IGM recurrence. The ultimate goal was to develop a reliable prediction tool that could aid clinicians in improving patient management and treatment outcomes for those suffering from IGM.

Results

Baseline Information Table for Idiopathic Granulomatous Mastitis Patients in Training Group and Validation Group

A total of 212 patients with idiopathic granulomatous mastitis receiving treatment in the hospital were randomly assigned to training, validation and test groups in a 0.7: 0.15: 0.15 ratio, with 148 patients in the training group, 32 patients in the validation group and 32 patients in the test group. The inclusion and exclusion process is shown in Figure 1. Among the patients in the training group, there were 89 patients over 30 years old, accounting for 60.1%. There were 49 patients with an ESR greater than 20 mm/hour, accounting for 33.1%, and 42 patients with a CRP greater than 5 mg/dL, accounting for 28.4%. In terms of the initial treatment regimen, there were 71 patients undergoing surgery, 40 patients receiving methotrexate, 50 patients receiving azathioprine, and 100 patients receiving antibiotics. The number of patients who eventually relapsed was 25, accounting for 16.7%. There were no statistically significant differences in various variables between the three groups (p > 0.05). Detailed data for the three groups are shown in Table 1.

Table 1 Baseline Characteristics of Patients with Idiopathic Granulomatous Mastitis (IGM) in the Training and Validation Cohorts (N = 212)

Figure 1 Flow chart for inclusion and exclusion of patients with idiopathic granulomatous mastitis.

Multivariate and Univariate Logistic Regression Analysis of Patients with Idiopathic Granulomatous Mastitis (IGM) in the Training Cohort

Univariate regression analysis showed that a history of oral contraceptive use, smoking, and surgical treatment were statistically significant (P < 0.05). These variables were included in the multivariate regression model. Multivariate Cox regression analysis indicated that a history of oral contraceptive use (HR = 1.246 [1.177–1.521], p = 0.031), smoking (HR = 1.649 [1.388–2.294], p < 0.001), and surgery (HR = 1.518 [1.408–1.931], p < 0.001) were significant (Table 2).

Table 2 Multivariate and Univariate Logistic Regression Analysis of Patients with Idiopathic Granulomatous Mastitis (IGM) in the Training Cohort

Model Characteristics of Different Predictors of Incisional Infections and the ROC Curve

Three variables derived from the multivariate logistics regression were used to construct a logistics model. The area under the curve (AUC) for the training cohort was 0.837, while the AUC for the validation cohort was 0.725 and 0.829 for test cohort (Figure 2). After multiple automated parameter adjustments using the random forest model, an optimal balance between performance on the training set and the test set was sought. Ultimately, 250 decision trees were selected, with a maximum depth of 12 and a minimum sample requirement of 2 for each split. The five most important variables in the model were identified as smoking, surgery, history of oral contraceptive use, and trauma to the breast, with importance coefficients of 0.25, 0.17, 0.13, 0.08, and 0.05, respectively (Figure 3A). ROC curve analysis showed an AUC of 0.797 for the training cohort,0.755 for the validation cohort and 0.793 for test cohort (Figure 3B).

Figure 2 Performance of the Logistics Regression Model for ROC curve.

Figure 3 ROC curves of the Random Forest model on the training, validation and test sets. (A) represents the five most important variables, and (B) represents the ROC curve of the model on the training, validation and test sets.

For the deep learning model architecture, it starts with the input layer, which feeds into the first hidden layer consisting of 108 neurons. From this layer, the model splits into a main path and a side path. The main path extends to the second hidden layer with 96 neurons, while the side path leads to the third hidden layer with 72 neurons. These two paths converge at the fourth hidden layer, which contains 48 neurons, and then connect to the output layer. This structure illustrates the combination of sequential and parallel processing paths, enhancing the model’s ability to perform complex feature extraction and decision-making processes. The model architecture is shown in Figure 4A, with AUC values for the training, d validation and test sets of 0.938, 0.880 and 0.913, respectively (Figure 4B).

Figure 4 Performance of the Deep Learning Model. A represents the model architecture diagram, and B represents the ROC curve of the model on the training, validation and test sets.

Performance of Multiple Models

The overall best-performing model was the deep learning model, with F1-scores of 0.901, 0.871 and 0.894 on the training, validation and test sets, respectively, and AUCs of 0.938, 0.880 and 0.913, respectively. Additionally, the random forest model performed relatively poorly, with ROC-AUCs of 00.797, 0.755 and 0.793 for the training, validation and test sets, respectively (Table 3).

Table 3 Evaluation Indicators for Each Model

Discussion

The results of this study indicate that machine learning models, particularly neural networks, outperform traditional statistical methods such as logistic regression in predicting the recurrence of idiopathic granulomatous mastitis (IGM). The superior performance of neural networks can be attributed to their ability to capture complex nonlinear relationships in patient data, which are often overlooked by simpler models. Additionally, traditional models suffer from lower accuracy and the general inefficiency of conventional machine learning models. This finding aligns with the latest advancements in medical predictive analytics, where machine learning methods have been shown to significantly improve prediction accuracy and reliability.

Previous studies on IGM recurrence prediction mainly employed traditional statistical methods with varying degrees of success.^6,7,14,15 For instance, a 10-year study by Co et al¹⁶ using a multi-center clinical database employed a logistic regression model to identify risk factors for IGM recurrence. It pointed out that smoking and the isolation of Corynebacterium were independent risk factors for disease recurrence (p < 0.05). However, their study did not evaluate the predictive accuracy of the model, leaving a gap in understanding the true efficacy of logistic regression in this context. In contrast, our study not only confirmed the importance of these factors but also demonstrated that while logistic regression is useful, it performs poorly compared to advanced machine learning models. The AUC of the logistic regression model in our study was 0.828, indicating moderate predictive ability.

Previous research also explored random forest models.^17–20 Jin et al’s¹¹ clinical trial showed that the random forest model outperformed logistic regression in predicting breast disease recurrence, with logistic regression’s AUC at 0.653 and random forest’s AUC at 0.829, and relatively high in the external training set at 0.779. Contrary to this study, our results showed that the performance of the random forest model did not surpass that of logistic regression, with the random forest AUC at 0.787. This discrepancy may be due to suboptimal parameter tuning; however, the top five variable importance rankings included three variables from the logistic regression model, indicating that variables such as history of oral contraceptive use, smoking, and surgical treatment are crucial for predicting IGM recurrence. Due to the ensemble nature of random forests, they combine multiple decision trees to enhance predictive accuracy and robustness.

However, the most significant improvement was seen in the neural network model, with an AUC of 0.921. The ability of neural networks to model complex interactions and their flexibility in handling various data types provides a distinct advantage over traditional methods and even other machine learning models.^21–24 This finding is supported by recent literature on the application of deep learning in medical diagnostics, where neural networks consistently outperform other predictive models in various clinical settings. With the widespread adoption of artificial intelligence, for doctors, upon the admission of IGM patients, entering the patient’s examination data into a deep learning system allows the model to automatically calculate the probability of recurrence and guide doctors in early intervention to reduce recurrence rates. Accurate recurrence prediction can inform treatment decisions and subsequent strategies, ultimately improving patient outcomes. For example, patients identified as high-risk for recurrence may benefit from more aggressive treatment plans or closer monitoring, while low-risk patients can avoid unnecessary interventions.

The feature importance analysis in our study highlighted key variables, such as the presence of abscesses and a history of oral contraceptive use, as significant predictors of recurrence. These findings are consistent with previous research and provide actionable insights for clinicians. The international multidisciplinary consensus on IGM (2021 edition) mentions a recurrence rate of 15.4–24.8% for GLM, with Corynebacterium infection, breast skin lesions, and differences in prolactin (PRL) levels before and after treatment being important risk factors.²⁵ Our feature importance analysis identified smoking, surgical history, and oral contraceptive use as the most significant predictors of IGM recurrence. These findings are consistent with prior studies in the field, further validating our model’s results. For example, Baron et al²⁶ demonstrated that smoking contributes to chronic inflammation, increasing the risk of breast disease recurrence. Similarly, O’Connor et al²⁷ highlighted the association between surgical intervention and increased recurrence rates, suggesting that tissue trauma and local inflammation play critical roles in disease progression. Moreover, Abbi et al²⁸ identified hormonal factors, such as oral contraceptive use, as significant contributors to IGM recurrence, which aligns with our results. These consistent findings not only enhance the interpretability of our model but also underline the clinical relevance of the identified features. Targeted interventions, such as smoking cessation programs, cautious surgical planning, and hormonal monitoring, could potentially reduce the recurrence risk in high-risk patients. Future studies are warranted to validate these findings in larger, multicenter cohorts and to explore additional biomarkers that could further improve predictive accuracy.

Despite the encouraging results, our study has some limitations. The retrospective nature of the analysis and the relatively small sample size may limit the generalizability of the findings. Future research should aim to validate these models in larger prospective cohorts and across multiple institutions to ensure broader applicability. Additionally, while neural networks demonstrated superior performance, their interpretability remains a challenge. Clinicians may be reluctant to adopt models they do not fully understand, highlighting the need to develop interpretable AI techniques that can provide insights into the decision-making processes of these complex models.

Conclusion

This study demonstrates that neural networks significantly outperform logistic regression and random forest models in predicting the recurrence of IGM. The high accuracy and robustness of neural networks highlight their potential clinical utility in improving patient management and treatment outcomes. Moreover, deep machine learning-based methods provide researchers with powerful tools for uncovering complex patterns in data, offering significant advantages in advancing studies related to idiopathic granulomatous mastitis and similar medical conditions.