Machine Learning Radiomics for Predicting Response to MR-Guided Radiotherapy in Unresectable Hepatocellular Carcinoma: A Multicenter Cohort Study

Introduction

Hepatocellular carcinoma (HCC) is the most prevalent form of primary liver cancer and poses a major global health challenge.¹ It accounts for approximately 75–85% of all liver cancers and is the fourth leading cause of cancer-related mortality worldwide.^2–4 In recent years, there has been a surge in research exploring various innovative treatment strategies for HCC, including immunotherapy, alternative therapeutic approaches, and multi-omics integration.^5,6 These research avenues have provided novel insights into tumor biology and patient stratification, enriching our understanding of HCC’s heterogeneous nature.

However, despite these advances, many HCC patients are diagnosed at an advanced stage due to the insidious onset of the disease, rendering them ineligible for surgical resection. In these cases, radiotherapy has emerged as a critical treatment option, especially when other localized therapies such as transarterial chemoembolization (TACE) or radiofrequency ablation (RFA) are not feasible.^7,8 Delivering highly conformal radiotherapy in HCC is associated with several challenges. One key challenge is that visualization of tumors and organs-at-risk (OARs) is limited during treatment setup by using computed tomography (CT)-based imaging. Additionally, the close proximity of HCC tumors to radiosensitive organs such as the pancreas, stomach, and bowel, which are affected by respiratory motion, complicates the delivery of high radiation doses without damaging these vital structures.^9,10 On exploring the toxicity of radiotherapy in unresectable HCC patients, Furuse and colleagues¹¹ reported a 33.3% incidence of grade 3/4 toxicities. The severe adverse events observed were ascites, hyperbilirubinemia, and hypoalbuminemia, and they significantly affected survival.

Magnetic resonance (MR)-guided radiotherapy has made substantial breakthroughs in radiotherapy technology by improving the precision of radiation treatment through real-time imaging.¹² Compared with CT, MR imaging (MRI) provides better soft tissue contrast, thereby offering clearer images of tumors and surrounding organs, which allows for more accurate radiation targeting while better protecting OAR.^13,14 This real-time guidance is particularly beneficial for treating moving targets such as liver tumors. This is because it allows the adjustments to be made to the treatment plan during radiotherapy based on changes in the tumor’s position or shape, thereby achieving adaptive radiotherapy.¹⁵

Radiomics, which entails extracting quantitative features from medical images at a high-throughput rate, is well recognized for detecting tumor heterogeneity invisible to the naked eye.¹⁶ Upon integration with machine learning, radiomics can process large datasets, uncovering patterns and correlations possibly valuable for predicting treatment outcomes.¹⁷ Machine learning algorithms, especially those trained on extensive datasets can identify intricate relationships between imaging features and clinical results, potentially allowing personalized treatment predictions.^18,19 This methodology can assist in identifying patients more likely to benefit from particular treatments, thus leading to more customized and effective healthcare.

This study represents the first application of machine learning-based radiomics to predict outcomes in unresectable HCC patients undergoing MR-guided radiotherapy. By combining imaging with computational analysis, our approach aims to enhance patient stratification and inform individualized treatment decisions, ultimately leading to improved clinical outcomes. This innovative methodology underscores the transformative potential of integrating radiomics and machine learning into the clinical management of HCC, offering new avenues for precision oncology.

Materials and Methods

Eligibility and Study Population

This retrospective study included 118 unresectable HCC patients who received radiotherapy at two tertiary hospitals between May 2019 and November 2023. We included patients in the study if (a) their HCC was confirmed clinically or pathologically; (b) they received MR-guided hypofractionated radiotherapy; (c) had an Eastern Cooperative Oncology Group Performance Status (ECOG PS) score of 0−1; (d) had Child A/B; and (e) had at least one measurable lesion according to the Response Evaluation Criteria in Solid Tumors (RECIST). The patients were excluded if they (a) had diffuse lesions; (b) had severe ascites or hepatic encephalopathy; or (c) exhibited a presence of other malignancies; or (d) their information was incomplete.

HCC is considered when the following conditions are met: the patient has evidence of cirrhosis along with HBV and/or HCV infection; for a hepatic lesion measuring ≥ 2 cm, one imaging modality (either CT or MRI) demonstrates typical HCC features, that is, the lesion exhibits rapid heterogeneous vascular enhancement in the arterial phase with rapid washout in the venous or delayed phase; for a hepatic lesion measuring < 2 cm, both CT and MRI must show these typical imaging characteristics; and serum AFP levels are persistently elevated (≥ 400 μg/L for at least 1 month or ≥ 200 μg/L for at least 2 months).

Patients’ informed consent was collected, and the study was approved by the Ethics Committee of The Second Affiliated Hospital of Hainan Medical University and Shandong Cancer Hospital (SDTHEC2022012021). All patient data were kept confidential and anonymous.

MR-Guided Radiotherapy

The treatment plan was established according to the recommendations of an experienced multidisciplinary team. All study participants were initially deemed unresectable and received radiotherapy.

In the Eclipse (Varian, V15.5) planning system, preprocessed CT images obtained during simulation positioning were rigidly registered and fused with T2-weighted MR simulation positioning images. Based on the MR simulation images, the gross target volume (GTV) was delineated on the CT images of 10 phases. The GTV was then combined and the internal target volume (ITV) was created, which was subsequently expanded by 5 mm in all three dimensions to create the planning target volume (PTV). The median radiation dose was 50 Gy (40–60 Gy). OARs included kidneys, stomach, small intestine, spinal cord within the scanning range, healthy part of the liver, heart, duodenum, and lungs. The treatment plans were completed using the Monaco (Elekta, V5.40.02) system, with all plans ensuring at least 95% PTV.

Once the patient was positioned on the operation table, the radiotherapy technician performed an online MR image scan, which was then rigidly registered with the planned CT images in the Monaco system. All study patients used the Unity system to implement the Adapt to Position online adaptive workflow.

Follow-up

Radiotherapy’s effectiveness was monitored by performing CT/MRI scans every 2−3 months. Based on RECIST 1.1 guidelines, responses were classified into complete response (CR), partial response (PR), stable disease (SD), and progressive disease (PD). Progression-free survival (PFS) was defined as the time from radiotherapy initiation to PD or death. Overall survival (OS) was measured from radiotherapy initiation until death or the last follow-up.

Extraction of Radiomics Features and Radiomics Score

Before feature extraction, all CT images were preprocessed to ensure consistency and accuracy in subsequent analyses. Specifically, images were resampled using linear interpolation to a standardized voxel size of 1 × 1×1 mm³. The intensity values were discretized using a bin width of 25 hounsfield Units, and Laplacian of Gaussian (LoG) filters with kernel sizes of 1, 3, and 5 were applied to capture multi-scale texture features. In total, 1130 radiomics features were extracted from the GTV in each patient’s CT scan by using the 3D Slicer software. The features were subsequently standardized through Z-score normalization. Later, Lasso regression was employed to identify radiomics features relevant to the response. The radiomics score (Rad-score) was calculated by taking each selected radiomics feature and multiplying it with its corresponding coefficient and performing a summation of these products.

Unsupervised Clustering of Radiomics-Based HCC Subtypes

To classify HCC into various subtypes on the basis of radiomics features, the K-means clustering algorithm was used. By calculating the silhouette coefficient and using the elbow method to evaluate clustering performance, the optimal number of clusters was determined. The principal component analysis (PCA) was performed to validate clustering outcomes. Finally, the χ²-test was conducted for comparing the clinical baseline between the two groups. Additionally, the response, PFS, and OS were compared between the two subgroups.

Machine Learning Algorithms

The study cohort was derived from Shandong Cancer Hospital, where patients were randomly divided into two groups in a 1:1 ratio: the Training cohort (n=52) and the Internal validation cohort (n=51). Additionally, data from The Second Affiliated Hospital of Hainan Medical University served as an External validation cohort (n=15). The training cohort was used to identify predictive features, develop algorithms, and perform validation. Nine machine learning models were built to assess the response to radiotherapy, namely light gradient boosting machine (LightGBM), K-nearest neighbor (KNN), logistic regression (LR), decision tree (DT), random forest (RF), elastic network (Enet), eXtreme Gradient Boosting (XGBoost), regularized support vector machine (RSVM), and multi-layer perceptron (MLP), with 5-fold cross-validation applied to robustly evaluate the models built.

Model Evaluation

In the validation cohort, the stability of the nine models was evaluated by comparing metrics such as receiver operating characteristic-area under the curve (ROC-AUC), accuracy, precision, kappa statistic, specificity, Matthews correlation coefficient (MCC), recall, F1 score, and using decision curve analysis (DCA) curves and calibration curves.

Partial dependence plot (PDP), SHapley Additive exPlanations (SHAP) values, and feature importance plot were used to assess the significance of features and understand their impact on model predictions.

Statistical Analysis

Categorical variables were analyzed using the chi-square test to ensure a thorough evaluation. To identify predictive factors for response, univariate and multivariate logistic regressions were performed. Kaplan–Meier curves were used to assess OS and PFS, with the Log rank test providing further validation. All analyses were performed using R software version 3.3.2, considering a two-sided P < 0.05 as statistically significant.

Results

Patient Characteristics

The103 patients from Shandong Cancer Hospital (response group, 56; non-response group, 47) were included in the study. Of all patients, those aged ≥ 60 years and with Child B classification, BCLC stage A, portal vein tumor thrombosis (PVTT), lymph node metastasis, and extrahepatic metastases accounted for 40.8%, 6.8%, 55.3%, 24.3%, 3.9%, and 10.7%, respectively, of the total population. Furthermore, the two groups were significantly different in terms of PVTT (P = 0.005), BCLC stage (P = 0.023), and lymph node metastasis (P = 0.040, Table 1).

Table 1 Clinical Characteristics of HCC Patients Stratification by Response

PFS, OS, and Response

Among the 103 patients who received MR-guided hypofractionated radiotherapy, the number of HCC patients achieving CR, PR, SD, and PD was 13 (12.6%), 43 (41.7%), 41 (39.8%), and 6 (5.8%), respectively. The overall response rate (ORR), median PFS (mPFS, Figure 1A), and median OS (mOS, Figure 1B) were 54.4%, 21.7 months, and 40.7 months, respectively. The response group had significantly longer mPFS (36.8 vs 13.6 months, P = 0.047, Figure 1C) and mOS (40.7 vs 31.3 months, P = 0.0051, Figure 1D) than the non-response group.

Figure 1 PFS (A) and OS (B) curves for the entire cohort are presented, derived using the Kaplan–Meier method with significance determined by the Log rank test (p < 0.05 considered significant). Comparison of PFS (C) and OS (D) between the response and non-response groups, indicating that the response group had significantly longer survival durations. Abbreviations: PFS, progression-free survival; OS, overall survival.

Logistic Regression Analysis of Response-Affecting Clinical Factors

The univariate logistic analysis revealed that the BCLC stage (P = 0.008), tumor size (P = 0.020), and PVTT (P = 0.004) significantly influenced the patient’s response. However, no independent prognostic factor was identified using the multivariate logistic analysis (Table 2).

Table 2 Logistic Regression Analysis of Clinical Factors Affecting Response

Treatment-Related Adverse Events

No patient experienced Grade 3/4 adverse events (Table 3). The most common Grade 1/2 adverse events observed were bone marrow suppression (20.4%), abdominal pain (16.5%), nausea (15.5%), and abdominal bloating (13.6%).

Table 3 Treatment-Related Adverse Events

Extraction of Radiomics Features

Through Lasso regression, 7 features from 1130 radiomic features were selected (Figures 2 and 3A). The selected features and their coefficients were as follows: original_shape_Maximum3DDiameter (−0.1954), wavelet.LHH_glszm_GrayLevelNonUniformityNormalized (0.0241), wavelet.HLL_firstorder_Median (−0.0428),

Figure 2 The plot displays the variation in variable coefficients as the regularization parameter (λ) changes, illustrating how each coefficient is shrunk toward zero (A) The cross-validation curve shows the process for selecting the optimal λ in the Lasso regression model (B).

Figure 3 Heatmap displaying the expression levels of the 7 selected radiomics features between responders and non-responders (A) Heatmap illustrating both radiomics and key clinical features across the two radiomics-based HCC subtypes identified by the unsupervised clustering algorithm (B). Abbreviations: HCC, hepatocellular carcinoma; ORR, overall response rate; PVTT, portal vein tumor thrombus; BCLC, Barcelona Clinic Liver Cancer; ALP, alkaline phosphatase; ALT, alanine aminotransferase; AST, aspartate aminotransferase; AFP, alpha-fetoprotein; PLT, platelet; TBIL, total bilirubin; Hb, hemoglobin; Neu, neutrophil; WBC, white blood cell; ECOG, eastern cooperative oncology group; HBV, hepatitis b virus; BMI, body mass index.

wavelet.HHL_glszm_GrayLevelVariance (−0.0196),

wavelet.HHH_firstorder_Skewness (0.1775),

wavelet.HHH_glcm_MaximumProbability (0.0670),

and wavelet.LLL_ngtdm_Coarseness (0.0204).

The box plot indicates a significantly higher Rad-score in the responder group when compared to that in the non-responder group (P < 0.001, Supplementary Figure 1)

Identification of Radiomics-Based HCC Subtypes

Based on the 7 identified radiomics features, the 103 radiotherapy-treated HCC patients were distinctly clustered into two subtypes. The PCA confirmed that the two subtypes were significantly different and fully distinguishable (Supplementary Figure 2). Differences in aspartate aminotransferase (AST, P = 0.040), BCLC stage (P = 0.004), tumor number (P = 0.011), tumor size (P = 0.002), and target therapy (P = 0.017, Supplementary Table 1 and Figure 3B) were noted between the two subtypes. Subtype 1 had a significantly higher response than subtype 2 (59.8% vs 25.0%, P = 0.003, Figure 4A). In subtype 1, the proportion of patients exhibiting CR, PR, SD, and PD was 14.9%, 44.8%, 35.6%, and 4.6%, respectively. In Subtype 2, the proportion was 0%, 25.0%, 62.5%, and 12.5% (P = 0.010, Figure 4B). Additionally, subtype 1 had a longer mPFS (36.8 vs 11.1 months, P = 0.013, Figure 4C) and mOS (40.7 vs 28.4 months, P = 0.017, Figure 4D) than subtype 2.

Figure 4 Comparison of ORR between Subgroup 1 and Subgroup 2 (A) Comparison of CR, PR, SD, and PD between Subgroup 1 and Subgroup 2 (B) Kaplan–Meier curves showing significantly longer PFS in Subgroup 1 compared to Subgroup 2 (p = 0.013, Log rank test) (C) Kaplan–Meier curves demonstrating significantly longer OS in Subgroup 1 than in Subgroup 2 (p = 0.017, Log rank test) (D). Abbreviations: ORR, overall response rate; CR, complete response; PR, partial response; SD, stable disease; PD, progressive disease; PFS, progression-free survival; OS, overall survival.

Machine Learning Model Evaluation

Supplementary Table 2 shows that the baseline characteristics of the training, internal validation, and external validation cohorts are comparable, with no significant differences observed. In the training cohort, clinical baselines and Rad-score were incorporated into all the nine machine learning models for the analysis. In the internal validation, the nine models were compared for accuracy, kappa, specificity, MCC, precision, recall, F1, and AUC (Table 4). Additionally, the ROC (Figure 5), calibration curves (Supplementary Figure 3A), and DCA curves (Supplementary Figure 3B) were plotted for each model. According to the results, the MLP model performed the best, with an ROC-AUC of 0.804. The DCA and calibration curves both exhibited good predictive performance. In the external validation, The MLP model also demonstrated good predictive performance, achieving a ROC-AUC of 0.842 (Supplementary Figure 4).

Table 4 Performance Comparison of Machine Learning Models

Figure 5 The ROC curves for predicting response based on 9 different models in the internal validation. The models include decision tree (DT), elastic network (Enet), K-nearest neighbor (KNN), light gradient boosting machine (LightGBM), multi-layer perceptron (MLP), random forest (RF), regularized support vector machine (RSVM), and eXtreme Gradient Boosting (XGBoost).

Model Interpretation

The feature importance plot confirmed that the Rad-score was the most critical factor influencing the response. The next top five features in terms of importance were AST, tumor size, cirrhosis, and immunotherapy (Supplementary Figure 5). According to the PDP (Supplementary Figure 6) and SHAP (Supplementary Figure 7) plots, a higher Rad-score, lower AST, smaller tumor diameter, absence of cirrhosis, and the inclusion of immunotherapy were closely associated with a positive response.

Clinical Application of MLP Radiomics Models

The MLP radiomics models were found to be effective in predicting the outcome of radiotherapy in unresectable HCC patients (Supplementary Figure 8). By analyzing radiomics features based on GTV by using our machine learning algorithms, these models could assess positive responses to radiotherapy. Unresectable HCC patients predicted to respond well to radiotherapy are advised to proceed with this treatment. Conversely, alternative treatments such as TACE or RFA might be more appropriate in a patient expected to respond poorly. These models offer remarkable value in refining treatment plans and aiding clinical decisions.

Discussion

In our previous studies, as a local treatment modality, CT-guided radiotherapy was found to enhance local control in HCC patients.^20–22 However, the complex anatomical structure of the liver, along with changes in the tumor position and respiratory motion, considerably affects radiotherapy outcomes.²³ MR-guided radiotherapy is highly promising for liver cancer treatment because of advantages such as superior soft tissue contrast, real-time imaging guidance, and high adaptability. This modality is particularly beneficial in scenarios requiring highly precise radiotherapy.²⁴ Exploring the efficacy and safety of MR-guided radiotherapy for HCC is crucial.

Predicting HCC patients likely to benefit from radiotherapy is crucial for improving treatment outcomes.²⁵ However, no study has used machine radiomics to predict the HCC response to MR-guided radiotherapy. This study evaluated the effectiveness and safety of MR-guided radiotherapy for HCC and developed a predictive machine learning radiomics model.

Radiotherapy is effective against unresectable HCC. A Phase II study in unresectable HCC patients²⁶ confirmed that TACE combined with radiotherapy resulted in a longer mOS than TACE alone (13 vs 7 months, P < 0.05). In a retrospective study on 49 hCC patients who received stereotactic body radiotherapy (SBRT),²³ 1- and 2-year OS rates were 67% and 62%, respectively, and PFS rates were 53% and 38%. In a related study, Wang et al²⁷ reported that HCC patients with PVTT who were treated with intensity-modulated radiotherapy, atezolizumab, and bevacizumab achieved an ORR of 76.6%, mPFS of 8.0 months, and mOS of 9.8 months. Our previous study also confirmed that radiotherapy can benefit HCC patients. The mOS of HCC patients with a tumor diameter of ≥5 cm who received radiotherapy was 14.9 months.²⁸ All these studies involved CT-guided radiotherapy. Currently, no study has reported the efficacy and safety of MR-guided radiotherapy for HCC. In our study, the ORR, mPFS, and mOS of HCC patients who received MR-guided hypofractionated radiotherapy were 54.4%, 21.7 months, and 40.7 months, respectively. Additionally, no patient experienced Grade 3/4 treatment-related adverse events. These results collectively demonstrate the superiority of MR-guided radiotherapy in HCC treatment.

Radiotherapy has improved HCC prognosis.²⁹ Therefore, predicting the efficacy of radiotherapy is an urgent requirement. Radiomics is now extensively used to evaluate the biological features, treatment outcomes, and survival of HCC. CT is considered among the most frequently used diagnostic methods for HCC, with favorable cost-effectiveness.³⁰ Using Rad-scores and clinical features, Wang et al³¹ constructed nomograms for predicting OS in SBRT-treated advanced HCC patients. Their model achieved AUCs of 0.76, 0.79, and 0.84 for the 6-, 12-, and 18-month survival predictions, respectively. Huang et al³² retrospectively analyzed 131 patients by constructing nomograms based on the radiomics features and clinical characteristics for predicting OS of radiotherapy-treated HCC patients with PVTT. The model achieved a concordance index (C-index) of 0.73 and an AUC of 0.71. Park et al³³ incorporated data from 409 patients to develop a clinical model, radiomics model, and combination of both features model (CCR model). When evaluating treatment response, the radiomics, clinical, and CCR models achieved AUCs of 0.647, 0.729, and 0.759, respectively. Among the three models, the performance of the CCR model was the best. Although these predictive models assessed patient survival, their predictive performance remained suboptimal.

By applying the K-means clustering algorithm, two HCC subtypes were identified based on radiomics. Patients with subtype 1 hCC exhibited superior ORR, PFS, and OS compared to those with subtype 2 hCC. This result suggests substantial radiomic variations between responders and non-responders. To substantiate that these findings were robust and reliable, nine machine learning algorithms were built and employed along with repeated 5-fold cross-validation. Multiple machine learning models offer higher predictive accuracy and greater flexibility than single models. The models can integrate various features and data sources, capturing complex nonlinear relationships to offer a more comprehensive analysis.^34,35 Additionally, the models built are adaptive, which allowed for continuous optimization with new data, thereby reducing the risk of overfitting and improving generalization capabilities. Because of these characteristics, machine learning models are superior to single models when used for analyzing complex datasets and multidimensional problems.^36–38 Using machine learning models based on radiomic features and clinical characteristics, we developed nine models for assessing the patient treatment response. The MLP model exhibited the best performance, with an ROC-AUC of 0.804. Thus, integrating radiomic features, clinical characteristics, and machine learning offers a promising approach for predicting the radiotherapy response noninvasively. The incorporation of interpretability methods, such as SHAP values, provided valuable insights into the mode’s decision-making process. These methods revealed the relative contributions of key features—especially the radiomics score, AST, and tumor size—thereby enhancing the model’s transparency and aiding clinicians in understanding the rationale behind the predicted results.

While our study reported no serious adverse events, this conclusion must be interpreted with caution. The current follow-up period, although sufficient for assessing short- to mid-term toxicity, may not capture long-term adverse effects. Therefore, extended follow-up is necessary to fully assess the long-term safety of MR-guided radiotherapy.

In our study, several baseline variables—including AST levels, BCLC stage, the presence of PVTT, and tumor size—showed differences between the response and non-response groups. Although AST did not reach conventional statistical significance (P = 0.095), its observed trend suggests that liver function may influence radiotherapy outcomes.³⁹ Moreover, the differences in BCLC stage and the presence of PVTT are consistent with their well-established prognostic roles in hepatocellular carcinoma, reinforcing their potential utility in predicting treatment response. Tumor size also remains a critical determinant, as larger tumors are generally associated with poorer outcomes.⁴⁰ Together, these findings underscore the clinical relevance of these variables and support their inclusion in predictive models. Future studies with larger sample sizes and standardized protocols will be essential to further validate these trends and clarify the underlying mechanisms.

This study initially revealed that MR-guided radiotherapy is effective and safe for HCC. Additionally, the model developed enhanced the rational application of radiotherapy in clinical practice, thereby providing substantial guidance for clinical decision-making, particularly in patient selection for radiotherapy. For patients expected to respond effectively, radiotherapy can be performed to improve local control in inoperable HCC. For patients anticipated not to benefit from radiotherapy, switching to other treatments early is advisable.

In our study, we acknowledge that the retrospective design and small sample size are key limitations. In particular, the limited sample size may restrict the robustness and generalizability of our predictive model. First, a major difficulty in expanding the sample size is the heterogeneity in data collection across different centers, including variations in imaging protocols, patient management, and data quality. To address this, we propose establishing multicenter collaborations with standardized protocols for data acquisition, image processing, and treatment procedures. This will help harmonize data across institutions and reduce variability in radiomics feature extraction. Second, future studies will be designed prospectively, allowing for real-time data collection and quality control, thereby minimizing retrospective biases. Additionally, advanced statistical methods, such as data harmonization and normalization techniques, will be applied to adjust for potential inter-center variability. These measures will collectively enhance the robustness of our predictive models and facilitate the translation of our findings into clinical practice.

Conclusion

MR-guided radiotherapy is effective and safe for HCC. Our developed machine learning radiomics model accurately predicts the response of inoperable HCC patients to radiotherapy, offering new perspectives to improve treatment plans and inform clinical decision-making.