Machine Learning Analysis to Identify Predictive Factors of Caudal Epidural Pulse Radiofrequency in the Treatment of Coccygodynia

Introduction

ML has emerged as a phenomenon that has been in demand in many different fields in recent years. Using algorithms that can develop their own rules from the data set on hand, ML can make predictions against future uncertain situations using this existing data for learning purposes.

Consistent with the literature on other fields, the use of ML in healthcare systems is also increasing daily. Unlike traditional statistical models, ML models are generally more flexible and adaptable to various data types and tasks. Recently, studies are using the technique for different types of diseases or treatments, eg, oncological,¹ cardiovascular,² diabetes,³ infectious diseases,⁴ digestive system diseases,⁵ etc. However, pain medicine has not yet taken its place among these varying areas. ML can significantly impact pain medicine by providing more accurate diagnoses, personalized treatment plans, and more effective treatments for patients suffering from pain.

Coccygodynia is a condition characterized by pain, tenderness, or discomfort associated with the coccyx and surrounding structures, localized in the sacrococcygeal region. The most common cause is direct trauma caused by falling (50–65%), and also it may develop in women due to trauma triggered by giving birth or repetitive microtraumas.^6,7 Diagnosis is based on patient history and clinical examination.

In treating chronic coccygodynia, interventional treatments can be applied when conservative treatments such as analgesic medications, physical therapy modalities, and manipulation are insufficient. It has a close anatomical relation with the terminal end of the sympathetic chain, which it proposes as the sacrococcygeal nerve fibers and ganglion impar.⁸ Interventional methods, including nerve blocks and radiofrequency applications targeting these two structures, are essential in treatment.^6,9,10 CEPRF therapy is used for pain syndromes such as coccygodynia, post laminectomy syndrome, post herpetic neuralgia, and polyneuropathy.^9,11–13

In a study by Cho et al in a rat model, they found that CEPRF administration suppressed microglial activity and reduced nociceptive cells at the L3-S1 levels, thus demonstrating its positive effect on nociceptive and neuropathic pain.¹⁴

Although there are many factors that are hypothesized to be effective on treatment success, parameters such as the presence of permanent subluxation and prolonged symptom duration may be major factors.⁶ However, the literature information is still limited. In this study, we used the pre-treatment data of 108 patients who applied to our clinic with complaints of coccygodynia. In order to define the predictive factors, we used ML techniques explained in the following sections, and we analyzed the post-treatment success of CEPRF treatment for 6-months.

Materials and Methods

Data Definition

This is a single-center, retrospective study conducted in the University of Health Sciences, Gulhane Training and Research Hospital pain medicine department. Data were collected from the medical records of patients who were diagnosed with coccygodynia with the help of patient history, physical examination and magnetic resonance imaging in the pain medicine department between January 1, 2018, and June 1, 2022, and who received CEPRF for treatment.

Related data include sex, age, BMI (body mass index), symptom duration, coccygodynia etiology, history of coccyx fracture, presence of angular deformation, and Numerical Rating Scale (NRS) (0–10, verbal) at hospital application and at the end of the 6th month, respectively. In our clinic, demographic data and clinical parameter data of patients undergoing interventional procedures are routinely recorded prospectively. Additionally, pain severity is monitored using the NRS. Patients are re-evaluated with the NRS for treatment success at the 6th month after treatment. For this study, the prospectively collected data of the patients were retrospectively reviewed. Treatment success was defined as a ≥50% decrease in the NRS scores at the 6th month after treatment compared to baseline.

A total of 122 patients are considered for inclusion in the present study. Of the 122 patients, 14 are excluded because of a lack of follow-up data, leaving 108 patients for the final analysis.

The inclusion criteria were as follows: 18–80 years of age, presence of coccygodynia resistant to conservative treatments, such as analgesic drugs and physical therapy modalities, symptoms persisting for >3 months, NRS pain score ≥4.

The exclusion criteria are as follows: Having undergone interventional procedures such as caudal epidural steroid injection and ganglion impar block for coccygodynia within the last year, having undergone coccygectomy or anorectal or perineal surgery, pregnancy, regular opioid use, opioid addiction, drug and/or alcohol abuse, and a lack of follow-up data.

As is the routine practice for interventional procedures at our hospital, all the patients are asked to complete and sign the standard patient consent form before the procedure. The institution’s ethics committee approved the study protocol (meeting no. 2022/08; decision no. 2022-85). Permission to use data from the medical center has been obtained. Present study was conducted following the principles of the Declaration of Helsinki.

Procedure

After vascular access was established, the patient was taken to the operating room and monitored. The patient was then placed in the prone position. Antisepsis was achieved with povidone iodine, and the patient was covered in sterile fashion. Sacral hiatus was visualized from the lateral view under the guidance of fluoroscopy. Short-acting local anesthesia (3 cc of 2% lidocaine) was applied to the skin and subcutaneous tissue. A 10-cm with 10-mm active tip, 20-gauge radiofrequency cannula (RF) was inserted to the caudal epidural space under the guidance of fluoroscopy. The RF cannula was advanced into the epidural space to the S3-S4 vertebral level (Figure 1). It was then confirmed by fluoroscopy that the RF cannula was in the midline in the anteroposterior plane. Sensory stimulation (50 hz, 0.2–0.5 V) was given with the RF generator (Boston Scientific, G4™ RF Generator) and the presence of a feeling of fullness in the sacrococcygeal region was confirmed. Impedance measured ranged between 150 and 450 Ohms. No motor contraction was observed with 2 hz motor stimulation up to 2 V. Then, CEPRF cycles were applied for 240 sec with a pulse frequency of 2 hz and pulse width of 20 ms at 42°C. The patients were discharged after no complications were observed in the first hour after the treatment.

Figure 1 Lateral View: Active tip of the radiofrequency cannula (RFC) is in the epidural space (ES) between S3 and S4 levels.

To be used in the ML approach, related values are presented together with their data types and corresponding definitions in Table 1. According to the table, attributes at the first 8 rows correspond to the inputs for the model; whereas the 9th one is the output. Success of rate, the target attribute, is defined according to the improvement over the NRS levels. As the standard way of this consideration is following the 50% improvement, we used this percentage. Thus, comparing the NRS levels at admission with the ones at month 6, we refer the treatment to be successful or not.

Table 1 Statistical Definitions of Categorical Attributes

At the stage of data pre-processing, we categorized the numerical data on hand:

• Age values are grouped into 6 categorical intervals.
• Height and weight data are used to obtain BMI values, and these two attributes are discarded from the data set. As there are referenced intervals to be used concerning the BMI values in the literature, we categorize this data as is given in Table 1.
• Symptom durations are grouped starting from 12-months’ intervals and taking wider ranges for over 6 years.

After pre-processing, the p-values of the chi-square, showing the association of the attributes to the NRS at Month 6, are shown in Table 2.

Table 2 Chi-Square p-values Related to NRS at Month 6

P-values are computed based on the frequencies between related two variables. According to the chi-square matrix and evaluated p-values; there are high correlations for the inputs of symptom duration and angular deformation with the output of NRS at Month 6.

Model

ML enables computers to learn from data and entails building models that automatically spot trends and forecast events based on input data.^15,16 The objective of the supervised ML method is to develop a mapping function that can correctly forecast the intended result from novel, unseen inputs. The well-known supervised learning methods are Linear Regression, Logistic Regression (LR), Decision Trees, Random Forest, Support Vector Machines (SVM), Bayesian Network (BN), and Neural Networks (NN). In this subsection, the models used in the research are briefly explained.

The C5.0 algorithm is employed to build either a decision tree or a rule set.¹⁷ One categorical target field is required for training a model. Typically, training time is short in estimating the label of the target. Furthermore, because the rules that are produced from the model have a very clear interpretation, C5.0 models are generally easier to grasp than certain other model types. C5.0 models are highly interpretable, allowing healthcare professionals to comprehend the decision-making process. It provides information on which variables are most significant for predicting treatment success. C5.0 is rather quick and efficient, and it includes measures such as pruning to prevent overfitting. C5.0 is ideal for predicting treatment effectiveness because of its interpretability and ability to handle a variety of data formats. It can be especially beneficial for explaining the reasoning for predictions to healthcare professionals.

NNs are strong models modeled after the human brain.¹⁸ An NN’s layers are often divided into three types: input layers, hidden layers, and output layers. The input layer receives the original data, and succeeding hidden layers process and change it, extracting progressively higher-level features. The final output layer produces the network’s prediction or classification result. NNs can simulate complicated linkages and interactions in data. They frequently have remarkable prediction accuracy. NNs can automatically learn features from data, which eliminates the need for manual feature engineering. A key disadvantage is that NNs are frequently viewed as “black boxes”, making it difficult to understand the decision-making process. NNs are appropriate for predicting treatment success when accuracy is the key concern and interpretability is less important. They are especially useful for working with large, complicated datasets that have non-linear relationships.

SVMs seek the best hyperplane that separates various classes in the feature space by the most significant margin possible.¹⁹ SVMs can discover a linear decision boundary that effectively divides non-linearly separable classes by translating the data into a higher-dimensional space. SVMs work well when there are more features than samples. They operate best when classes are clearly separated. SVMs can handle non-linear relationships using kernel functions. They can be computationally intensive and inefficient on huge datasets. SVMs are appropriate for predicting treatment success when the dataset is small and complicated, non-linear relationships must be modelled. Their capacity to offer a distinct line of boundary can be beneficial.

Despite its name, LR is not a regression approach but a classification procedure.²⁰ It is a simple and efficient method for dealing with binary classification problems. The LR model estimates the relationship between the input factors and the binary outcome probability. The model is easily interpretable, with coefficients reflecting the intensity and direction of the association between variables and outcomes. It works best when the connection between the predictors and the outcome is roughly linear. LR assumes predictors are independent and have a linear connection with the log chances of the outcome. LR is appropriate for predicting treatment effectiveness when the data’s relationships are predicted to be linear and interpretability is important. It works especially well with smaller datasets and when predictor independence is reasonable to expect.

BNs are practical tools for describing and reasoning about uncertain knowledge.²¹ They are based on probability and graph theory principles, allowing us to represent complex interactions between variables. A BN can express the joint probability distribution of a set of variables by employing conditional probability distributions. This allows for efficient reasoning and inference regarding the relationships between variables. They succeed at dealing with uncertainty and incorporate prior knowledge into their models. BNs offer a clear graphical picture of the interactions between variables. They are useful in understanding causal links, which can be critical in medical settings. Constructing and querying BNs can be computationally costly, especially when dealing with a high number of variables. BNs are appropriate for predicting treatment effectiveness when understanding causal links and dealing with uncertainty are critical. They are especially beneficial in medical applications that rely heavily on past information and probabilistic reasoning.

Results

We run our model using SPSS Modeler 18.3 on five models; C5.0, NNs, LR, BN and SVM. We performed hyperparameter tuning for five ML models to improve its classification performance. The best hyperparameter tunings we obtained after various experiments are as follows. The first is C5.0 in which mode is expert, pruning severity is 50, minimum records per child branch is 5, use global pruning is enabled; and winnow attributes are disabled. Second one is NNs, in which the model is a multilayer perceptron (MLP), and the overfit prevention set is 10%. The third one is LR, where the procedure is binomial since the target field is flagged. In this one, we enabled the option of including a constant in the equation, as this move has increased the model’s accuracy. In the next model, BN, we use the structure type of Tree Augmented Naïve Bayes model (TAN), and the parameter learning method is set to be Bayes adjustment for small cell counts. The last model is SVM, in which we run in the expert mode using the stopping criteria of 10⁻², regularization parameter of 10, and the kernel type of polynomial, together with the gamma value of 2, the bias of 0, and degree of 3. The SPSS Modeler stream of the model is given in Figure 2.

Figure 2 SPSS Modeler Stream for Prediction of the 6th Month.

To validate ML models, we first divided the dataset into training and test sets with the proportions of 70% and 30%, respectively. The classification performance of specific classifiers can be summarized using confusion matrices.^22,23 Utilizing the confusion matrix with two classes, the positive and the negative, we obtain the true positives (TP), false positives (FP), true negatives (TN), and false negatives (FN) for each model. Related matrix for the models is given in Table 3. Then, classification performance measures; namely accuracy (A), precision (P), recall (R) and F1 score (F1), are defined in terms of these outcomes. The performance metrics obtained during validation are determined and presented in Table 4, using the following equations:

(1)

Table 3 Confusion Matrix of Models

Table 4 Performance Evaluation of Models

When Table 4 is examined, it is seen that five ML models give similar good results. Especially recall scores are high. It means we have not missed any patients who would be treated successfully. Precision scores are also relatively good. It minimizes FP and optimizes the resource utilization. Both recall and precision are important in treatment success. The F1 score assesses precision and recall in a balanced manner. The high and consistent F1 scores achieved using five distinct ML models show that the models are reliable and robust on the data. This shows that each model has a balanced success rate in identifying TPs while minimizing FPs.

As seen from the Table 4 together with Figure 3, SVM model is the one that comes into prominence concerning the accuracy levels. Furthermore, LR and C5.0 provide satisfactory results with respect to the precision, recall and F1-scores. We present our concluding remarks using C5.0 due to the ease of explanation for the solutions obtained and its user-friendly form, making the clinicians follow the decision rules without expertise. Besides, the decision trees given by this model have the strong explainability.²⁴ As we aim to guide healthcare users through this study, it is essential to be able to present the results effectively. Using this model and the decision tree given in Figure 4, clinicians will know the success rate of treatment for a potential patient at the application.

Figure 3 Radar chart of the performance evaluation.

Figure 4 Decision Tree for Month-6.

This decision tree model predicts treatment success based on the patient’s symptom duration, angular deformation, and NRS at application. We start at the root of the tree (Node 0), where the first decision rule checks if the patient’s symptom duration is less than or equal to 6 (between 61 and 72 months). If the symptom duration is less than or equal to 6 (Node 1), we move to the next node, which checks if the patient’s angular deformation is present (1) or absent (2). The tree follows the path to the next decision rule until we reach a leaf node. At the leaf node, the model predicts whether the treatment will be successful or unsuccessful based on the accumulated conditions. At Node 2, the model predicts that the treatment will be successful with a 79.5% rate.

Discussion

Use of ML methods makes it possible to predict the results of the treatment, even at the moment of application. In this study, we observe that the success of CEPRF treatment depends mostly on symptom duration, angular deformation and NRS at admission.

The analgesic effect of CEPRF may be explained by the neuromodulation effect of pulse radiofrequency therapy. The electric fields generated by pulsed radiofrequency have been shown to suppress glial (particularly microglial) activity in the dorsal horn of the spinal cord, thereby reducing central sensitization.²⁵ Additionally, pulsed radiofrequency selectively modulates nociceptive transmission by altering signal conduction in small-diameter afferent fibers, particularly unmyelinated C fibers, without causing thermal damage.²⁶ Experimental studies have also suggested that pulsed radiofrequency may activate descending inhibitory pathways from the brainstem, including serotonergic and noradrenergic systems.²⁵ These combined effects lead to sustained suppression of pain transmission, possibly through changes in gene expression such as c-Fos in dorsal horn neurons, contributing to long-term attenuation of central sensitization.²⁶

ML techniques not only determine the factors affecting treatment success but also help to identify the distinctions where the most significant differences are observed. Analyzing Figure 4, it can be seen that for patients having pain for less than 6 years, 71.83% of success is observed at the end of 6 months. This rate decreases to 16.67% for longer periods of pain. It is an expected fact in the literature that the chances of success of the treatment decrease as the pain becomes chronic. Chronic pain has the capacity to become increasingly complex in its pathophysiology, and thus potentially more difficult to treat over time.²⁷ Also, for patients with pain for shorter periods, if the patient has no angular deformity, the success rate becomes 79.55%; otherwise, it is observed to be 59.26%. The design of the body can provide excellent positional adaptation of the musculoskeletal system in postural positions such as sitting, standing and lying down, allowing the body to maintain that position painlessly. In the presence of normal sacrococcygeal angulation, body weight is distributed in the coccygeal area optimally. However, in the presence of angular deformity, the weight distribution may be disrupted and may cause pain to be felt more due to increased pressure in a certain area. In conclusion, although it is not the only factor that triggers pain in these patients, the biomechanical structural disorder due to angular deformity persists. This may decrease the success of treatment in these patients.

For those having angular deformity, NRS applications are the next decision point. If the patient declared that he/she has NRS less than 8 and angular deformity, the success rate is 91.67%; whereas if the NRS is higher and also there exists angular deformity, 33.33% of all patients are successfully treated. These results are consistent with the current literature, claiming that the responses of patients experiencing severe pain to treatment are more resistant than patients experiencing low and moderate pain. In patients who report severe pain, the factor in pain perception is not only the magnitude of the pain but also may depend on psychosocial and behavioral reasons such as perceived pain controllability, catastrophizing and/or pain interference. Here, it is not enough to relieve the pain; psychological and social treatments may also be required.²⁸

To compare traditional statistical methods with ML methods, each has key differences, advantages and limitations. Traditional statistical methods are based on several assumptions and theory-driven, while ML methods are based on fewer assumptions and data-driven. ML methods handle complex data and capture complex relationships that traditional methods may miss. Our case has high-dimensional data, and relations are complex. Therefore, ML methods are thought to be more appropriate than traditional methods. However, ML methods require more computational efforts and area expertise.

The study’s main limitations are its retrospective design and the small number of patients in the cohort. Although the design of the study is retrospective, the data used in the study were collected prospectively, which increases the consistency and validity of the obtained results. To the best of our knowledge, this is the first study to use ML techniques for predictability in interventional pain management procedures. We think it is valuable to inspire future studies on this field.

Conclusion

The present study’s findings suggest that ML techniques can be used efficiently to evaluate influential predictive factors concerning the success of CEPRF application for coccyx pain and to determine patients suitable for the treatment. It is thought that different clinical studies using ML techniques may be helpful by providing insight into the decision-making process of patient-treatment matchings.