The Relationship Between Computerized Face and Tongue Image Segmentation and Metabolic Parameters in Patients with Type 2 Diabetes Based on Machine Learning

Introduction

In ancient Greek medicine and Chinese Traditional Medicine (TCM), physicians examined the tongue and face’s surface for various characteristic features, including color, hydration, coating, and texture, to obtain insight into a patient’s overall health.^1–3 The tongue and face observation are critical components of the numerous tools for diagnosis.^4,5 Tongue and face diagnosis, also referred to as tongue and face inspection, can provide valuable insights into the efficacy of treatment, the progression of the disease, and the overall health status of a patient.⁶ The tongue and face’s color can vary from colorless to dark red, suggesting a variety of pathologies, including blood deficiency and Qi deficiency. The tongue coating and texture also indicate conditions such as metabolic abnormalities in the internal environment, abnormal proliferation of gut microbiota, serum lipid imbalance, and microvascular damage.^7–9

A global health concern, type 2 diabetes is a prevalent metabolic disorder that is distinguished by elevated blood glucose levels as a result of insulin resistance and insufficient insulin production.¹⁰ Although blood tests and clinical symptoms are the primary diagnostic tools, recent research and traditional medical practices have underscored the potential of tongue and face observations as non-invasive indicators of this condition.^11–13 In TCM, DM was termed “XiaoKe”, or emaciation and thirst syndrome, in which the patient experiences weight loss and thirst secondary to the polyuria and hyperglycemia; XiaoKe could be traced to the TCM pathophysiology of lung dryness causing thirst, stomach heat causing hyperphagia and weight-loss, and renal asthenia causing polyuria and diabetes; therefore, the syndromes could be classified into lung-stomach dry-heat, spleen-stomach Qi asthenia, renal Yin-asthenia, Yin and Yang asthenia syndromes, which were accompanied by the distinct and specific tongue and facial signs. The TCM physicians could diagnose the DM patients with different signs according to the various manifestations of the tongue combined with other clinical symptoms and signs as well as pulse signs, acknowledge the condition or progress or treatment efficacies, and eventually administer respective medications (dialect therapy) and other therapies such as acupuncture. Thus, the facial and tongue signs are critical to diagnosing and treating DM by TCM physicians. For example, some TCM physicians could judge the patients with “spleen” (could be alternated to the modern medicine notion of pancreas) Qi asthenia by observing thick white tongue fur with other characteristics such as expanded tongue volume, slippery tongue, or scalloped tongue, indicating the accumulating of the dampness (a sign may be associated with incompetently handling the fluid transforming and transportation of the spleen caused by Qi asthenia), a classical sign in XiaoKe and obese patients, therefore alerting us if we should measure the pancreatic function and insulin resistance in these spleen Qi asthenia T2DM patients; another example would be the purple tongue or lips or even facial, suggesting stagnant of blood, a sign alert to exclude potential cerebrocardiovascular complications, or to implement respective preventive measures.

To validate and further explore the intimate metabolic relationships between the tongue and facial signs with T2DM via the modern medicine multiparameter assay, in this paper, we investigate the scientific foundation and clinical implications of observing the correlation between T2D and its manifestation through tongue and face characteristics. The incorporation of TCM and modern medicine would not only fill the gap between them, for instance, to promote the mutual understanding between them on the rationale or mechanism distinction of diagnosis and treatment, but also strengthen both of them, such as standardized and precise TCM diagnosis and treatment, or improve individualized therapy for the T2DM patients in modern medicine especially in chronic diabetic complications.

The use of image processing techniques to digitize and analyze tongue and face observation was becoming increasingly prevalent due to technological advancements.^14–17 The precision and reproducibility of tongue and face diagnosis are enhanced by these techniques, which increases its accessibility to all physicians worldwide who are not familiar with Traditional Chinese Medicine (TCM). These technologies can potentially improve the efficacy and reliability of ancient diagnostic methods by integrating precise laboratory metabolic data with the ancient tongue and face diagnosis in T2DM patients. This paper examines the implications of tongue-and-face observation on the metabolic parameters of patients with T2DM.

Materials and Methods

Source of Patient Data

The study includes 269 adult T2DM patients admitted to the Department of Endocrinology at Shanghai Pudong Hospital between March and May 2024. They were selected in a randomized manner and fulfilled the diagnostic criteria for T2DM, as defined by the established World Health Organization (WHO) and American Diabetes Association (ADA) guidelines.¹⁸ Oral anti-diabetic medication, such as metformin, sulfonylurea, glinides, pioglitazone, sodium-glucose co-transporter inhibitors (SGLT-2i), α-glucosidase inhibitors, and dipeptidyl peptidase-IV inhibitors (DPP-IV inhibitors), as well as injectables, such as glucagon-like peptide-1 receptor agonists (GLP-1RA) and insulin (basal insulin or combined with prandial insulin or premixed insulin), were administered to patients in accordance with disease progression, glycemic control, and complications such as liver or kidney insufficiency. The selection criteria excluded other types of diabetes, such as T1DM, secondary DM, gestational DM, and other specific forms of DM. The study also excluded a few severe conditions, such as diabetic acidosis, hyperosmolar hyperglycemic state, lactic acidosis, shock, hypoperfusion of circulation, severe systemic disorders, stroke, asthma, uremia, intestinal obstruction, and severe sepsis.

Blood Sampling and Methods of Laboratory Assessment

The metabolic parameters were collected from patients on the day following their admission. These laboratory parameters include serum blood glucose, pancreatic islet function, hemoglobin A1C (HbA1C), thyroid function, hepatic and renal function, and whole blood cells. All biochemical indicators, including fasting blood glucose, hepatic function, and kidney function indicators, were analyzed using a full-automatic biochemical analyzer (ADVIA Chemistry XPT, SIEMENS, USA). The TOSOH G8 analyzer was used to determine the glycemic control level over the past three months. C-peptide and thyroid function indicators were processed using chemiluminescence methods in a full-automatic chemiluminescence immunoassay analyzer (ADIVA Centaur XPT, SIEMENS, USA).

Data Collection of Face and Tongue Color

Upon admission to the ward, the patients were arranged to receive the facial and tongue information sampling in an exclusive room with sufficient light and quietness. A prior request was made to the patient to clean their face and mouth. Subsequently, the tongue device was equipped with advanced AI ((XiMaLife, Sinology, China) and automatic machine learning to analyze the segmented features of the tongue and face by incorporating the fundamental information of each patient, and the based rationale of AI has been previously established.^19–21 This analysis includes color correction (RGB value retraction, CIELab space analyzing based on machine learning, segmentation algorithm optimization utilizing ASPP module analyzing multiple alternative parameters), DetNet algorithm, a real-time Anchor-free model detection utilizing ATSS target-sampling and Generalized Focal Loss function to classification and boundary regression, the SegNet to improve the tongue segmentation analyses.

The face and tongue images of T2DM patients were subsequently acquired by the device equipped with a high-definition camera. Patients volunteered to receive the facial and tongue image sampling by the device at the standardized light source aligned to the CIELAB standards. The data were subsequently transferred to the private database of Shanghai Traditional Medicine University for backup. The technician subsequently retracted the backend information and privately transmitted it to the investigators of this study. Numerous previous studies have found this machine’s operating system information and detailed calculations and processes.

The categorical features for facial color are defined by machine calculation and include pale, normal, redness or yellow, and black. Lip color: red or dark red. Facial gloss: glossy or Dull. Tongue characteristics include pale white, light red, red, deep red, or dark red. Color of fur: yellow and white. Fur greasy: normal fur, oily fur, or lacking fur. Fur thickness: Thick or thin; Tongue lesion: normal or fission. Tongue volume: average or fat.

The face, lip, gloss, tongue, or fur color data were calculated using the Lab Color Space, also known as the CIELAB color space. This color-opponent space is based on the coordinates of the nonlinearly compressed CIE XYZ color space. It consists of three dimensions: L, a, and b, representing brightness, and two color-opponent dimensions, respectively. L (Lightness) refers to the pixel’s luminance, which ranges from pure black to pure white: 0–255. a (Red-Green): Indicates the range from red to green, with values ranging from approximately 0 (green) to 255 (red). b (Yellow-Blue): Indicates the spectrum from yellow to blue, with values ranging from approximately 0 (blue) to 255 (yellow). The discrete dimensional color parameter allows the machine to analyse the tongue and facial condition of individual T2DM patients. The sampling procedure is illustrated in Figure 1.

Figure 1 The tongue imaging device discern the different pathological features of tongue displayed as normal color (A), coating (B) tongue lesion (C), and both (D).

Statistical Analyses

The statistical analyses were conducted using Prism (GraphPad, version 10.0) and SPSS (IBM, version 26.0). We utilized Spearman correlational analysis, a nonparametric analysis, to establish correlational relationships between facial and tongue imaging features and dimensional parameters with glucose metabolic, hepatic, renal function, cardiac injury, tumor, thyroid function, and blood cell indicators. We screened the p-coefficient with values below 0.05 as two parameters significantly correlated. A multilinear regression analysis was employed to determine the independent facial and tongue imaging features associated with significant indicators. Multiple receiver operating characteristic curves (ROCs) were drawn to assess the predictive value of significant facial and tongue imaging parameters for metabolic parameters. p < 0.05 was established as the significance threshold for all statistical analyses. Data were presented as the mean ± standard error of the mean (SEM).

Results

The Characteristics of Patients with T2DM

Our study included T2DM patients for facial and tongue feature analyses. Table 1 displays the demographic characteristics, glucose metabolic status, lipid profile, and serum uric acid level.

Table 1 The Demographic Characteristics of Included T2DM Patients

The Relationship Between the Facial and Tongue Features and Parameter of Glucose Metabolic State

We analyzed the correlational relationships between the facial and tongue features with dimensional color parameters and glucose metabolic status represented by HbA1c, GA, C-peptide, and insulin levels. The significant results are shown in Tables 2 and 3.

Table 2 The Correlational Relationship Between Significant Facial and Tongue Feature with the Glucose Metabolic Status

Table 3 The Correlational Relationship Between the Significant Dimensional Parameters of Facial and Tongue Colors and Glucose Metabolic Status

The Relationship Between the Tongue Color and Parameters of Hepatic and Renal Function

We analyzed the correlational relationships between the facial and tongue features with dimensional color parameters and hepatic and renal function. The significant results are shown in Tables 4 and 5.

Table 4 The Correlational Relationship Between the Significant Facial and Tongue Feature and Critical Hepatic and Renal Function Indicators

Table 5 The Correlational Relationship Between the Significant Dimensional Facial and Tongue Colors and the Critical Hepatic and Renal Function Indicators

The Relationship Between the Facial and Tongue Imaging Feature and the Cardiac Injury Markers

We analyzed the correlational relationships between the facial and tongue features with dimensional color parameters and cardiac injury markers. The significant results are shown in Table 6.

Table 6 The Correlational Relationship Between the Significant Facial and Tongue Imaging Feature and Dimensional Color with the Cardiac Injury Markers

The Relationship Between the Facial and Tongue Imaging and the Tumor Markers in T2DM

We analyzed the correlational relationships between the facial and tongue features with dimensional color parameters and tumor markers. The significant results are shown in Tables 7 and 8.

Table 7 The Correlational Relationship Between the Significant Facial and Tongue Imaging Feature with the Tumor Markers in T2DM

Table 8 The Correlational Relationship Between the Significant Facial and Tongue Imaging Dimensional Color with the Tumor Marker in T2DM

The Relationship Between the Facial and Tongue Imaging with the Thyroid Function, Hemoglobin, White Blood Cell, and Palate in T2D

We analyzed the correlational relationships between the facial and tongue features with dimensional color parameters and thyroid function, blood cells. The significant results are shown in Tables 9 and 10.

Table 9 The Correlational Relationship Between the Significant Facial and Tongue Feature and Dimensional Color with the Thyroid Function, Hb, WBC, and Plt

Table 10 The Correlational Relationship Between the Dimensional Color of Facial and Tongue Imaging and Thyroid Function, Hb, WBC, and Plt

The Multilinear Regression Analyses Use Parameters of Facial and Tongue Imaging as the Independent Determinators for the Significant Metabolic Indicators

We performed the regression analyses to determine the facial and tongue features with dimensional color parameters to the aforementioned significant metabolic parameters. The significant determinants are displayed in Table 11.

Table 11 The Multilinear Regression Models of Variables to the Significant Metabolic Parameters’ Changes

The Predictive Value of the Significant Facial and Tongue Imaging Parameters in Glucose Metabolism, Renal Function, Cardiac Injury, Tumor Markers, Thyroid Function and Blood Cells via ROC

Finally, we created the receiver operating characteristic curve (ROC) to assess the predictive value of significant facial and tongue imaging parameters for glucose metabolism (Figure 2), eGFR (Figure 3), cardiac injury (Figure 4), tumor markers (Figure 5), thyroid function and blood cells (Figure 6).

Figure 2 The ROC curve of facial and tongue imaging feature and dimensional color for glucose metabolic parameters including HbA1c (A and B), FPCP (C–E), and PPCP (F and G). Abbreviations: HbA1c, glycated hemoglobin A1c; FPCP, fasting plasma C-peptide; PPCP: 2 hours postprandial plasma C-peptide.

Figure 3 The ROC curve of facial and tongue imaging feature and dimensional color for renal function parameters represented by eGFR: Lip L (A); Face b (B); Fur (C); Tongue color (D). Abbreviation: eGFR: estimated glomerular filtration rate.

Figure 4 The ROC curve of facial and tongue imaging feature and dimensional color for cardiac injury markers represented by MYO (A and B) and CKMB (C and D): Lip L (A); Fur b (B); Fub (C); Gloss L (D). Abbreviations: MYO: myoglobin; CKMB: creatine kinase MB type.

Figure 5 The ROC curve of facial and tongue imaging feature and dimensional color for Tumor markers represented by CA125 (A and B), CEA (C), and SCC (D and E): Fur b (A); Tongue b (B); Gloss M (C); Gloss L (D); Fur b (E). Abbreviations: CA125, carbohydrate antigen 125; CEA, carcinoembryonic antigen; SCC, Squamous cell carcinoma antigen.

Figure 6 The ROC curve of facial and tongue imaging feature and dimensional color for thyroid function represented by FT3 (A and B), and Hb (C–E). Abbreviations: FT3, free triiodothyronine; Hb, hemoglobin.

Discussion

TCM is a comprehensive system of diagnosis and treatment that is predicated on an understanding of the internal equilibrium and harmony of the human body.²² The four diagnostic procedures in Traditional Chinese Medicine are inspection, inquiry, sensing and listening, and palpation. Inspection diagnosis, particularly tongue observation, is considered the most fundamental and initial stage of these, as it provides an insight into the patient’s internal state. The tongue is regarded as the external entrance of the heart in Traditional Chinese Medicine (TCM), and it is responsible for the cardiovascular system’s condition. In recent decades, emerging consensus on the tongue pattern as the diagnostic tool in multiple diseases has provoked scientists and clinicians to develop instruments for observing the condition of the tongue in order to assist therapy of both local and systemic diseases,^23,24 and even advanced the predictive or diagnostic tools in diabetes with minimal or non-invasive paradigms.^11,25

In the present research, we discovered that the metabolic status and related organic laboratory parameters, including hepatic and renal function, cardiac injury markers, tumor markers, thyroid function, and blood cell count, were correlated with the facial, tongue, lip, fur, and other features, as well as dimensional color parameters. This suggests that there is a close correlation between the facial, tongue, and color characteristics and the metabolic status of T2DM patients, as evidenced by the precise analysis. The predictive value of these indicators was evaluated for the laboratory markers that were evaluated in the present study using the ROC assay. Consequently, we could combine these features with other clinical parameters to create a more comprehensive model for predicting the condition and progression of T2DM in patients. This model would be particularly useful for assisting physicians in evaluating the risks of microvascular and macrovascular complications and the therapeutic effects of these patients in future clinical practices.

Additionally, there is an increasing practice of incorporating tongue and facial diagnosis from Traditional Chinese Medicine (TCM) into Western medical practices. Numerous medical professionals, such as physicians, nurses, and therapists, realize the importance of integrating traditional Chinese diagnostic methods into their clinical practices.^26,27 This integration enables a more comprehensive approach to patient care by integrating the strengths of both traditional and modern medical systems.^28,29 It is imperative to conduct ongoing research on tongue diagnosis in order to enhance its scientific foundation and clinical application. In order to improve the scientific foundation and clinical application of tongue and face diagnosis, it is essential to conduct supplementary research, such as the current study. The efficacy of this diagnostic instrument could be enhanced by conducting research and AI deep learning on the correlations between tongue manifestations and specific biomarkers, pathological changes, and genetic factors.^30–32 Furthermore, it is imperative that the next generation of healthcare professionals be provided with a thorough comprehension of this valuable diagnostic technique through educational programs and initiatives that are designed to enhance TCM knowledge and skills, such as tongue observation.

In resource-limited environments where advanced diagnostic instruments are not readily accessible, the integration of tongue and face observations into routine assessments can offer valuable insights into a patient’s overall health status. This is particularly true in clinical settings. Nevertheless, it is imperative to underscore that these observations should be employed as supplementary tools rather than as independent diagnostic criteria.

Limitations

We did not conduct risk comparisons between individuals with T2DM and those who are not. Additionally, future predictive models must incorporate additional tongue and facial features, laboratory data, and imaging data, including CT, coronary arterial imaging, cranial MRI, and MRA, to evaluate the severity and risks of macrovascular complications. It is also necessary to quantitatively evaluate the TCM syndromes in order to identify and reconstruct the relationships with the condition of T2DM in modern medicine practice. This will enable us to provide more precise and personalized treatment.

Conclusion

In conclusion, tongue and face observation is a critical element of clinical diagnosis in TCM, as it assists to inform treatment decisions, prognosis, and diagnosis by providing a glimpse into the patient’s internal state. The integration of modern technologies, such as AI deep learning and digitization, has enhanced its accuracy, reproducibility, and accessibility. Tongue and face diagnosis remain a valuable instrument for the promotion of comprehensive wellness. Although these observations offer a non-invasive, cost-effective approach to obtaining insight into a patient’s health status, they require further scientific investigation to verify their diagnostic accuracy. Modern medical research and traditional wisdom can be combined to create a more comprehensive diagnostic and management strategy, thereby improving our understanding of T2DM.