Analysis of Consistency Between Forensic Psychiatric Evaluation Diagnosis and Previous Clinical Psychiatric Diagnosis: A Retrospective Study

Introduction

Forensic evaluation of mental illness is an important professional activity in modern judicial practice, and its core task is to assess the mental state of the evaluatee and the ability to bear legal responsibility.^1,2 With the deepening of the rule of law, the importance of forensic psychiatric evaluation in criminal proceedings has become increasingly prominent. In forensic psychiatric identification, criminal responsibility assessment is one of its core contents and has important guiding significance in criminal punishment of suspects by judicial authorities.^3,4 In the practice of forensic evaluation, the consistency between evaluation diagnosis and clinical diagnosis has gradually attracted the attention of academia. Studies have shown that there are significant differences between the two diagnostic conclusions, which affect the accuracy of judicial decisions and bring challenges to forensic psychiatric evaluation work.^5,6 A multicenter study in 2021 found that the consistency rate between forensic evaluation diagnosis and clinical diagnosis was only 47.3%, and there were significant differences among different disease types.⁷ Clinical diagnosis of psychosis is mainly based on the data described by family members and patients, but forensic diagnosis requires other sources of data as collateral evidence to form a “chain of evidence”, such as data from units, friends, neighbors, hospitals, public security inquiry transcripts, and other channels, and the diagnostic basis is more enriched. Therefore, there are often inconsistencies between the diagnostic conclusions of clinical diagnosis and forensic diagnosis.⁸ Recent studies have pointed out that the proportion of cases diagnosed as “no mental illness” in forensic psychiatric evaluation is showing an upward trend.^9,10 In evaluation practice, cases with a history of mental illness but diagnosed as having no mental illness by evaluation are increasing, and the diagnostic changes involve complex medical and legal judgments.¹¹ A systematic review in 2023 indicated that diagnostic differences are not only related to assessment criteria but also closely related to factors such as changes in the social environment and increased legal awareness.^12,13 At present, domestic and foreign scholars have reached a consensus on the standardized construction of forensic psychiatric evaluation, but specific practices still face many challenges.¹⁴ For example, in the actual identification process, there will still be inconsistent identification opinions, whether the identification data are true, complete, and sufficient, and the interpretation of identification data by the appraiser and the analysis of mental inspection results will have an impact on the identification results, ultimately affecting the judicial judgment and endangering the judicial justice of the state. Therefore, in criminal cases, accurate assessment of mental status and determination of responsibility capacity require more empirical research support.¹⁵ Existing studies mostly focus on the evaluation characteristics of a specific type of mental illness or a specific case, and there is a relative lack of systematic research on the issue of diagnostic consistency.^16,17 Based on the above research background, by analyzing the data of 78 criminal cases in the Forensic Evaluation Department of a mental health center in 2021–2022, this study explores the consistency and influencing factors of evaluation diagnosis and previous diagnosis, in order to provide reference for improving the quality of forensic psychiatric evaluation.

Methods

Research Subjects and Design

A retrospective study design was used to select 78 criminal cases evaluated by the Forensic Evaluation Department of Wuhan Mental Health Center from January 1, 2021 to December 31, 2022. The research protocol was reviewed and approved by the Medical Ethics Committee of Wuhan Mental Health Center (Approval No.: WHPH-2023-01). Data were desensitized to protect the privacy of the evaluators, and therefore informed consent was waived.

Inclusion criteria: (1) criminal cases; (2) identification data (including personal basic identity information, case information, witness testimony, previous mental status information) are complete and reliable. Exclusion criteria were (1) unknown age of the appraised individual; (2) inability to form an identification diagnosis; and (3) no relatives’ survey material.

Data Collection

A self-made “Forensic Evaluation Case Survey Form for Criminal Cases Diagnosed as No Mental Illness” was used for data collection (See Appendix 1). The survey content covered demographic data, clinical data, and forensic-related data. Demographic data included basic information such as gender, age, marital status, education level, occupation, etc; clinical data included history of mental illness, diagnostic type, treatment status, etc.; forensic-related data included cause of case, evaluation diagnosis, legal capacity assessment, etc. Two psychiatrists who had received training independently collected data, and any discrepancies were determined by a third senior physician. Diagnostic criteria Psychiatric diagnosis was based on the “Chinese Classification and Diagnostic Criteria of Mental Disorders (CCMD-3)”, and criminal responsibility capacity assessment was based on the “Guidelines for Criminal Capacity Assessment of Mentally Disordered Offenders” (SF/Z J00104002-2016) issued by the Ministry of Justice. The previous diagnosis and evaluation diagnosis being completely the same or the main diagnosis being the same was defined as consistent; the diagnosis being completely different or the main diagnosis being different was defined as inconsistent. For cases with comorbidity, the first diagnosis was used as the main diagnosis for determination.

Quality Control

The investigators received standardized training, established a data collection, entry and verification system, used a double-person double-entry method and conducted data consistency testing, conducted regular data quality spot checks, and promptly corrected any problems found.

Statistical Analysis

SPSS 26.0 statistical software was used for data analysis. Count data were expressed as number of cases and composition ratio. Chi-square test was used for comparison between groups, and kappa test was used for agreement between clinical diagnosis and identification diagnosis. Kappa values between 0.01 and 0.20 were considered slight agreement, 0.21 and 0.40 were considered fair agreement, 0.41 and 0.60 were moderate agreement, 0.61 and 0.80 were good agreement, and 0.81 and 1.00 were very good agreement. P < 0.05 indicated a statistically significant difference.

Results

General Demographic Characteristics

Seventy-eight criminal cases were included, with 67 males (85.90%) and 11 females (14.10%), aged 19–64 years, with a mean age of 40.98±11.26 years. Marital status was mainly unmarried (41 cases, 52.56%), education level was mainly junior high school and below (51 cases, 65.38%), and occupation was mainly unemployed (55 cases, 70.51%). Except for marital status, there was no significant difference in other demographic characteristics (P>0.05). See Table 1 for details.

Table 1 Distribution of Basic Characteristics of Criminal Cases (n=78)

Distribution of Case Types

Theft cases accounted for the highest proportion (15 cases, 19.23%), followed by intentional homicide (12 cases, 15.38%), intentional injury (11 cases, 14.10%), robbery (9 cases, 11.54%), and picking quarrels and provoking trouble (8 cases, 10.26%). Other types (drug trafficking, rape, molestation, etc.) accounted for a total of 23 cases (29.49%). Mental illness history and diagnosis 43 cases (55.13%) of evaluatees had a history of mental illness. Among the previous diagnoses, schizophrenia accounted for 21 cases (48.84%), mood disorders 14 cases (32.56%), and other types 8 cases (18.60%). Evaluation diagnoses showed no mental illness in 38 cases (48.72%), physiological passion in 8 cases (10.26%), schizophrenia remission period in 7 cases (8.97%), and other types in 25 cases (32.05%). See Table 2 for details. Diagnostic consistency analysis among 43 cases with a history of mental illness, the evaluation diagnosis was consistent with the previous diagnosis in 12 cases (27.91%) and inconsistent in 31 cases (72.09%). The Kappa value was 0.243, 95% CI (0.0302, 0.4240) (P < 0.05), indicating poor consistency. See Table 3 for details.

Table 2 Distribution of Mental Illness History and Evaluation Diagnosis (n=78)

Table 3 Concordance Analysis Between Identification Diagnosis and Previous Clinical Diagnosis (n = 43)

Analysis of Cases with Inconsistent Diagnosis

Of the 31 discordant diagnoses, 8 (25.81%) were previously diagnosed with schizophrenia but evaluated as having no psychiatric disorder and 5 (16.13%) were diagnosed as other types; 4 (12.90%) were previously diagnosed with affective disorder but evaluated as having physiological passion and 6 (19.35%) were diagnosed as having no psychiatric disorder; 5 (16.13%) were previously diagnosed as other types but evaluated as having no psychiatric disorder and 3 (9.68%) were diagnosed as other types (other types: anxiety disorder; neurosis; mental disorder caused by organic brain disease; marginal intelligence; mental disorder caused by active substance, etc). See Table 4 for details.

Table 4 Analysis of Cases with Inconsistent Diagnosis (n=31)

Analysis of Evaluation Conclusions

Twenty-eight cases (35.90%) were assessed as having full criminal responsibility, 9 cases (11.54%) were assessed as having the ability to serve a sentence, 5 cases (6.41%) could have compulsory treatment terminated, and the remaining 36 cases (46.15%) were other types of conclusions. Cases of full criminal responsibility were most common in theft (8 cases) and intentional injury (6 cases).

Discussion

There are significant differences between forensic psychiatric evaluation diagnosis and previous clinical diagnosis, with low consistency (Kappa=0.243), which is consistent with recent domestic and foreign research results.^18,19 The causes of the differences involve multiple aspects such as the specificity of forensic psychiatry, diagnostic thinking patterns, and assessment criteria. Psychiatric disorders were assessed in 55.13% of previous diagnoses, which was lower than in the study by Gao Ying et al,²⁰ and 63.3% of their previous diagnoses had psychiatric disorders. In another study,⁹ a total of 8302 forensic cases of mental illness over a 20-year period were investigated, and 56.10% were found to have different types of mental illness, similar to the results of this study.

Forensic psychiatric evaluation has obvious time dimension characteristics. Evaluation diagnosis requires a retrospective assessment of the mental state at the time of the crime, while clinical diagnosis makes an immediate judgment on the current state.²¹ In 8 cases with a previous diagnosis of schizophrenia, the evaluation diagnosis was no mental illness, which is related to the dynamic change characteristics of psychiatric symptoms. Studies have shown that about 40% of schizophrenia patients achieve clinical remission, and some patients have good recovery of social functioning.²² The previous diagnosis cannot accurately assess the mental state at the time of the crime. In addition, some information that patients or their families may conceal in order to protect their personal privacy during the visit, such as criminal predecessors, drug use, etc, the information received by clinicians is not comprehensive, and schizophrenia is sometimes difficult to differentiate from hallucinations and delusions caused by drug use from symptoms alone, which can lead to diagnostic errors. Because forensic identification has obtained extensive data evidence, it can be more comprehensive judgment and improve the accuracy of diagnosis, which also shows that evidence plays an important role in forensic psychiatric identification. In subsequent studies, more similar cases can be included to find the rule of changes in their condition and find the main symptoms and manifestations at different stages to avoid misinterpretation due to changes in their condition.

Forensic evaluation and clinical diagnosis adopt different thinking patterns. Clinical practice follows the principle of “presumption of illness” and comprehensively considers psychiatric symptoms; forensic evaluation adopts the thinking of “presumption of no illness” and requires sufficient evidence to support the diagnosis of mental illness.^23,24 Among 10 cases with a previous diagnosis of affective disorder, 6 cases were diagnosed as having no mental illness and 4 cases were diagnosed as physiological passion by evaluation, reflecting the strict definition requirements of forensic evaluation for psychiatric symptoms. Studies have pointed out that distinguishing between affective symptoms and stress response requires more objective assessment tools.²⁵ Therefore, we can refer to relevant domestic research and development of forensic identification evaluation tools, such as the evaluation of false mental illness, its evaluation and diagnosis content and criteria, as well as diagnostic procedures should be more suitable for forensic identification, so that its results are more accurate.

The environment of the crime and litigation pressure affect the presentation of psychiatric symptoms. Studies have found that 25% of evaluatees exhibit abnormal mental states in the judicial environment compared to usual.²⁶ The phenomenon of diagnostic inconsistency is common in actively criminal cases such as theft and intentional injury, which is related to coping styles and defensive psychology.²⁷ Neuroimaging studies suggest that there are significant differences in cognitive function and behavioral control ability under stress.²⁸

The phenomenon of diagnostic inconsistency varies among different case types. The diagnostic consistency of violent crimes (intentional homicide, intentional injury) is lower than that of property crimes, which is consistent with international research results.^29,30 Violent behavior involves complex psychological motives and emotional factors, increasing the difficulty of assessing mental state.³¹ Violent criminal behavior is caused by multiple factors, not single or multiple influencing factors simply summed. At present, the common influencing factors include previous history of violence, low economic status, unemployed, no partner, drug use, severity of psychotic symptoms, no specialist treatment, etc. Violent criminal personnel have certain demographic characteristics and psychological and cognitive deficits and abnormal brain imaging structure. Therefore, more attention should be paid to violent crime cases, not only medical history data but also the collection of data from many aspects and multiple channels as far as possible, including previous crime data, illegal data, and testimony data of other witnesses, so as to ensure that mental identification of the parties to the case can be more comprehensive and improve the consistency level of diagnosis.

The determination of full criminal responsibility is closely related to the nature of the case. In theft cases, the proportion of those with a history of mental illness assessed as having full criminal responsibility is relatively high, which is consistent with the results of recent multicenter studies. In criminal cases, burglary accounts for a high proportion, similar to the results of many studies.^32,33 Xu Xuebing³² and others analyzed forensic psychiatric identification cases in Ningxia and found that intentional injury (homicide) cases, rape cases, and theft cases accounted for the top three criminal cases. The assessment criteria for cognitive and control abilities in property crimes are relatively clear, while the assessment of abilities in violent crimes is more complex.

Because forensic psychiatric identification involves knowledge from multiple disciplines and the object of identification is complex and variable, previous clinical diagnoses and forensic psychiatric identification sometimes have inconsistent circumstances affecting judicial decisions. According to the problems found in this study, temporal changes, different thinking patterns, and different types of cases can affect the consistency of diagnosis, and it is recommended that forensic psychiatric identification can be supplemented in combination with the current artificial intelligence system. Wang Yinglong³⁴ et al showed that guided by the principle of “double embedding”, following the design and R & D path of “block development”, “system integration”, and “process integration”, relying on natural language processing technology, psychological computing technology, and intelligent auxiliary decision-making technology resources to develop a set of artificial intelligence assisted mental disorder identification system, which can empower judicial psychiatric identification practice, solve some problems existing in judicial mental identification, and improve the reliability of identification opinions. In addition, it can strengthen the communication between clinicians and forensic identification personnel, or provide judicial diagnosis training for clinicians and increase the professionalism of clinicians.

Limitations are reflected in three aspects: the limited sample size of a single-center study, which may affect the extrapolation of the results; the retrospective study design has collection bias, which may affect the accuracy of the results; the lack of long-term follow-up data due to the short data collection time makes it impossible to assess the dynamic characteristics of diagnostic changes; in addition, the identification results may also be affected by the identification personnel ‘s own experience and thinking pattern, which may have potential bias. Future studies will address this issue through a multicenter, prospective study design, such as collecting identification data from other regions around Wuhan for comparative analysis. In addition, long-term follow-up studies can be conducted on the parties to the case to collect data on the dynamic changes of the condition.

Conclusion

This study found that the agreement between the evaluation diagnosis and previous diagnosis was low (kappa = 0.243), and the diagnostic discordance rate reached 72.09%, with patients with schizophrenia and affective disorders having higher diagnostic discordance. The diagnostic consistency of violent crime cases is obviously lower than that of property crime cases. In this study, most of the parties in criminal cases have mental illness, suggesting that people with mental disorders will have a greater probability of committing crimes. A relatively high proportion of theft cases had a history of mental illness assessed as having full criminal responsibility. Therefore, according to different disease types and different case types, it is necessary to establish corresponding diagnostic criteria, and then develop more objective evaluation tools, combined with the use of new technologies, such as artificial intelligence, to improve the current shortcomings of forensic identification. It is suggested to strengthen the standardization construction of forensic psychiatry, such as formulating national forensic evaluation guidelines; implementing standardized training programs for evaluators or establishing quality assurance mechanisms; and carrying out interdisciplinary cooperative research, such as communication in law, medicine, psychology, and other disciplines, to avoid the situation that a single discipline is difficult to cope with complex cases.