Interobserver Variability of Coronary Stenosis Characterized by Coronary Angiography: A Single-Center (Toronto General Hospital) Retrospective Chart Review by Staff Cardiologists

Introduction

Research Questions

1. Is there statistically significant interobserver variability amongst cardiologists in the interpretation of fluoroscopic images obtained during coronary angiography?
2. Do these variabilities in interpretation affect clinical management, i.e., PCI (percutaneous coronary intervention) vs. medical management or CABG (coronary artery bypass graft)?
3. Are there differences in interpretation based on the proximal, midportion, and distal parts of each coronary vessel?
4. Are some stenosis percentages reported on coronary angiograms more frequently than others?

Background

Background evidence from numerous studies of coronary angiography shows differences among observers’ assessments of 15%–45%.¹

Methods

Retrospective Cohort Study Subjects

Inclusion Criteria

1. Adult patients (≥18 years of age) who had undergone ICA (invasive coronary angiography) for diagnostic and therapeutic intervention during 2018 and 2019.
2. Minimum sample size of 200 patients.

Exclusion Criteria

• Patients with a history of cardiovascular disease other than CAD (coronary artery disease; e.g., valvular heart diseases, second- or third-degree atrioventricular block, congenital cardiac diseases, arrhythmias).

• Previous cardiac surgeries. Applicable data were collected anonymously and unique codes provided to match the patient MRN (medical record number) to protect patient privacy and confidentiality. A total of 200 eligible University Health Network patient medical records were used after ethics committee approval. Ethics approval was provided by the University Health Network Coordinated Approval Process for Clinical Research (CAPCR ID:18–6278.0). Since this was a retrospective study on completely deidentified patients’ charts, as per our CAPCR approval, informed consent was not required. Three cardiologists with high experience in coronary angiography, including the primary cardiologist of each patient, read the angiograms of 200 patients independently. In other words, three participating cardiologists read the angiograms on different occasions. Each of them read the data in a blinded manner on the severity of the stenosis and determined the percentage for each stenosis. Even though the background and experience of the three cardiologists were different, each of them had a history of performing at least 1000 coronary angiograms. The observers did not have any role in the selection of patients, and none of them had more clinical follow-up data than the others. No data, such as the name of the patient, date of imaging, or diagnosis, were revealed to the observers.
• CAD is the leading cause of death around the world.² The stenosis of each coronary artery was graded from 0 to 100% based on the maximal diameter of possible narrowing identified by looking at different views taken during the procedure by each cardiologist separately. There are two main coronary arteries, i.e., the LM (left main) and RCA (right coronary artery), even though there is a third main coronary artery — the ramus intermedius — in around 20% of people. In the rest of the population (around 80%), the LM divides into two branches: LAD (left anterior descending) and LCX (left circumflex). Moreover, the main branches that originate from the LAD are diagonals (D1, D2, D3, etc.) that supply the anterolateral regions of the LV (left ventricle). Obtuse marginal branches (OM1, OM2, etc.) are the most important branches of the LCX. There is high anatomic variability in the number of diagonal, septal, and OM branches present in the left coronary artery of each individual person.³

The RCA’s main distal branches are the PDA (posterior descending artery) and PLV (posterolateral ventricular), which were assessed separately in this study. Around 80% of the general population are RCA–dominant, meaning that both PDA and PL branches are supplied via the RCA, while 10% of population has a left coronary dominance with PDA and PL branches originating from LCX,³ The remaining 10% display codominance with the PDA arising from the RCA and PL branches, in turn arising from the LCX. Furthermore, for each coronary artery stenosis, >70% is considered significant stenosis, with the exception of the LM, for which ≥50% stenosis is defined as significant. The main coronary arteries (LAD, LCX, RCA, ramus) were divided in our study into the proximal part (⅓ proximal), mid portion (⅓ middle) and distal part (⅓ distal) in order to localize the site of narrowings more accurately. In the present study, analysis of interobserver variability was based on the percentage of stenoses in coronary arteries only, which is the most objective aspect of coronary angiography.

Data Analysis

The primary analysis compares the level of interobserver agreement among different observers (cardiologists) using kappa statistics, i.e., ICC with 95% CIs. The ICC values are used to interpret agreement among the observers as per Landis and Koch.⁴ An ICC of 1 (100%) represents complete agreement, whereas values below 0 represent poor agreement among observers. Furthermore, an ICC of 0 means that the results are similar to results obtained by chance alone.

Results

This study consisted of 200 patients who were selected randomly from the coronary angiograms completed at Toronto General Hospital in 2018 and 2019. In order to assess disagreement among observers for each coronary vessel of every single patient, the results for comparison of three reporters showed that the ramus had the highest agreement (ICC 99%, P<0.0001) while the D2 branch of the LAD indicated the lowest agreement (ICC 52%, P<0.0001).

The ICC for the LM was also high 90% (P<0.0001). Focusing on the LAD, its proximal portion had the highest agreement among readers (ICC 0.92, P< 0.0001), while the midportion and distal part of the LAD showed relatively lower agreement, i.e., 62% and 68%, respectively. The highest agreement for the LCX was found in the OM2 branch (ICC 91%, P<0.0001), while the midportion of the LCX yielded an ICC of just 54% (P<0.0001). Agreement regarding the proximal RCA was high (ICC 84%, P<0.0001), while the ICC for the midportion of the RCA was 72% (P<0.0001), even though the distal part of the RCA showed the highest agreement in regard to this vessel (ICC 90%, P<0.0001).

For the proximal parts of the LAD, LCX, and RCA, the highest agreement (90%) was for the proximal LAD, whereas the RCA was second among the three observers (84%). Moreover, the proximal LCX showed 83% agreement among the three observers. Regarding the midportion of the aforementioned vessels, agreement among the three observers was 72% for the midportion RCA, 62% for the midportion of the LAD, and 54% for the midportion of the LCX.

In regard to the distal parts of vessels, the highest agreement was on the distal RCA, with an ICC of 90%, with the LAD second on 68%, whereas the LCX distal showed just 63% agreement. All in all, mean agreement among the three observers regarding the proximal parts of the LAD, LCX, and RCA was 86.3% and that for the midportion of these vessels was 62.6%. Furthermore, mean agreement among the three readers regarding distal parts of the aforementioned vessels was 73.6%. In other words, the proximal portions of coronary arteries showed the most consistent readings among observers.

For the proximal parts of the LAD, LCX, and RCA, the RCA had the most consistent agreement among participant cardiologists, so the clinical outcome of a majority of patients would have been the same if any of the participant cardiologists were the responsible physician for each of the patients. There was some interobserver variability among the three readers, with overall agreement of 77.4% (Table 1, Figure 1). In terms of reasons for the differences, among items with sufficiently similar levels of statistical significance, it was not surprising to find that judgment about distal portions of arteries was less reliable than readings of the proximal portions.

Figure 1 Agreement (ICC) among the three observers’ reports.

Table 1 ICCs of three observers for different coronary arteries

Certainly, one of the reasons could be due to attenuation of the injected dye in distal parts of the vessels compared to proximal parts, especially when there is a significant narrowing in the proximal or midportion of the vessel, which can attenuate the dye in distal parts more. The total stenoses reported by participant cardiologists was 660. In the assessment regarding percentage of stenosis reported by the participating cardiologists, it was found that certain stenosis figures (e.g., 90%, 30%, 80%, and 50%) were quoted more frequently in angiography reports than predicted by chance alone. In other words, some percentages were reported more than others. The most prevalent proportions reported by reviewers were 90%, 30%, 80%, and 100%. On the other hand, some were reported less, e.g., 17%, 15%, and 10%, and some were not used at all by any of the participating cardiologists, e.g., 18%, 76%, or 89%, among many others (Figure 2). In other words, this study shows that the trend of the cardiologists while reading each coronary stenosis is traditionally towards reporting certain percentages more than others. One of the reasons could be the routine usage of these percentages, e.g., 90%, 80%, or 50%, by most cardiologists, as they are easier for them than for stenoses, e.g., 89%, 81%, or 53%. However, this issue usually does not interfere with clinical decision-making, i.e., stenoses with severity of 80% and 77% do not usually differ in terms of the clinical decisions made by cardiologists.

Figure 2 Frequency of each specific percentage reported by participant cardiologists (total number of reported stenoses 660).

Discussion

The aim of this study was not to determine which of the participating cardiologists read angiograms best/better, but rather just to compare agreements and disagreements regarding the severity of stenoses and also to what extent this interobserver variability can impact on clinical decisions regarding patients. Our study found interobserver agreement of 22.6% among three reviewers. Interobserver variability between two reviewers was 22.5% (Table 2). Kappa statistics are a common method of evaluating agreement among observers. This method is most useful when observations are frequent, e.g., in this study.⁵

Table 2 ICCs for two observers regarding different coronary arteries (all P<0.0001)

Although coronary angiography may underestimate or overestimate a lesion’s functional severity, it is still the standard technique for guidance towards PCI and CABG. In other words, it allows physicians to judge the necessity for revascularization or continuing medical treatment. However, confounding factors, including vessel tortuosity, the overlap of structures, and the effects of stenosis shape, can cause gaps between the apparent severity of the lesions and corresponding flow limitation. These limitations are exacerbated by fluoroscopic images, which are essentially two-dimensional projections of the three-dimensional vessels.

Although high interobserver agreements do not necessarily assure that the observer is right in his or her judgment, it is certain that he or she could hardly be right if disagreeing significantly with the other two reporters. It is obvious that interobserver variability should be considered meticulously when the approach to the patient has been via ICA (invasive coronary angiography). On the other hand, the gold standard for diagnosing coronary stenosis, i.e., ICA, is an overall-reliable method for the detection of coronary lesions.⁶

A visually estimated diameter stenosis severity of ≥70% for non-LM disease and ≥50% for LM disease has been utilized to define significant stenosis and to provide guidance for revascularization strategies.⁷ Even though the length of a lesion may contribute to physiological lesion severity (e.g., a longer moderate lesion may result in more ischemia than a focal severe lesion), there are no standard cutoffs for lesion lengths used to classify a severe stenosis.⁷

One of the most important criteria considered by cardiologists that inclines their clinical decision toward revascularization vs. medical follow-up is the existence of significant lesions in proximal parts of coronary arteries rather than the distal parts per se. When the patient has a significant lesion in the proximal parts of the LAD, LCX, or RCA, the trend for PCI or CABG is obviously higher than that for significant lesions in the distal parts, for which most clinicians prefer to choose medical follow-up as the treatment of choice.⁶

The reasons for interobserver variability in coronary angiography should be considered from different points of view, e.g., the performance of the angiography, angiography equipment, and the observers. It is crystal clear that coronary arteries should be catheterized selectively for optimal resolution. Multiple projections are necessary to fully delineate stenotic lesions, as most plaques are eccentric rather than concentric and lesions that appear nonsignificant in one or two projections may appear significant in other projections. In other words, failure to get multiple necessary views during coronary angiography may leave stenotic vessels overlapped by normal vessels. Coronary artery spasms and coronary artery muscle bridges are other controversial issues in coronary angiogram reading that may be responsible for some degree of interobserver variability.

It is worthwhile mentioning that while some errors in diagnosis by coronary angiography have been found to be very real, they are not different from errors found in other areas of medical diagnosis where human judgment must finally be translated into quantitative terms. While the goal of studying observer variability is obviously to recognize and attempt to eliminate the source of errors, as the complexity of human judgment remains the basis for the final numerical values, some irreducible minimal variations will always have to be accepted.

Improper interpretation of angiographic images can result from inadequate projection views, CAAs (coronary artery anomalies), vessel foreshortening or superimposing branches, and deep engagement of catheters into the coronary artery, potentially resulting in oversight of the ostial lesions.⁵ Moreover, obesity instrument malfunctioning can lead to low image quality and erroneous image interpretations. Prevalence of CAAs (eg coronary fistulas) in patients undergoing coronary angiography is between 1%–5%. CAAs are the second–most common cause of sudden cardiac death in young athletes.⁸ A variety of techniques for intravascular imaging has improved the understanding of CAD and delivered additional information to invasive coronary angiography, especially in regard to intracoronary stenting.Actually, ICA is luminogram with poor specificity⁵ IVUS (intravascular ultrasonography) and OCT (optical coherence tomography) are the main imaging modalities currently available in the catheterization labs for intravascular imaging.

Coronary intravascular imaging such as IVUS and OCT can improve the detailed assessment and characterization of CAD and aid optimization of revascularization via stents (Figure 3). IVUS is done by an intracoronary catheter with a transducer at the tip, which creates ultrasound waves by converting electrical energy into acoustic energy.⁹ On the other hand, OCT, using light-wave technology, permits higher resolution than IVUS. In other terms, OCT delivers better definition of the endothelium and fibrous cap of atheroma, whereas IVUS has better penetration into the vessel walls and so can provide better information about the atheroma core.¹⁰ IVUS shows the three layers in vessel architecture — intima, media, and adventitia clearly. The ACC and AHA recommend IVUS use for assessment of indeterminate lesions in the LM (class IIa, level of evidence B) and non-LM (IIb, B) coronary arteries to determine the need for revascularization. IVUS is also recommended for stent implantation optimization.¹¹ On the other hand, OCT can yield information about which coronary atheromata are higher-risk, including those containing a large lipid core, thin fibrous cap, and high macrophage infiltration. OCT is the only imaging technique that allows a precise evaluation of the fibrous cap and macrophage content of atherosclerotic plaques.¹² It can determine which atheroma is adequately prepared for optimal stent deployment. Moreover, it can be quite helpful in stent selection, i.e., stent diameter and length.¹³ OCT can even identify intimal thickening, an early stage of atherosclerosis.¹⁴

Figure 3 Top row, coronary angiogram; middle row, IVUS; bottom row, OCT of the same coronary lesion.15

When reading coronary angiograms, borderline lesions may require multiple views and potentially intracoronary imaging or evaluation of fractional flow reserve (FFR) to adequately assess the severity of the lesion. FFR can be assessed during routine coronary angiography in order to determine quantitative functional stenosis. FFR measurement involves determining the ratio between the maximum achievable blood flow in a diseased coronary artery and the theoretical maximum flow in a normal coronary artery. FFR is determined by the ratio of pressure distal to the lesion/pressure proximal to the lesion under maximal hyperemia induced by pharmacological vasodilation.¹⁶ FFR values >0.75 are usually associated with good outcomes at 15-year follow-up if angioplasty is not performed on patients with stable ischemic heart disease and significant lesions in coronary angiography followed by medical therapy only.¹⁷ FFR has been proposed as the most important invasive tool to complement coronary angiography.¹⁸ Hybrid catheters that combine FFR-IVUS or IVUS-OCT have been used recently to provide more data from these dual technologies. When reading a coronary angiogram, the entire extension of every coronary artery and its branches should be carefully evaluated in all of the acquired views. Coronary dominancy should be investigated, i.e., right, left, and codominant.

The following elements should be part of the evaluation of diseased coronary vessels: extension and localization of the lesion (number of diseased vessels single-vessel disease, multivessel disease, involvement of the LM, number of lesions in each vessel and distance between lesions, lesion length, whether there is any ostial involvement, existence of bifurcation or trifurcation lesions); the severity of the stenosis, including estimation of minimal lumen diameter, visual assessment, and QCA (quantitative coronary angiography); morphological characteristics of the lesion, e.g., ACC–AHA lesion classification; evaluation of the downstream flow (the most common classification is the Thrombolysis in Myocardial Ischemia (TIMI) flow grade [TIMI 0 flow 100% occlusion of the vessel, TIMI 1 flow 99% stenosis in the vessel, TIMI 2 flow partial perfusion of the vessel, TIMI 3 flow complete perfusion of the vessel;¹⁹ evaluation of collateral blood vessel circulation (arteriogenesis, angiogenesis); and changes compared to previous angiograms if available, including extent of CAD progression between two angiograms and type and size of stents implanted during previous angioplasties.

Stenosis is defined as mild if the narrowing is usually <50%, moderate between 50% and 70%, and severe with diameter reduction ≥70%.²⁰ Evaluation of stenosis severity can be estimated by the cardiologist reading the angiogram or it can be measured with QCA, methodologies based on the selection of the area of interest and vessel-diameter measurements, which can be automatic or manual.

Most programs can be collaborated using the diameter of the catheter and can automatically detect the edge of the vessel across its lengths and measure the minimum diameter of the stenosis and length of the stenosis. Alternatively, instead of edge detection, the densitometric methodology can be used. This technique avoids the errors of edge reduction caused by geometric assumptions required for software calculations.

Densitometry measures the stenosis based on the area containing ICA when the vessel is fully opaque. There is usually good agreement between edge detection and densitometry techniques. QCA reduces interobserver variability in reading coronary angiograms, which is estimated to be around 20% and usually results in 10%–20% lower values than visual stenosis estimation.²¹

Conclusion

The main purpose of this study was to compare the variability among different reviewers in reading the same angiograms of 200 randomized patients in our center who had undergone coronary angiography during 2018 and 2019 by different staff interventional cardiologists. This research showed that the mean interobserver variability among the three observers was 22.6% and for two observers was 22.5%. The interobserver variability in proximal parts of the aforementioned vessels (LAD, RCA, and LCX) was found to be the lowest (i.e., highest agreement), while the midportion parts were the highest (i.e., lowest agreement), and intermediate results were achieved in the distal parts. Our aim has not been determining the accuracy of reviewers via comparison with functional flow studies, e.g., FFR, but just to determine the interobserver variability among cardiologists with different experience who had analyzed a high number of coronary angiograms previously.

It will be beneficial to carry out more interobserver-variability studies on same-day readings of each coronary angiography by different readers, which would decrease one of the confounding factors, i.e., reporting a specific stenosis that is equal regarding but different regarding the name of reported coronary artery. For instance, one of the reporters mentions 80% lesion in the proximal part of D1 and two other reporters mention the same stenosis as 80% stenosis in the proximal part of D2. In this case, reporters assessing the same stenosis but with discrepancy merely in regard to the naming of the diseased diagonal artery would be calculated as variability among observers that is not real. A practical solution for reducing this error could be using of coronary angiography report sheets that are prefilled with the name of each coronary branch so that the reporting cardiologists could just determine the percentage of narrowings for each predefined coronary artery.