Validation and Final Results from the First Cardiac Lead Post-Approval Study Using Real-World Data

Introduction

The US Food and Drug Administration (FDA) has recognized the growing importance of real-world data (RWD) and real-world evidence (RWE) in clinical research and regulatory decision-making and has emphasized the potential of RWD to reduce the need for costly long-term post-market clinical trials.¹ With this growing acceptance of RWD and evolving analytic techniques to generate real-world evidence (RWE) within clinical research, a collaboration of cardiovascular implantable electronic device (CIED) industry (Abbott Laboratories, BIOTRONIK, Inc., Boston Scientific Corporation, LivaNova PLC, and Medtronic, Inc.), academia, Heart Rhythm Society, American College Cardiology, and the FDA called the Electrophysiology Predictable and Sustainable Implementation of National Registries (EP PASSION) informatics working group was formally initiated in August 2017.

Although national databases such as the Manufacturer and User Facility Device Experience (MAUDE) and the Medical Product Safety Network (MedSun) aim to characterize medical device performance in real-world clinical settings by collecting data on adverse events and device issues, they rely heavily on participant reporting, similar to manufacturers’ product performance reports. EP PASSION, which has been described previously,² was formed with the mission to address limitations of traditional long-term post-market clinical trials (such as high costs, lengthy time to study initiation and completion, and high follow-up attrition) by developing sustainable and more efficient approaches to fulfilling FDA Condition of Approval Post-Approval Studies (PAS), specifically for cardiac leads, through the use of RWD. An example of a cardiac lead PAS is BIOTRONIK’s Solia S clinical study (Clinical Trials.gov ID: NCT01791127),³ which evaluated the long-term safety and reliability of the Solia S (formerly named Siello S during pre-market period) pacing lead as an FDA condition of market approval following the completion of the investigational device exemption portion of the study. The Solia study was ongoing at the inception of the EP PASSION project. As a result of this initiative, BIOTRONIK, Inc. (Lake Oswego, OR) was able to successfully transition the Solia study and other ongoing cardiac lead post-approval studies from a traditional long-term prospective clinical study with clinical sites and direct patient follow-up to utilizing RWD and RWE methodologies established by the EP PASSION project. This article focuses on the proof-of-concept results and the final clinical report results of this first FDA-approved EP PASSION PAS for a cardiac lead, Solia S.

Methods

Prior to FDA approval to transition the Solia study to the EP PASSION model, proof of concept analysis was performed to evaluate the feasibility and estimate the accuracy of using RWD to identify chronic cardiac lead complications by comparing the rates of lead-related complications derived from RWD to the rate of corresponding adverse events reported in the traditional PAS. This retrospective analysis was conducted utilizing real-world evidence methodology developed in conjunction with and oversight by the FDA and has been approved by an institutional review board. The research reported in this study adhered to the guidelines set forth by the Office of Human Research Protection that is supported by the US Department of Health and Human Services.

Data Sources

After evaluation of required outcome measures for cardiac lead PAS and available data sources, the EP PASSION informatics working group selected fee-for-service (FFS) Medicare insurance claims data obtained through the CMS Chronic Conditions Warehouse (CCW) as the primary RWD source fit for the event type and patient population of interest. To augment Medicare insurance data, manufacturer-specific device tracking data collected for CIED systems per FDA Guidance⁴ including device registration and Medical Device Reporting (MDR) data was used to associate lead complications with the lead of interest. Table 1 below lists a brief description of each data source.

Table 1 Real-World Data Sources

Patient Population

All patients registered in BIOTRONIK’s device tracking database and implanted with at least one study lead (Solia S) on or between March 13, 2013, and July 20, 2015, who met the following criteria at the time of data analysis were included in the study:

1. Matched a unique beneficiary ID in the Medicare claims dataset
2. Fee-for-service Medicare coverage during the month of the study lead implant (FFS coverage defined as: Part A and Part B coverage, with no Part C coverage)
3. ≥1 month of FFS Medicare coverage
4. ≥1 Medicare claim in the inpatient, outpatient, or carrier files during the study observation period

Using linking methods based on exact matches using patient identifiers such as social security number or combination of partial social security number, name, date of birth, patient reported gender, and zip code, Medicare FFS beneficiaries with a study lead were identified within the Medicare datasets by ResDAC, a CMS contractor that provides technical assistance with CMS data.⁸ Medicare data from these patients were then combined with BIOTRONIK device tracking data and made accessible to licensed BIOTRONIK personnel within the CMS Virtual Research Data Center (VRDC). Only licensed persons on the data use agreement who have been given access by CMS may utilize the VRDC, and all datasets exported from the VRDC required review and approval by designated CCW personnel to confirm export met the CMS cell suppression policy.⁹

Identification of Chronic Lead Complications

Medicare fee-for-service claims data were analyzed to identify potential chronic lead complications requiring invasive intervention using a combination of diagnosis and procedure codes from the International Classification of Diseases, Ninth and Tenth Revisions, Clinical Modification (ICD-9-CM, ICD-10-CM), and Current Procedural Terminology^® (CPT).

Complication-defining diagnosis codes include diagnoses relevant to lead-related complications, such as lead dislodgements, mechanical breakdown, etc, or other relevant clinical events related to the pulse generator or implanted system. Complication-defining procedure codes include invasive interventions such as lead replacement, lead repositioning, replacement of cardiac device, etc. which might be performed to resolve a lead or generator complication. Complications occurring within 30 days of implant were excluded as these complications were considered to be acute events and likely associated with patient-specific anatomy and/or the implant procedure. The list of diagnosis and procedure codes of interest are listed in Supplementary Tables S1 and S2.

Potential study lead complications were identified from claims containing at least one complication defining diagnosis code and one complication defining procedure code. For each claim containing a complication of interest, the dates and details of the complication-defining procedures were identified and compared with data from BIOTRONIK’s device tracking database to identify the system component and implant location associated with the complication (Figure 1). For example, if a lead complication claim matched the date of a study lead date of explant and either no other devices or leads were explanted or sufficient device tracking data was available, it was concluded that the complication was “attributable” to the study lead. Complications that could be linked to other system components were considered “not-attributable” to the study lead and excluded from analysis. Complications that could not be linked to a specific component were considered “possibly attributable” to the study lead.

Figure 1 Lead complication flowchart.

Validating Lead Complication Methodologies

To validate EP PASSION methodologies for identifying lead complications, a proof-of-concept exercise was performed. The proof-of-concept patient population included Medicare FFS patients that were enrolled in the Solia study and met the patient eligibility criteria described above (Figure 2). To standardize events between the Solia study and Medicare claims cohort for event rate comparisons, the subset of adverse events that could practicably be detected in claims data (ie, complications resolved by invasive intervention) were selected as relevant lead complications for evaluation. These lead complications resolved with invasive intervention identified in the Solia study were then compared to the lead complications identified using the EP PASSION methodologies.

Figure 2 Venn diagram of proof-of-concept patients.

Statistical Analysis

The Solia EP PASSION PAS was designed to be similar to the traditional cardiac lead PAS with endpoints evaluating study lead complication-free rates from implant through 5 years post-implant.³ Incidence of lead complications associated with complications “attributable” and complications “possibly attributable” to the study lead were evaluated. Kaplan–Meier estimates were constructed for each analysis using the following censoring rules:

1. Patients with continuous 100% FFS coverage who remain implanted with an eligible study lead will be censored as of the claim’s dataset cutoff date.
2. Patients who remain implanted with an eligible study lead and are 100% FFS as of day 1826 post-implant will be censored as of day 1826.
3. Patients without continuous 100% FFS coverage will be censored as of the first day of the month when Part A and/or B coverage is lost or Part C (Medicare Advantage) coverage is initiated.
4. Patients who have all eligible study leads explanted without a new eligible study lead implanted and the explant is not due to a study reportable complication will be censored at the date of explant. Lead explant will be determined using the Medicare database and/or BIOTRONIK’s device tracking database.
5. Patients will be censored at their reported death date using the Medicare database.

The proof-of-concept analysis utilized a 2 × 2 contingency table to evaluate the relative sensitivity and specificity of the Solia study lead complications against the EP PASSION methodologies. For this analysis, any subject who was both in the Solia study cohort and had FFS Medicare coverage from implant through the duration of their PAS follow-up period was included. McNemar’s test was performed to determine if the difference in relative sensitivity and relative specificity between Medicare claims data compared to the PAS data source was statistically significant. All analyses were performed with SAS 9.4 analytical software (SAS Institute, Cary, NC).

Results

Proof of Concept Results

A total of 896 patients enrolled in the Solia study with FFS Medicare coverage from implant through the duration of their PAS follow-up period were included. The EP PASSION methodologies identified all 12 lead complications reported in the Solia study plus an additional three lead complications that were not reported in the PAS. Three patients had two lead complications, therefore, a total of 899 cases (sum of patients with no event and number of events) were analyzed for diagnostic agreement utilizing McNemar’s test applied to a 2 × 2 contingency table. The result showed no statistically significant difference (p = 0.0833). There was a 99.7% proportion of overall agreement between both methods (896/899), including agreement on negative and positive event status as seen in Table 2.

Table 2 2 × 2 Diagnostic Agreement Test

Final Solia EP PASSION PAS Results

Overall, 57.4% (1015/1767) of patients implanted with a study lead during the Solia study enrollment period of March 13, 2013, and July 20, 2015, met all eligibility criteria that were included in the RWE analysis (Figure 3). Solia S leads can be implanted in the right atrial (RA) and right ventricle (RV) of the heart; therefore, a total of 826 patients were implanted with two study leads making the total count of evaluable study leads 1841 (839 with a RA implant location and 1002 with RV implant location).

Figure 3 Solia S EP PASSION cohort.

The mean age and standard deviation of the patients was 77.3 ± 8.3 years with an age range of 39 to 98. The majority of the patients reported their gender as male (54.7%) and their race as white (91.4%). The total follow-up time was 7228.3 lead-years. A total of 33 chronic lead complications “attributable” or “possibly attributable” to the study lead including all lead locations through 5 years post-implant were identified in 33 out of 1841 study leads for a rate of 0.005 complications per lead-year. Figure 4 provides the Kaplan–Meier actuarial graph for these lead complications. An estimated freedom from the last complications identified at 1521 days is 98.1% (SE 0.39%; 95% CI: 97.3%, 98.8%). The complication-free rate at 5 years post-implant using the 1174 study leads with either 5-years of follow-up or a lead complication is 97.19% (1141/1174; 95% CI: 96.1%, 98.1%).

Figure 4 Kaplan–Meier Survival Curve for the attributable and possibly attributable Solia S lead complications.

Discussion

These real-world methodologies were developed as part of the EP PASSION project, which included a validation step (ie, proof-of-concept analysis) designed to estimate the accuracy of using Medicare administrative claims data and device tracking data to identify “true” chronic cardiac lead complications. Results of McNemar’s test showed no statistically significant difference in relative sensitivity or specificity between methods (p = 0.0833), although evaluation of the 2 × 2 contingency table showed three events identified with RWE methodologies were not identified in the Solia study. Further evaluation of these three complications revealed that these events were missed in the Solia study due to patient loss to follow-up (LTF). This finding suggests additional missed events occurred in the unmatched PAS patients and an analysis of the full PAS cohort may have resulted in a superiority results for RWD methods over traditional methods. LTF is an unfortunate but expected occurrence in all clinical studies as patients may transfer care to a non-participating clinical site for a variety of reasons. This underreporting of events due to LTF highlights a benefit of using RWD, such as Medicare data, as the patient is less prone to LTF as long as they maintain Medicare FFS coverage. There was a 99.7% proportion of overall agreement between the two data sources (896/899) including agreement on negative and positive complication event status. The results of the Solia EP PASSION proof-of-concept analysis validated the Medicare claims-based model, which produced very similar results to the Solia study, demonstrating the feasibility of using a claims-based approach to collect long-term lead safety data. With approval from FDA on April 15, 2019, the traditional Solia study protocol was updated to document the transition of subject-level adverse event data reporting through long-term clinical follow-up to a RWD collection model based on identifying lead-related complications from Medicare claims data and manufacturer’s device tracking and Medical Device Reporting (MDR) data. The updated protocol received Advarra Institutional Review Board approval (Columbia, MD) and after successful transition to EP PASSION, final results of the Solia S EP PASSION PAS were submitted on October 14, 2021, with FDA approval of the long-term safety and reliability of the Solia S lead.

In addition to the Solia study transition, subsequent BIOTRONIK studies, including Protego DF4 PAS (NCT02243696)¹⁰ and QP ExCELs PAS (NCT03155724),¹¹ which evaluated a defibrillation cardiac lead and a left ventricular cardiac lead, respectively, were successfully transitioned to utilizing methodologies developed in EP PASSION. As a result of transitioning these studies, the resource burden on clinical sites to recruit and manage long-term follow-up in a PAS was eliminated and translated to a significant resource savings for BIOTRONIK as well as clinical research infrastructure. This resource savings along with the improved follow-up attrition, shortened study duration, and opportunity to expand patient populations compared to traditional PAS conduction confirmed that the goals of EP PASSION were met. In turn, this new approach to PAS using RWD creates opportunities to shift site and sponsor resources to further investment in innovation and pre-market trials. Other CIED observational studies have since utilized CMS data with similar methodologies such as a validation study of Boston Scientific’s HeartLogic PAS results¹² and Medtronic’s leadless pacemaker studies (ClinicalTrials.gov ID: NCT03039712¹³ and NCT04235491¹⁴), Abbott’s leadless pacemaker studies (ClinicalTrials.gov ID: NCT05336877,¹⁵ NCT05932602,¹⁶ NCT06100770¹⁷), and BIOTRONIK’s ICD lead PAS, Pamira.¹⁸ The FDA’s approval of these methodologies demonstrates future applications for new products going through the FDA approval cycle.

Limitations

It is important to note that while RWD has the potential to increase sample size, reduce costs, produce results that are more generalizable, and is suitable for post-market surveillance studies, there will always be a need for traditional clinical studies, particularly for feasibility and interventional based trials, due to the limitations of RWD. For example, while lead complications were successfully identified using Medicare claims data, the level of complication detail is not comparable to a traditional study as the claims-based information is collected for billing purposes and not research. Currently, these methods do not provide enough details to confidently identify specific types of lead complication. For example, a diagnosis code of T82.110 indicates “breakdown (mechanical) or cardiac electrode” which lacks information to classify the complication further such as an insultation breach, fracture, subclavian crush, etc. With the implementation of ICD-11-CM and future revisions, the level of granularity within Medicare claims data will continue to improve and increase feasibility of identifying specific lead complication types.

As the initial goal was to transition existing PAS to this RWD approach, the enrollment window of the PAS was implemented which limited patient inclusion in the analysis. Additionally, these methods required invasive intervention to be considered a lead complication as these are the most crucial complications to capture, but further refinement, such as incorporating remote monitoring data from devices, may allow for capturing complications that were addressed electrically (through device programming) or left in use despite a known issue. Further, this method was developed to evaluate chronic lead-related complications (defined as those occurring more than 30 days after implant) and the current method does not support evaluation of acute complications – something that can be refined in future analyses. Although data fields used within the Medicare claims data can impact reimbursement and are therefore generally higher quality, potential coding errors still exist.

Lastly, the methodology described supports analysis of a subgroup of patients, specifically those with Medicare fee-for-service insurance. While we demonstrated in a separate proof-of-concept exercise that the patient demographic did not have a statistically significant difference in mean age when comparing the EP PASSION cohort to the Solia PAS cohort, Medicare data inherently reflects an older patient population as eligibility for the program requires US patients to be 65 years of age or older. Different RWD sources may need to be explored to better evaluate lead complication rates in younger patient populations and Medicare patients using a Medicare Advantage plan.

Conclusion

The Solia study was successfully transitioned from a traditional clinical site-based study data collection model to a Medicare claims-based approach, which confirmed the long-term safety and efficacy of the Solia S lead. The study marks a significant milestone as the first completed PAS employing RWD and RWE methodologies for long-term safety assessment of a cardiac lead. The transition to RWE represents a promising advancement in cardiac lead surveillance, demonstrating the potential of routine clinical data for more efficient post-market device evaluation.