A Mobile Application for Anticoagulation Management in Patients After Heart Valve Replacement: A Usability Study

Introduction

Heart valve replacement is the main treatment for severe heart valve diseases.¹ However, patients face risks of negative events such as thrombosis and embolism after valve replacement, making thrombosis prevention essential, and generally requires prolonged anticoagulant therapy. Warfarin is presently the most widely used oral anticoagulant, but it has the disadvantages of a narrow therapeutic window, large inter-individual dose variation, and susceptibility to diet and combined medication.² Patients who have undergone bioprosthetic valve replacement need to take warfarin for 3–6 months, while patients who have undergone mechanical valve replacement need to take warfarin for life.³ Presently, anticoagulation management of Chinese patients is mainly based on outpatient follow-up and self-management, regular blood sampling, medication guidance from doctors, and adjustment of warfarin dose according to the medication habits and experience of different doctors. However, outpatient doctors can only access single-time test results, making it difficult for them to understand the trend of previous continuous laboratory indicators, which are also affected by many factors. Therefore, the accuracy of doctors’ judgment is often affected, and the accuracy of warfarin dosing has a significant impact on patients’ postoperative rehabilitation and prevention of complications.⁴ Achieving precise administration of warfarin is the focus of anticoagulation management in the future.

Artificial intelligence (AI) technology has been widely used in medical diagnosis and treatment, especially in personalized precision medicine and new technology research and development.⁵ AI can learn and simulate human behavior based on data-driven algorithms, process massive and high-dimensional data, and effectively extract and manage a large amount of case information. Their role in medical diagnosis and clinical decision-making has attracted increasing attention, greatly improving medical quality and expanding the field of medical services.⁶

Based on the aforementioned situation, our team and the technical team of the Shanghai Heart Valve Engineering Technology Research Center, for the first time, innovatively developed a mobile application for anticoagulation management called Whole Life cycle management, which uses AI technology to assist patients in achieving individualized and precise anticoagulation management. It can accurately recommend the dosage of warfarin and achieve real-time online unified management. It also functions in the preliminary screening and warnings of adverse cardiovascular events. Therefore, this study aimed to evaluate the usability and user experience of this newly developed mobile application (app) for managing anticoagulant therapy in patients after heart valve replacement and the perceptions of physicians and nurses after using the app.

Materials and Methods

This study used a mixed-methods design to evaluate the usability of a fully lifecycle-managed anticoagulation management application by collecting quantitative and qualitative data. Battista A⁷ believes that mixed methods research can effectively fulfill the research purpose by collecting and analyzing data drawn from different research methods, using specific designs to organize research data, and integrating research results.

Participants

From June 1, 2023, to July 30, 2023, cardiac surgery patients, doctors, and nurses involved in clinical work at a tertiary general hospital in Shanghai were selected as our study participants. Eligible patients were aged 18 years or older, required anticoagulation after heart valve replacement, had understanding and communication skills, and could operate smartphones. The inclusion criteria for physicians and nurses were formally registered physicians and nurses in clinical practice for cardiac surgery. On the day of discharge, doctors and nurses jointly explained the recruitment notice and specific information on the entire life cycle anticoagulation management application to eligible patients. Patients willing to participate in the study were thoroughly informed about the purpose of this study, the process, and the specific use of the anticoagulation management application, and signed a written informed consent form.

Development of APP

The idea for the app came from a focus group discussion during a doctor-patient communication session with 24 patients, family members after heart valve replacement, and nine healthcare workers. During the communication process, they mentioned the current universality of mobile applications and the powerful advantages of AI technology. They also expressed challenges in anticoagulation self-management and their expectations for the emergence of intelligent and precise anticoagulation management tools in the future. Therefore, we developed a mobile application for self-management of anticoagulation, aiming to leverage the advantages of AI to achieve intelligent and precise anticoagulation management. To realize this idea, we collaborated with a medical team, contacted the Shanghai Heart Valve Technology Center, and explained our ideas. Simultaneously, recognizing patients as the main beneficiaries of our research, our research and development comprised the medical team, information technology team, and patients. First, we collected a large amount of data of patients who had received anticoagulant therapy from the hospital information system, mainly including basic information, diagnostic information, treatment plan, laboratory tests, etc., analyzed its characteristics, and preliminarily designed the network structure according to the characteristics of the existing historical anticoagulation data. Subsequently, according to the continuous input of anticoagulation data from patients, the gradient descent method was used in the background of the software to train the network model continuously while simultaneously evaluating and optimizing the trained recurrent neural network model. Finally, an anticoagulation management application was developed based on the obtained recurrent neural network model. Using this recurrent neural network model, the application can automatically recommend the patient’s next warfarin dose as long as the patient continues to input each dose taken. Therefore, this recommendation for precise drug dosing is the core of our entire application. Simultaneously, during the investigation of the functional requirements of the patient application, it was found that patients had obvious needs for laboratory reminders and medication guidance.

Consequently, we added laboratory reminders and medication guidance functions to designing the patient interface. At the interface of medical staff, personnel can view the patient’s medication data anytime and anywhere, ensuring the patient’s safety by verifying the software’s recommended drug dose. They can also communicate with the patients anytime to achieve unified online management. The international normalized ratio (INR) value and dose of each blood test can be accurately recorded, and a clear, dynamic trend map of the data can be formed. Additionally, we designed a preliminary screening questionnaire for adverse cardiovascular events based on a literature review in the early stages. The application of anticoagulation management can also realize the real-time analysis of various data uploaded by patients and regularly push the preliminary screening questionnaire for adverse cardiovascular events. Once the screening results indicate high risk, the application will automatically send an early warning to remind medical staff to focus on observation and intervention, ensuring comprehensive anticoagulation management for patients after heart valve replacement to ensure their safety. After extensive discussions and modifications by our research and development team, our mobile application software was finally composed of three modules: patient terminal, medical terminal, and background algorithm. The interactive interface was rich in content, including patient laboratory reminders, recommended doses of warfarin medication, anticoagulation medication guidance, doctor-patient communication, anticoagulation data trend reports, and high-risk warning reminders. Appendix 1 shows part of the user interfaces of the app.

Data Collection

Quantitative Data

A questionnaire survey was used to collect quantitative data. The System Usability Scale (SUS) is a usability assessment tool developed by Brooke in 1986.⁸ The research tool used in this study is the Chinese version of the SUS, which was translated and verified by Chinese scholar Wang Yuhui in 2019 (Appendix 2). It has demonstrated good reliability and validity and has been experimentally confirmed to be suitable for testing the usability of systems, websites, and applications.⁹ The scale comprises ten items, including the two dimensions of usability and ease of learning. Using the likert-5 scoring method, 5 points indicated full agreement, and 1 point indicated full objection. After scoring, it needs to be converted to a hundred-mark system; the higher the score, the better the usability.⁹ We completed the SUS test in the form of an electronic questionnaire (Appendix 3) distributed by the researcher via WeChat after participants had used the app for 2 weeks. Additionally, we collected the number of days during which patients uploaded their anticoagulation data to reflect app engagement.

Qualitative Data

Semi-structured interviews were conducted to collect qualitative data and further understand the app’s usability. The interview tool was a self-designed outline, initially designed according to the purpose of the interview. It was finally formed after a thorough discussion by the research team (Table 1). The interview was conducted in a quiet location, such as an outpatient conference room or a doctor’s office. Each interview lasted 20–30 minutes, with participants having used the app for 2 weeks before the interview. During the interview, changes in nonverbal information, such as the expressions and movements of the interviewees, were observed and recorded. The interviews were recorded, transcribed, and analyzed using qualitative software to identify key content and common themes. When the qualitative data reached saturation, the interviews were terminated.

Table 1 Interview Outline

Data Analysis

Quantitative Data

EXCEL software was used for data entry, and the statistical software SPSS version 25.0 was used for analysis. Descriptive statistics were employed to describe the characteristics of the participants and their SUS scores.

Qualitative Data

The seven-step data analysis method from Colaizzi’s¹⁰ phenomenological studies was adopted: (1) carefully reading all interview records, (2) extracting significant statements, (3) coding recurring ideas, (4) pooling the coded views, (5) writing a detailed, non-missing description, (6) identifying similar ideas and sublimating themes, and (7) returning to the respondent for verification. A professional recording pen was used to record the entire interview process, and the recorded content was transcribed into text data within 24 hours of the end of each interview. Qualitative data were imported into the professional analysis software NVivo11 for coding analysis to create themes. Any disagreements were resolved through discussions with the respondents until a consensus was reached.

Results

Quantitative results

Participants’ Characteristics

Eighty participants participated in the UE survey for usability evaluation, comprising 38 patients, 18 physicians, and 24 nurses. The average age of the patients was 52.40±13.33 years old, with 23 males (60.50%) and 15 females (39.50%). Regarding the educational level of the patients, 5 (13.15%) had primary school or below, 7 (18.42%) had junior high school, 16 (42.11%) had senior high school, and 10 (26.32%) had college degree or above. Among them, 25 (65.79%) were married, eight (21.06%) were unmarried, three (7.89%) were widowed, and two (5.26%) were divorced. In terms of living arrangements, 33 (86.85%) lived with family members, three (7.89%) lived alone, and two (5.26%) lived with others. Among the patients, 26 (68.42%) underwent mechanical valve replacement, and 12 (31.58%) underwent biological valve replacement. The average age of nurses was 32.40±8.35 years old, with three nurses (12.50%) holding junior college education, 20 (83.33%) having bachelor’s degrees, and one (4.17%) having master’s degrees. Regarding work experience, six nurses (25.00%) had 1–5 years of experience, eight (33.33%) had 6–10 years, eight (33.33%) had 11–20 years, and two (8.34%) had over 20 years experience. The average age of the doctors was 38.40±7.46 years old, with two (11.11%) doctors holding master’s degrees and 16 (88.89%) having doctoral degrees or higher. In terms of work experience, three doctors (16.67%) had 1–5 years of experience, five (27.77%) had 6–10 years, seven (38.89%) had 11–20 years, and three (16.67%) had over 20 years of experience. No patient experienced adverse events such as bleeding or thrombosis. Table 2 presents the participants’ general information.

Table 2 Participant Characteristics (N=80)

Usability Score

Overall, 38 patients, 18 physicians, and 24 nurses participated in the study and completed the SUS, resulting in a 100% response rate. The patient’s usability score was 82.37±5.45, with an ease of learning score of 84.21±12.22, with an overall usability score of 81.91±5.91. The nurses’ usability score was 81.88±6.44, with an ease of learning score of 80.21±12.18, and an overall usability score of 82.29±9.25. The doctor’s usability score was 84.17±5.82, with an ease of learning score of 81.95±13.71, and overall usability score of 84.72±9.52. Figure 1 shows the SUS score profiles of the patients, nurses, and physicians.

Figure 1 SUS score profiles of the patients, nurses, and physicians (higher scores reflect higher usability).

APP Usage

Among the 38 patients surveyed, two (5.26%) had no anticoagulant data upload records within 2 weeks after discharge. Nine patients (23.69%) had 1–7 days of anticoagulant data upload records within 2 weeks after discharge. Twenty-two patients (57.89%) had 8–13 days of anticoagulant data upload records within 2 weeks of discharge. Five patients (13.16%) had 14 days of anticoagulant data upload records within 2 weeks of discharge. Figure 2 shows the details of app use in patients.

Figure 2 Details of app use in patients.

Qualitative Results

Interviewees

Eighteen participants including 9 patients (50.00%), 3 doctors (16.66%), and 6 nurses (33.34%), were approached for a face-to-face interviews. Table 3 shows the characteristics of the participants.

Table 3 Participant Characteristics (N=18)

Results of the Interview

The theme was finally obtained through thematic analysis, and the details of the interviews are shown in Table 4.

Table 4 Details of the Interviews

Themes Based on the Patients’ Results

Theme 1: Application is easy to operate and learn.

Most respondents found the process easy and believed that as long as they could use a smartphone, they could use the application and had no difficulties in using the application.

Theme 2: Concerns about personal information security.

A few respondents expressed concerns about the security of their personal information. They believed that the professional and technical personnel of the hospital should do an excellent job in the related protection.

Theme 3: Great expectations for the recommendation of precise anticoagulation dose.

Most of the respondents expressed their willingness to try this application, hoping that it could genuinely assist patients in managing anticoagulant drugs. Additionally, they hoped that the recommended dose would be more accurate and the possibility of complications would be really reduced.

Themes Based on the Nurses’and Doctors’ Results

Theme 1: Elderly patients need family support to use the App.

Most doctors and nurses mentioned that the application was easier for young people. However, to ensure the accuracy of the uploaded data, older patients still required the assistance of younger family members for safety.

Theme 2: The app can help patients improve their anticoagulant self-management ability.

Most doctors and nurses observed that patients’ awareness and understanding of anticoagulation management improved after using the application, and their anticoagulant management abilities significantly improved.

Theme 3: The application made it convenient to provide transitional care at later stages.

Most nurses noted that the application brought great convenience to their follow-up management and continuous nursing work for discharged patients.

Discussion

A whole-life anticoagulation management application was made available for patients after heart valve replacement, and both quantitative and qualitative research results showed that the application was usable. In the quantitative study, the usability survey results showed that the SUS score of patients was 82.37±5.45, that of doctors was 84.17±5.82, and nurses was 81.88±6.44. Some studies⁹ believe that a SUS score ≥62.7 is acceptable, while a SUS score ≥68 is higher than the average level in terms of usability quality. In this survey, the total SUS scores of patients, doctors, and nurses are significantly higher than 68, indicating that all three groups consider the application usable. The application usage data showed that 94.74% of the patients participated in the use of the application, 71.05% of the patients were able to upload the dosage of anticoagulant drugs and the results of each test almost every day or every other day, 13.16% of the patients were able to upload the anticoagulation data every day. At the same time, we found that the patients who uploaded anticoagulation data daily were all patients after mechanical heart valve replacement, which showed that the vast majority of patients after heart valve replacement were willing to try to use the program, and some patients after heart valve replacement showed even greater interest. This may be attributed to the need for lifelong anticoagulation in patients after mechanical valve replacement,¹¹ as there is a greater risk of bleeding after valve replacement than after biological valve replacement. In the qualitative study, interview results showed that most patients expressed that the application was easy to operate and learn. Under the guidance of doctors or nurses, patients quickly learned the application method and did not encounter difficulties in subsequent use. However, in interviews, doctors and nurses also mentioned that the application was relatively simple for young patients. Still, it needed to upload laboratory data and drug doses, which significantly impacted the recommended dose of anticoagulant drugs. To ensure the accuracy of the anticoagulation data, some elderly patients may need the help of young family members for subsequent use of the application. This result is consistent with the quantitative study results, which showed that the ease of learning scores in the SUS of patients were higher than those in the SUS of doctors and nurses. Simultaneously, a report¹² on the research and development status of smart-home-based telemedicine technology for older people also mentioned that family members play an essential role in the elderly participating in smart-home-based telemedicine technology projects. Therefore, elderly patients need to seek the help of family members in the use of the app to ensure the accuracy of the uploaded anticoagulant data and, at the same time, to improve the accuracy of subsequent anticoagulant drug recommendations. The SUS score results and APP usage data showed that the application was feasible, and patients were willing to try the application and showed great interest in it.

The application of anticoagulation management throughout the life cycle can meet the needs of intelligent and precise anticoagulation management for patients after heart valve replacement. The results of qualitative research showed that patients had high expectations for precise anticoagulation dose recommendations, hoping to reduce complications such as bleeding or embolism. Studies¹³ have shown that anticoagulation medication guidance based on AI recurrent neural network can improve the accuracy of anticoagulation prediction, thereby reducing the dosing error to achieve the target INR value. Safe and effective INR control is an important determinant of anticoagulation complications.¹⁴ Therefore, the precise recommendation of the anticoagulation dose can be an important means to reduce the complications of anticoagulation, whereas international normalized ratio (INR) self-monitoring management is another important determinant for improving INR stability and reducing the incidence of complications.¹⁴ The interview results also mentioned that the application could help patients improve their anticoagulant self-management ability. Previous studies¹⁵ have shown that reminders, medication guidance, and other app functions can improve patients’ self-management behaviors. Other studies¹⁶ have suggested that the use of AI on mobile devices can improve adherence to anticoagulation therapy and change patient behavior. Therefore, the results of previous studies^15,16 are consistent with those of our study. The use of whole life cycle anticoagulation management applications can make anticoagulation monitoring and management of patients after heart valve replacement more convenient. The qualitative results also indicate how to further optimize and promote the application and development of anticoagulation management applications in the future. It is suggested that hospital professionals should focus more on information protection with the widespread application of information technology in healthcare, as ensuring information security is indeed an important topic. Ensuring the security of patient information is an essential condition for carrying out information technology-based health care.¹⁷ App developers should provide users with clear privacy policies that guarantee patient privacy and the security of personal information. The interview results of nurses also suggested that the application brought convenience to the later development of transitional care. Studies¹⁸ have shown that transitional care based on mobile APP can significantly improve the self-efficacy, quality of life, and nursing satisfaction of stroke patients. Therefore, we can believe that transitional care based on a whole life cycle anticoagulation management application will also produce excellent income effects and truly benefit patients. Artificial intelligence technology has long been a global research hotspot. A study¹⁹ using various machine learning techniques to construct a dynamic anticoagulation regimen for hospitalized patients after heart-valve replacement surgery has shown significant optimization of anticoagulation quality.Compared with the previous dose according to the subjective recommendation of doctors, artificial intelligence technology can greatly meet the needs of patients with intelligent and precise anticoagulation management. The application of whole-life-cycle anticoagulation management assisted by artificial intelligence technology also provides a smarter, more convenient, and more scientific anticoagulation management mode for patients after heart valve replacement.²⁰

Through a literature review, we found that few studies have applied AI technology to anticoagulation management, which has shown great potential in addressing anticoagulation management issues in patients who often face challenges after heart valve replacement. Anticoagulants are still essential drugs for patients after heart valve replacement, but patients after heart valve replacement have been plagued by problems such as narrow therapeutic window, susceptibility to external factors and, various complications.The whole-life anticoagulation management application leverages the powerful advantages of AI to accurately assess the efficacy of anticoagulation therapy and recommend warfarin doses using recurrent neural networks. It also features functions including patient blood test reminders, anticoagulation guidance, doctor-patient communication, anticoagulation data trend report, and high-risk warning reminders, greatly improving the timeliness and safety of anticoagulation and reducing the probability of adverse events. Therefore, these materials have considerable application prospects and value. In addition, this application can be extended to all patients requiring anticoagulant management, such as those who undergo atrial fibrillation and other cardiac surgeries.

Limitations

This study only addressed the availability of whole-life anticoagulation management applications in patients after cardiac valve replacement. It did not explore the effectiveness of the application in improving adverse cardiovascular events after cardiac valve replacement. In addition, an investigation will be conducted in the future to determine whether there are differences in safety and effectiveness between this application and traditional anticoagulant management methods. Moreover, the sample size involved in this study was small, and more research subjects will be included in the future. At the same time, we will continue to pay attention to user feedback after the application is used and continue to improve and update according to the feedback suggestions to meet the needs of more patients and clinicians.

Conclusion

Through this study, we confirmed the feasibility of whole-life anticoagulation management in the management of patients after heart valve. To further develop the application of anticoagulation management, making full use of the iteration of artificial intelligence to continuously improve the accuracy of the recommended dose of medication and simultaneously obtaining family support and ensuring the information security of patients is the direction of our continuous efforts.