Development, Implementation, and Assessment of an Online Modular Telehealth Curriculum for Health Professions Students

Introduction

COVID-19 and the associated broadening in insurance coverage and relaxation in regulatory policies led to a rapid expansion in the provision of telehealth care across disciplines and professions.^1,2 These changes required providers from various health professions to develop a skillset for providing virtual care for which they had no prior training.³ Several professional organizations have either developed competencies or expressed the need for telehealth competency adoption in health professions training programs.^4–6 Review of these competencies demonstrates a great deal of overlap between professions. However, no standardized telehealth curriculum exists, and most existing curricula are designed as electives for small cohorts of interprofessional students or are directed to a single profession.^7,8 Developing a telehealth curriculum for health professions students presents several challenges. First, there is limited available curricular space for implementing stand-alone telehealth courses. Second, given telehealth’s relatively sudden expansion into “mainstream” healthcare delivery, educators have limited expertise and evidence at their disposal to create formal learning experiences.³ Logistically, health professions programs utilize various schedules and learning management systems, and due to the proprietary nature of each, developing telehealth course content is complex and requires attention to compatibility for successful tracking of student course participation, evaluation, and completion. Given the necessity of telehealth training for health professions students, development and implementation of telehealth curricula that address the stated challenges is needed.⁹ We describe the development, implementation, and pilot assessment of an online, modular telehealth curriculum designed specifically for health professions students.

Methods

Curricular Design

Beginning in January of 2021, we utilized Kern’s six-step approach to curricular design to develop a modular, online, interactive telehealth curriculum designed for health professions students as part of a 12-month faculty development program. Kern’s six steps include (i) problem identification and general needs assessment, (ii) targeted needs assessment, (iii) goals and objectives, (iv) educational strategies, (v) implementation and (vi) evaluation and feedback.¹⁰

For the general needs assessment, we reviewed the literature for existing telehealth curricula searching PubMed using the following query: (((curriculum OR curricul* OR course OR coursework OR education OR class OR classes) AND (telehealth OR telemedicine OR telemed OR “tele health” OR “tele med*” OR telemed*)) AND (student OR learner)) AND (integrat* OR standards OR standard* OR competencies) with a return of 817 articles. Of these, 44 were selected by the authors for more detailed review based on their relevance and currency to the specific needs assessment being performed. Additionally, the internet was searched using publicly available search engines for educational videos on clinical telehealth. This revealed gaps in telehealth curricula designed for health professions students, including lack of a standardized program, single profession design, and relegation to a specific educational experience.^7,8 Additionally, online resources were geared towards practicing clinicians, as opposed to learners, and generally lacked attention to the logistical aspects of telehealth. A targeted needs assessment was conducted by contacting key educator stakeholders within each of the 5 health professions programs at our institution to understand if and how telehealth education was provided in each program. This assessment yielded results that were similar to the literature review, demonstrating that while all programs included some telehealth training, experiences varied greatly with respect to content, depth of exposure, and number of students reached. These stakeholders also believed that this curricular development could be pursued in alignment with the institution’s priority for interprofessional education and collaboration. Additionally, in reviewing telehealth competencies endorsed or proposed by different health professions organizations, there was significant overlap, suggesting that a multi- and interprofessional design may present an efficient way to provide a standardized curriculum for teaching the clinical essentials of telehealth.^4–6 Finally, by approaching this curriculum with an interprofessional perspective, planned future expansion modules can teach the logistics of interprofessional team-based telehealth building on the principles taught in the original modules.

Given this needs assessment, we recognized an opportunity to develop a telehealth curriculum targeting health professions students and developed goals and objectives for the planned curriculum. Our primary goal was that by the end of the curriculum, students would have increased confidence in describing and performing the logistics of engaging in telehealth visits and performing virtual assessments in children and adults. Additional specific goals were developed based on the scope of each of the 9 telehealth modules produced (Box 1) and each mapped to matching telehealth competencies previously discussed.

Box 1 Web-Based, Interactive, Telehealth Modules

From July 2021 through July of 2022, we produced 9 online interactive modules. For the educational strategy, we opted for an online, interactive modular design that allowed for flexible integration of one or more modules into existing curricula of health professions programs. Additionally, such a design has been shown to be effective for achieving cognitive objectives for developing knowledge and understanding in learners, utilizes reusable learning objects, and allows self-paced engagement and learning that is consistent with principles of andragogy as described by Knowles.¹¹ In creating the content, we specifically focused on providing practical aspects of telehealth delivery including the logistics of preparing and engaging in a telehealth encounter and performing and coaching patients on virtual physical assessment. We developed the modules with an interprofessional team composed of 1 nurse, 1 physician assistant, 1 occupational therapist, 1 dentist, 1 audiovisual specialist, and 11 physicians representing various specialties and disciplines who were locally recruited because of their reputation for experience and expertise in providing clinical care through telehealth. A subgroup of the team was assigned to each module, based on their specific expertise in that module’s topic. Each of the interprofessional teams used its collective knowledge, and any resources identified by the panel from the available literature, to develop the content and script for the module. The proposed content for each module was then reviewed by 2 physicians, 1 physician assistant, and 1 nurse practitioner (authors M.H., M.B., N.H., and E.L.) who represent the leadership of the Duke Health Center for Interprofessional Education and Care in addition to the authors D.O. and K.W. to ensure module content aligned with the listed objectives and mapped to at least one of the AAMC telehealth competencies for entering residency.¹² Following this, a video was produced and then turned into a module on the interactive video platform Playposit (Playposit, Inc.), which integrates with the two learning management systems at our institution (Canvas and Sakai) and can also be accessed extramurally through WordPress, a web content management platform (sites.duke.edu/telehealthessentials). The final production was again reviewed by both the content development team and the interprofessional leadership team for final approval prior to implementation. In terms of construction, while the modules were designed to stand alone, each follow a similar template, educational style, and use of health profession inclusive language.

Survey Measures

Following development, modules were integrated into courses within the MD, PA, and NP programs at Duke during the 2022–2023 academic year. All 9 modules were added to the MD fourth-year Capstone course as an elective experience. Both the PA and NP programs integrated the logistics of performing a telehealth visit module and the general virtual physical assessment module into their curriculum as required modules for all students. These 2 modules utilized by students from differing health professions programs provided the data for a subgroup analysis assessing differential efficacy of the curriculum for different types of health professions students as described below. At the conclusion of each module, there were two embedded Qualtrics surveys that measured student self-assessed confidence in performing the virtual assessment concepts taught in the module, knowledge gained in the module, and likelihood of using the skills in the future. These measures were selected to assess both student self-efficacy and attitudes toward telehealth following engagement with the modules as these aligned with the main objectives of the overall curriculum and individual modules. Surveys were based on the assessment tool used by Cornes et al in their publication describing development of a telehealth workshop.¹³ In addition to the above measures, participants were asked to provide a free-text response to the survey question, “Do you have any other feedback for us on this module?” One survey was designed for the logistics of performing a telehealth visit module and the other was designed to assess the system-based virtual physical assessment modules. Student completion of the surveys was encouraged but not required and students provided electronic informed consent as the surveys provided a radio button where they could opt-out from having their survey and module analytic data included in our study. The purpose of the assessment was to provide learning outcome efficacy data for the student participants. The Duke Institutional Review Board (IRB) reviewed all materials and identified the study as category 1 exempt from full IRB review (Pro00110478) finding it to be education research that is unlikely to adversely affect students’ opportunity to learn required educational content. Using the survey data from Qualtrics, we summarized the student reported measures of confidence and knowledge gained after watching a module and compared these outcomes between professions (MD, NP, and PA). This analysis was performed to assess whether learners from different professions reported differential outcomes which might indicate content bias. The data were analyzed using R 4.2.2 for Windows (R Development Core Team, Vienna, Austria). Since the questions were on a Likert-type scale, we utilized the numeric value of the responses, ie, 1 = not at all confident to 5 = very confident, and calculated the median responses (25th percentile – Q1, 75th percentile – Q3) for each health profession and for the overall cohort. Reporting the median with quartiles was chosen over reporting the mean due to its more accurate representation of the distribution of Likert scale data with a narrow range. We used the Kruskal–Wallis test to compare whether the median score differed between professions. We excluded participants who did not consent or answer any questions from analysis.

Results

(Table 1 and Figure 1) represent the results of the questions from the survey associated with the “Practical Approach to Performing a Telehealth Visit” module. We recorded a total of 142 survey responses. After excluding those who did not consent or answer any survey questions, 70 remained for analysis for a response rate of 49%. We observed that the median reported confidence scores were between 4 and 5 in all the questions (corresponding to a response of moderately to very confident). The median self-assessed knowledge gained was 3.0 (corresponding to “moderate knowledge” gained), and the median likelihood of utilizing the telehealth skills taught in the module was 5.0 (corresponding to “extremely likely”).

Table 1 Descriptive Statistics from Self-Assessed Confidence and Knowledge Gained After Reviewing the “Practical Approach to Performing a Telehealth Visit” Module

Figure 1 Bar graphs of results from self-assessed confidence and knowledge gained after reviewing the “Practical Approach to Performing a Telehealth Visit” module.

We similarly summarized the Qualtrics survey data associated with the 8 system-based virtual physical assessment modules. We recorded a total of 211 responses, each reflecting engagement with one of these modules. A single learner may have completed one or more surveys depending on the number of modules taken. After excluding those who did not consent, 125 remained for analysis. Of note, student responses to the system-based module evaluations did not identify which module was being evaluated and so virtual physical assessment module evaluations were analyzed as a single group. Across all professions, the median reported confidence in performing the virtual exam skills taught in the module and likelihood of skill utilization was 4.0, corresponding to “moderately confident” and “somewhat likely”, respectively (Table 2 and Figure 2). The median score for knowledge gained was 3 indicating “moderate knowledge gained”.

Table 2 Descriptive Statistics from Self-Assessed Confidence and Knowledge Gained After Reviewing the System-Based Telehealth Modules

Figure 2 Bar graphs of results from self-assessed confidence and knowledge gained after reviewing the system-based telehealth modules.

In a sub-group analysis, we compared responses between students from different health professions, including the “Practical Approach to Performing a Telehealth Visit” module survey. NPs demonstrated a statistically significantly lower median response than MDs or PAs in their reported confidence to “prepare your workspace to create a supportive clinical environment” (4.0 vs 5.0, p = 0.002) and reported they were less likely to “utilize telehealth skills in future practice” (4.0 vs 5.0, p = 0.039). Otherwise, we found no statistically significant differences in responses between professional programs for the surveys associated with the system-based physical assessment modules. The number and scope of free-text comments were limited, prohibiting meaningful qualitative analysis and are not reported.

Discussion

In developing and piloting this telehealth curriculum, we attempted to provide a learning experience uniquely designed for health professions students. While there are other telehealth education resources available, there are several aspects of this curriculum that, to our knowledge, are novel. First, as mentioned, competencies for telehealth are widely overlapping for several health professions and our targeted needs assessment uncovered an opportunity to develop an efficient shared resource that could be used flexibly and interprofessionally both in categorical and interprofessional courses. The content developed for each module reflected this interprofessional education approach as each was developed by clinical educators representing 2 or more health professions with experience and specialization relevant to the specific module being developed, with additional oversight provided by a team comprised of institutional leaders of interprofessional education from the School of Nursing, School of Medicine, and the Physician Assistant Program. Such an approach helped to ensure that the content and language presented were inclusive and as widely applicable to multiple professions as possible. For future modules, it would be ideal to have even greater diversity and a more balanced representation of health professionals.

Next, with regard to teaching the telehealth content, the modules present not only clinical practical modeling of principles and techniques but there is a focus on providing logistics on how to perform them, including pragmatic technical details on patient positioning, camera angles, and use of technology. By adopting this approach, the goal was to create specific content that, if not directly applicable to every health profession, still provides content that has broad relevance.

The modular, web-based, and interactive design allows for flexibility and adaptability in implementation and delivery of curricular elements. Additionally, a previous multi-institutional web-based modular telehealth curriculum for medical students and family medicine residents demonstrated that learners found a web-based design to be acceptable to learners and they were able to successfully gain knowledge and skills.⁸ As mentioned, while the principles taught in the modules are designed to appeal to a wide range of health professions students, the system- based modules may, admittedly, be less applicable to certain health professions. The modular design enabled course directors to select the most relevant modules for their specific health profession to provide or augment existing telehealth education for their students. Also, given the scarcity of curricular space, having stand-alone modules offered a curated and efficient method for integrating into health professions programs while still providing standardized materials shared by all disciplines. This flexibility afforded by the stand-alone modular design was evidenced by the module usage pattern of the programs that implemented the curriculum. Both the nursing and physician assistant courses only required the “Logistics” and “General Assessment” modules be completed, with other modules available for elective review, while the medical student course required completion of all modules for credit. Without this flexibility, it is likely that successful implementation of the curriculum in multiple health professions programs would have been challenging. Finally, the modular nature of the curriculum allows for ongoing curricular expansion and development. Of note, 2 additional modules teaching interprofessional team-based telehealth have been developed and implemented as of May 2024.

With regard to assessment of the curriculum, our pilot demonstrated that students from multiple health professions were confident in performing the logistics of providing telehealth virtual care and performing virtual physical assessments after completion of the online, modular, telehealth curriculum. Additionally, students reported that completing modules provided them with a moderate knowledge gain and provided content relevant to their future practice. While there were two questions showing statistical differences between responses for students from different health professions, differences were small and of questionable practical significance, suggesting that overall the curriculum was effective for and relevant to multiple health professions when integrated at points in existing courses and clerkships within participating programs. We believe the interprofessional collaborative approach was responsible for its multi-profession pertinence. We suspect that the interactive, virtual design facilitated successful adult learning.

When analyzing results from the assessment of the curriculum, there were limitations. First, the study included a relatively low response rate, inclusion of only a single institution, and outcome measures focused on student perception rather than outcomes that track incorporation of knowledge and application. Additionally, students did not complete a pre-test survey as pragmatically we were limited to a single touch point with the students. We also did not perform a retrospective pre-test survey, although they were asked to rate their knowledge gained following module completion, which provides some insight into students’ self-assessment of knowledge acquisition. Additionally, given the anonymity of the responses, lack of resources, and the logistical challenges of bringing students together, focus groups were not practical. However, next steps, as discussed below, hope to address these limitations.

Given the positive results of initial implementation, there are several next steps we are taking. One step is to integrate the modules as part of simulated or actual clinical telehealth experiences along with tools designed to measure student application and performance of telehealth principles. This will allow for more objective measurement of knowledge and skills and application through direct assessment, rather than student self-report, of the curriculum’s ability to improve the cognitive, affective, and psychomotor objectives being taught. Finally, given the curriculum’s ability to be utilized outside of our institution, scaling would provide access to health profession and IPE programs and students at a national level, helping to address the calls for competency-based telehealth education. Challenges include maintenance of resources for curricular upkeep and administration, identifying additional educational and clinical partners to provide IPE telehealth experiences so all health professions students utilize the same curriculum in the course of their training, and responding to telehealth policy changes with the end of the public health emergency.

Conclusion

The tragedy of the pandemic had very few positives, but the expansion of healthcare access through generalized use of telehealth was one of them. Curricula, such as the one described, focused on providing clinical essentials of telehealth across professions as part of existing courses and experiences will help ensure future clinicians are prepared to utilize this alternative patient care platform. This study suggests an interprofessional collaborative telehealth curriculum using an online modular approach can convey the necessary skills and be relevant to students from various health professions. Future research will provide supporting evidence for the efficacy of online modules for teaching this important topic to health professions students.