Development and Validation of Virtual Reality Cognitive Training for Older Adults with Mild Cognitive Impairment: Protocol for a Mixed-Methods Program Evaluation Study

Background

Mild cognitive impairment (MCI) is a prevalent cognitive impairment affecting older individuals.^1,2 It is characterized by more pronounced cognitive difficulties in older adults compared to their peers of the same age, particularly in memory and reasoning abilities.³ These cognitive changes disrupt their engagement in daily activities and are noticeable to family and friends.^4,5 Recent research suggests a correlation between elevated levels of depression and reduced financial capacity in individuals with MCI when depression is present simultaneously.^6,7 Mild cognitive impairment is widely viewed as a preclinical stage of early dementia, indicating a higher likelihood of progression to dementia.³ However, studies indicate that older adults with MCI may have the potential to return to healthy cognitive function or delay the onset of dementia.^8–10 Longitudinal research shows that older adults with better cognitive performance^8,9 or those who actively participate in task-oriented activities¹⁰ are more likely to transition from MCI to healthy cognition. These findings highlight the potential for improving cognitive performance in older individuals with MCI, suggesting that cognitive training could be one effective strategy to help restore their cognitive function.

Cognitive training (CT) is a non-pharmacologic approach that involves structured and controlled exercises to improve specific cognitive functions. It is frequently used to enhance cognitive abilities in older adults, focusing on reversing cognitive decline, promoting cognitive stability, and maintaining independence.^11,12 Research has demonstrated that CT can improve overall cognitive performance, processing speed, memory, working memory, and executive function in older adults with MCI.^13–16 Additionally, studies suggest that integrating robot programming with a tangible and manipulative system could be beneficial in CT for individuals with MCI.¹⁷ However, it remains uncertain whether improvements in certain cognitive regions translate into enhanced performance in instrumental activities of daily living (IADL) for older adults with MCI.^13,18,19 The concepts of “near transfer” and “far transfer” describe the extent to which benefits gained from training in one activity can be applied to different tasks.^20,21 By aligning training activities more closely with real-world tasks and situations, there is potential for individuals to transfer the skills and abilities acquired through CT into their everyday lives. Nevertheless, few CT programs for older adults are designed around real-world activities.

Virtual reality (VR) is increasingly used to enhance and sustain physical,²² cognitive,²³ psychological,²⁴ and social skills²⁵ of older adults. As described by Burdea and Coiffet (2003), VR entails a high-end user-computer interface involving real-time stimulation and interactions of an embedded subject through multiple sensorial channels (eg, visual and auditory, sometimes haptic, even smell and taste, if possible), based on a synthetic environment in which the subject feels his presence.²⁶ The concept of presence refers to the subjective experience of existing within a virtual world, where individuals perceive the artificial environment as if it were real.^27,28 Virtual reality is classified into three types based on the level of immersion experienced by participants and their connection to the physical world: fully immersive, semi-immersive, and non-immersive. Fully immersive VR isolates participants from the physical environment and immerses them in a three-dimensional setting with multiple sensory inputs.^25,29 Enhanced immersion can intensify the sense of presence, which improves participant engagement and training effectiveness.^23,27 Immersive and lifelike VR environments can elicit appropriate behavioral responses, facilitating the application of skills learned in virtual settings to real-life situations.²⁹ Moreover, VR possesses several features that make it particularly advantageous for developing CT programs for older adults: (1) VR provides adaptable and safe environments for older adults,^25,29,30 (2) VR improves the transfer of training to real-life scenarios through lifelike surroundings,^29,31,32 (3) VR boosts motivation and engagement during training sessions,^25,29,32 and (4) VR offers tailored training programs for older adults.^{25,29,32–34}

Current evidence supports the effectiveness of fully immersive virtual reality cognitive training (fi-VRCT) as a beneficial intervention for enhancing cognitive abilities in older adults with MCI. Many studies have demonstrated improvements in various cognitive domains, including attention, processing speed, memory, working memory, visuospatial ability, executive function, global cognition, and language.^31,35–39 However, older adults with MCI may face challenges in transferring newly acquired skills from interventions to real-life tasks and situations.²⁰ Notably, one study reported improvements in IADL ability following fi-VRCT.³⁶ Other studies indicated that older adults generally enjoy using VR.^40–42 Feedback from older adults with MCI suggests they often feel more relaxed, comfortable, active, and adventurous when engaging with fi-VR, experiencing reduced anxiety, tension, and worry afterward.^40,42 Side effects or disorientation were minimal and rarely reported.^40,41 The engaging and enjoyable nature of fi-VR greatly enhances motivation, engagement, and adherence to CT, thereby positively influencing VR training outcomes.

While research on fi-VRCT in older adults with MCI is increasing, more evidence is needed to comprehensively assess its effectiveness. Specifically, there is a gap in fi-VRCT programs focusing on daily living tasks and IADL within existing gamified interventions. Furthermore, a deeper understanding of the long-term effects of fi-VRCT on older adults with MCI is necessary. Emotion and subjective experiences play crucial roles in learning and skill transfer,^43,44 yet few studies have thoroughly examined the impacts of emotional responses and personal experiences during fi-VRCT. Hence, this study aims to (1) develop and assess the feasibility of a fi-VRCT program centered on IADL for older adults with MCI, (2) implement and evaluate the effectiveness of this fi-VRCT program in older adults with MCI, and (3) investigate the underlying mechanisms of the fi-VRCT program related to IADL in older adults with MCI and refine this intervention accordingly.

Methods and Analysis

We propose a comprehensive longitudinal study comprising three phases: feasibility, intervention, and extension, to achieve our objectives. The feasibility phase will focus on developing a fi-VRCT and assessing its applicability and acceptability in real-world community settings. In the intervention phase, we will implement the fi-VRCT through a cluster-randomized controlled trial involving older adults with MCI. This trial will assess the effects of this fi-VRCT on cognitive and everyday functioning. The extension phase will incorporate participant feedback to explore the mechanisms behind training outcomes and further refine the fi-VRCT. This study will utilize a mixed methods program evaluation design as described by Creswell and Clark (2017)⁴⁵ (Figure 1). We will apply an exploratory core design from the feasibility to the intervention phase, followed by an explanatory core design from the intervention to the extension phase. This integrated approach will provide valuable insights that surpass what could be obtained from purely quantitative or qualitative methods, thereby providing a solid foundation for addressing our research objectives with robust evidence.

Figure 1 Flowchart of research design and the relationship among study phases.

Feasibility Phase

Development of the fi-VRCT

This intervention will be designed as a series of cognitive activities woven into a narrative centered around preparing meal for significant others. The cognitive journey starts at home, where participants select their dining companions and the meals they wish to share. They then familiarize themselves with the recipes, confirming and memorizing the necessary ingredients. Next, they move to a marketplace, where they have to skillfully identify and select the correct ingredients. Returning to the kitchen, participants review the recipe, memorize the cooking techniques, and determine the sequence for adding each ingredient. Following the recipe instructions, they proceed to prepare the meals step-by-step. The initial choices introduce a psychological element to enhance motivation, and as the narrative unfolds, the activities progressively increase in difficulty based on the participants’ decisions. The selection of dinner companions and dishes plays a crucial role in determining the overall challenge level of the activity.

Evaluation of the Applicability and Acceptability of the fi-VRCT

Procedure

Participants for a 70-minute fi-VRCT session will be recruited from various community settings, including community centers, senior centers, and adult daycare centers. Each session will include a 10-minute warm-up with instruction, three 15-minute training segments, two 5-minute eye massages, stretching breaks between the training segments, and a 5-minute wrap-up. Researchers will conduct interviews to gather insights into participants’ experiences and will administer questionnaires to evaluate their sense of presence, immersion levels, and potential for cybersickness. This evaluation will help assess the applicability and acceptability of the fi-VRCT for older adults with MCI.

Participants

Table 1 outlines the estimated number of participants along with the specific inclusion and exclusion criteria for recruitment. We will employ convenience and snowball sampling methods to recruit participants from various community settings, such as community centers, senior centers, and adult daycare centers, utilizing referral networks or flyer-based recruitment strategies.

Table 1 Participants Recruitment and Inclusion/Exclusion Criteria Across Three Study Phases

Apparatus: Meta Quest 3

The Meta Quest 3 provides a fully immersive VR experience with a head-mounted display and two handheld controllers. It offers 110° horizontal and 96° vertical field of view, featuring 2064 2208 pixels per eye and a smooth 90 hz refresh rate. Equipped with integrated stereo speakers with 3D spatial audio, tactile feedback controllers, as well as hybrid computer vision and machine learning sensors that track users’ gestures, the device significantly enhances the immersive experience. During the intervention, participants will wear the headset and use the controllers to actively interact with the virtual environment. The VR system provides substantial benefits due to its engaging and interactive features.

Qualitative Data Collection and Analysis

The study will use in-depth, semi-structured individual interviews to collect qualitative data on participants’ experiences, emotions, and perspectives regarding the fi-VRCT. Open-ended questions will be used to encourage detailed responses, and meticulous records will be kept to ensure data accuracy and completeness, including audio recordings, verbatim transcriptions, and interview notes. Data will be analyzed using thematic analysis, following Braun and Clark’s (2012) guidelines.⁴⁶ This systematic procedure includes data familiarization, initial code generation, refinement, reorganization, and identification of overarching themes. The final report will present these themes through an analytical narrative, supported by relevant examples and quotes. To facilitate data management and organization, Microsoft Excel and MAXQDA will be utilized for data collection and preliminary analysis. This process ensures the rigor and thoroughness of the analysis by developing a standard codebook and visualization tools.

Quantitative Data Collection and Analysis

Table 2 presents quantitative measures, including those for cognitive screening and VR experiences. Descriptive statistics will be used to summarize participants’ characteristics and their responses to the questionnaire. Statistical analyses will be performed using IBM SPSS Statistics software package, version 26, for Windows (IBM Corp., Armonk, NY, USA).

Table 2 Quantitative Measures Used in the Three Study Phases

Intervention Phase

Study Design and Procedure

This double-blind, cluster-randomized controlled trial is designed to assess the effectiveness of fi-VRCT on cognition and daily functioning in older adults with MCI. In this phase, we will employ convenience and snowball sampling methods to recruit participants from community centers, senior centers, and adult daycare centers. Recruitment will involve referrals and flyers, with individuals from the same facility randomly assigned to either the fi-VRCT group or the control group. The intervention will consist of 16 sessions over eight weeks, with two sessions per week, each lasting 70 minutes. Assessments will be scheduled by evaluators at the beginning, immediately after the intervention, and during a three-month follow-up, while ensuring blindness to participants’ group assignments.

Power Analysis and Sample Size Estimate

Drawing from fi-VRCT studies, effect sizes for cognitive performance vary from small to medium (d = 0.080–1.315; after transforming, f = 0.040–0.657),^35–38 while effect sizes for everyday functioning are large (d = 0.87; after transforming, f = 0.435).³⁶ Using a conservative estimate of a medium effect size (f = 0.25), this study will implement repeated measures analysis of variance (ANOVA) with a within-between interaction design for two groups and three measurements to determine the required sample size. With a statistical power of 90% and a significance level of 0.05 for a two-sided test, a total of 36 participants (18 in each group) is needed. Accounting for a potential attrition rate of 30%, the anticipated number of participants is 52 (26 in each group).

Participants

Table 1 displays the estimated number of participants along with the specific inclusion and exclusion criteria for recruitment.

Intervention

Each group, whether intervention or control, will consist of 2 to 3 individuals in a group setting.

fi-VRCT Group

The fi-VRCT, developed during the feasibility phase, aims to enhance attention, memory, and executive function. Each session will begin with a 10-minute instruction and warm-up, followed by three 15-minute training segments, each separated by two 5-minute breaks. A 5-minute wrap-up will conclude each section. Participants in the fi-VRCT will use the Meta Quest 3 head-mounted display and handheld controllers, with the intervention being administered by an experienced research assistant.

CT Control Group

The CT control program will feature cognitive board games aiming at improving various cognitive capacities, including attention, memory, and executive function. The duration and training dosage will be equivalent to those in the fi-VRCT group. This group will be led by certified community occupational therapists.

Data Collection

Table 2 shows the training outcome measures, which include assessments for cognitive screening, cognitive performance, and functional status. Participants will complete these assessments at baseline, immediately after the intervention, and during a follow-up assessment three months later.

Data Analysis

Descriptive statistics will be used to present participants’ demographics and characteristics. To identify baseline differences between the two groups, independent t-tests and chi-square tests will be conducted. Non-significant findings will suggest effective randomization allocation. A two-way repeated measures ANOVA will be performed to evaluate the effects of the intervention at three key time points: baseline, post-intervention, and during a follow-up assessment three months later. This analysis will evaluate the significant effects of group, time, and the interaction between group by time. The significance criterion for all tests will be set at a probability level of 0.05. If a significant main effect or interaction is observed, a Bonferroni post hoc test will follow. Paired t-tests will be used to assess changes within each group before and after the testing at each time point.

The effect size will be calculated to gauge the magnitude of the intervention impact on the outcomes. Cohen’s d, will represent the main effect, with effect size values of 0.2, 0.5, and 0.8 indicating small, medium, and large effects, respectively. The effect size of the interaction will be presented as partial eta squared (ŋ²), with classifications of small (ŋ² = 0.01), medium (ŋ² = 0.06), and large (ŋ² = 0.14).⁵⁷

We will conduct mediation analyses to clarify the relationship between an independent and a dependent variable, mediated by a third variable, known as a mediator. Two parallel mediation analyses will explore the influence of pre-intervention global cognition on post-intervention functional independence and mobility, considering mediator variables, such as age, sex, and pre-intervention processing speed. The mediation analyses will be performed using the PROCESS functions for SPSS developed by Hayes (2018).⁵⁸ Specifically, we will employ PROCESS Model 4 for the parallel mediation model. All mediation models will be thoroughly analyzed using 5,000 bootstrap samples and 95% bias-corrected confidence intervals. Statistical analyses will be conducted using IBM SPSS Statistics software package, version 26, for Windows (IBM Corp., Armonk, NY, USA).

Extension Phase

Procedure

We will use a purposeful sampling method, allowing individuals who participate in the fi-VRCR group during the intervention phase to continue into the extension phase. Both qualitative and quantitative data will be collected and synthesized to gain a comprehensive understanding of the intervention outcomes and to refine the fi-VRCT. The qualitative exploration will provide in-depth insights into participants’ responses to the outcomes and their adherence to interventions, thereby elucidating the reasons behind the training results. Furthermore, qualitative feedback will help identify factors that either facilitate or hinder the success of the fi-VRCT. This information is essential for improving cognitive interventions and preparing them for broader dissemination and implementation across various community contexts. The quantitative data will complement the insights gathered through qualitative analysis, providing additional perspectives and enhancing the overall understanding of the intervention effectiveness.

Data Collection

Qualitative Strand

Following implementing the interventions, we will conduct focus group interviews to gain a comprehensive understanding of participants’ experiences, feelings, perceptions, and thoughts regarding the fi-VRCT. Each focus group will comprise participants from the same intervention group who have been engaged for two months. This arrangement will facilitate the sharing of nuanced and authentic feedback among participants and promote dynamic discussions. Through the use of open-ended questions, we aim to elicit detailed descriptions of experiences, while follow-up and probing questions will help uncover previously undisclosed information. The interviews will be audio-recorded and transcribed verbatim.

Quantitative Strand

Quantitative measures related to VR experiences are outlined in Table 2. Participants will be required to complete these questionnaires at several intervals, including the initial, ninth, and final intervention sessions.

Data Analysis

Qualitative and Quantitative Data Analysis

The analysis process during the extension phase will closely align with the methods utilized in the feasibility phase. The qualitative data will be analyzed through thematic analysis, while the quantitative data will be subjected to descriptive statistics and paired t-tests.

Synthesis of Qualitative and Quantitative Findings

During the mixed stage, integration will be achieved through the use of joint displays, which will organize findings from both qualitative and quantitative approaches conducted during the intervention and extension phases. The themes identified from the qualitative interview analysis will help explain the quantitative findings obtained in the intervention phases. Joint displays will juxtapose these results, connecting and presenting the findings. This visual representation will enhance the analysis and comparison of information from various perspectives, facilitating the extraction of overarching conclusions that extend beyond either the quantitative or qualitative strand alone.⁵⁹

Ethics and Dissemination

This study will be conducted in community settings and has obtained ethical approval from the Research Ethics Committee at National Taiwan Normal University (202312EM009). Participants will be required to provide informed consent prior to enrollment. Researchers will give a clear explanation of the research content and address any questions that participants may have prior to obtaining formal consent.

Participants will be interviewed to gather their experiences, feelings, and thoughts regarding the fi-VRCT, and these interviews will be audio-recorded for subsequent analysis. Additionally, participants will complete questionnaires related to their VR experiences, cognitive function, and functional daily self-care abilities. Researchers will maintain confidentiality regarding all identifiable records and personal data, ensuring that participants’ identities are protected in published research results. Participants have the autonomy to decide their involvement and can withdraw consent at any time during the study without needing to provide reasons, ensuring that their rights remain unaffected.

The aim of this research is to develop and validate an effective fi-VRCT and share the findings with the community. To achieve this, we will conduct a feasibility study and a cluster-randomized controlled trial. Initial findings will be presented at international conferences, while primary results will be published in reputable, peer-reviewed journals to reach both academic and clinical audiences. We are dedicated to communicating our findings with clarity and transparency, using formal and concise language to reflect professionalism and expertise while preserving the original intent and meaning of the text.

Discussion

This trial utilizes a mixed methods approach to evaluate both the process and effectiveness of the intervention. By comprehensively gathering data from the target population, the study aims to quantitatively measure the intervention effects while qualitatively exploring the participants’ experiences. This approach will aid in developing a CT program tailored for older adults with MCI in community settings. The findings of this study will significantly contribute to the establishment of an evidence-based and practical intervention for cognitive enhancement in older adults with MCI.

While this study has numerous strengths, it also has certain limitations. One such limitation is cluster randomization, which may lead to discrepancies in key sociodemographic and other background variables among groups at baseline. However, any variations will be appropriately accounted for during statistical analyses. Additionally, participant dropout is a concern. Given that the study encompasses an eight-week program with assessments at three distinct time points, participants have the option to withdraw from the program or subsequent assessments. Anticipating a potential dropout rate of 30%, we will adjust the projected number of participants accordingly.

Conclusion

This program evaluation study is expected to yield valuable insights into the processes and outcomes of the fi-VRCT. We will share the lessons learned with program planners, implementers, and policymakers. The findings from the process evaluation can be utilized to inform future design, implementation, and enhancements aimed at benefiting older adults with cognitive impairment.