Impact of Simulation-Based Surgical Training in Laparoscopy on Satisfaction Level and Proficiency in Surgical Skills

Introduction

Laparoscopic surgery has become a mainstay in various disciplines, including General Surgery (GS). For the last 30 years, laparoscopic techniques have been commonly used in surgical procedures. The technique’s many benefits have led to its widespread adoption. However, effectively performing laparoscopic surgery requires specialized training and practice. As the demand for laparoscopic surgery grows, the operating rooms become a less conducive learning environment as they balance cost and efficiency.^1–3 In recent years, there has been a noticeable shift towards placing greater importance on surgical education in and outside the operating rooms. Institutions and organizations have recognized the need to provide comprehensive and specialized training programs to ensure that surgeons in training are competent to perform the required skills in their respective fields. This increased emphasis on surgical education aims to foster excellence, innovation, and patient safety in the surgical community.^4,5

In 1997, the Society of American Gastrointestinal and Endoscopic Surgeons (SAGES) created a standardized program to assess the skills and knowledge required for basic laparoscopic surgery. These programs provide a safe and appropriate environment for training and education without compromising patient safety.^6,7 Nowadays, passing the Fundamentals of Laparoscopic Surgery (FLS) test is a requirement for all GS residents applying for the American Board of General Surgery or Obstetrics and Gynecology.^8,9 FLS is a standardized and validated test that is a convenient tool for evaluating the minimum laparoscopic competency level required of surgical trainees before graduation.^10,11 It is based on five tasks performed in a simulated setting, including peg transfer, circle cut, loop ligation, intra- and extra-corporeal knot tying.

The acquisition and mastery of fundamental laparoscopic skills by surgical trainees are key objectives in the General Surgery (GS) training programs in Saudi Arabia, as outlined in the curriculum set forth by the Saudi Commission for Health Specialties (SCFHS).¹² However, the current level of laparoscopic proficiency among GS trainees in Saudi Arabia remains to be discovered. Two studies were conducted to gauge the subjective satisfaction of surgical residents in Riyadh and Jeddah with their surgical training and skills, yielding disheartening results: 65–80% of residents expressed dissatisfaction with their hands-on training.^13,14 A more recent mixed-methods study provides further elaboration to this issue, showing that trainees who were satisfied with their hands-on surgical experience were significantly more likely to report overall satisfaction with their training programs.¹⁵ These findings underscore the critical role of hands-on training in shaping the quality and effectiveness of surgical education. Despite these insights, there is a lack of objective data assessing the laparoscopic skills attained by GS trainees in Saudi Arabia. This gap in knowledge points to the need for reliable metrics to evaluate and improve training outcomes. Medical educators have long emphasized the value of incorporating simulation-based surgical training into programs to address these concerns and enhance the quality of education.^16–18

The objectives of this study is to focus on and uncover the subjective GS residents’ laparoscopic training satisfaction and self-appraisal of proficiency level in specific laparoscopic surgical skills, as well as to obtain an objective evaluation of laparoscopic proficiency through performing the tasks dictated by the FLS test. Our aim is to provide objective reference data that can guide improvement efforts in laparoscopic surgical training in Saudi programs.

Methods

This cross-sectional study was approved by Alfaisal University Institutional Review Board (IRB #20084) and received an internal research grant from Alfaisal University (Grant #21308). The study was conducted in two parts: an online survey and an objective evaluation of laparoscopic skills using standardized FLS task testing.

First, an Email was sent to all general surgery residents in the seven governmental training hospitals in Riyadh, Saudi Arabia, through the general surgery program secretary and program directors. (Appendix 1) The Hospitals included King Faisal Specialized Hospital, King Abdulaziz Medical City, King Fahad Medical City, King Saud Medical City, King Khalid University Hospital, Armed Forces Hospital, and Military Hospital. The Email included the details of the study purpose, scope all ethical consideration related to confidentiality, as well as a link to the survey. The survey aimed to capture the participants’ subjective self-appraisal of laparoscopic skills and experience, as well as information regarding laparoscopic training load and satisfaction during residency training. Responses were collected between October 2021 and August 2022. Participants were also invited to voluntarily participate in the objective evaluation of their laparoscopic skills. Those who agreed were directed to a separate page to provide their contact information. They were also provided with videos demonstrating each task they expected to perform at least one day prior to their participation.

The objective evaluation was conducted using standardized FLS task trainer and testing criteria. All equipment used was obtained from Limbs and Things Inc., the official supplier of authentic FLS equipment and materials. Participating residents were invited to take the mock FLS test in person at the Department of Clinical Skills at Alfaisal University between January 2022 and May 2023. Each participant was assigned a unique study ID and evaluated according to the FLS 2019 manual testing criteria¹⁹ in five tasks: peg transfer, circle cut, loop ligation, intra- and extra-corporeal knot tying. Scores were calculated using the McGill Inanimate System for Training and Evaluation of Laparoscopic Skills (MISTELS) reported in prior studies.^20,21 The score of 270 was adopted as the laparoscopic competency passing score based on McGill’s experience in evaluating residents and medical students.²² The Participants were allowed optional second attempt with one hour of supervised practice in between, and both attempts were scored separately. The tests were conducted in the presence of an expert medical educator and an FLS-certified general surgeon. The best of two trials for each task was used as the participant’s score for the purpose of data analysis.

The online survey structure was constructed in a manner similar to a published survey aimed at measuring self-appraisal of laparoscopic skills.²³ To establish face validity, an expert surgeon and a medical educator reviewed the survey. Subsequently, Cronbach’s alpha was used to assess the reliability of scores. Statistical analyses were performed using SPSS v29. Categorical data are presented as numbers and percentages, while continuous data are presented as means ± standard deviations. Comparative analyses were conducted using t-tests to compare participants’ mean scores. Analysis of covariant was used to determine the significance of confounding variables. Chi-square was used to compare proportions and Fisher’s exact test was alternatively used when Chi-square conditions could not be met due to the small number of participants who participated in the FLS test.

Results

A total of 70 residents, representing approximately 36% of the estimated outreach population (n=195), participated in the online survey. The study sample consisted of an equal proportion of male and female residents (50% each), with a slight majority being junior residents (PGY1-2, 51%). The mean age of the participants was 28.1 ± 2.1 years (Table 1).

Table 1 Demographics. The Demographic Data of the 70 Participants That Participated in the Online Survey

The survey elements for both training satisfaction and self-rated proficiency had good reliability scores with Cronbach alpha scores of 0.8 and 0.89, respectively. The survey revealed that satisfaction with the academic laparoscopic teaching and the intraoperative laparoscopic hands-on experience were 24% and 44%, respectively. On the other hand, a higher proportion of residents (62%) were satisfied with the case volume they encountered during their training (Figure 1A). Notably, 57% of the trainees reported participating in less than five laparoscopic cases per month. Overall, 37.1% of residents were satisfied with their laparoscopic training, while 24.2% were unsatisfied.

Figure 1 Self-reported Satisfaction Levels with Surgical training, Laparoscopic Skills and Operative Proficiency. (A) Satisfaction with Laparoscopic Training. (B) Self-Reported Comfort Level with Basic Laparoscopic Skills. (C) Self-Reported Comfort Level with Performing Common Laparoscopic Operations.

In terms of self-assessed proficiency levels, the residents reported relatively high satisfaction with basic laparoscopic skills such as the use of endo-scissors (68%), depth perception (58%), and hand-to-hand coordination (40%). Conversely, their satisfaction was low with their advanced skills, including extra-corporeal knotting (19%) and intra-corporeal knotting (7%) (Figure 1B). Residents were also asked to rate their comfort level in performing common laparoscopic surgical procedures. About a third of the participants reported feeling comfortable performing simple laparoscopic procedures independently, such as laparoscopic cholecystectomy and laparoscopic appendectomy (Figure 1C). Furthermore, only 34% of the residents had been exposed to laparoscopic trainers, of them 10 residents (14%) reported having obtained laparoscopic training certification such as the Fundamentals of Laparoscopic Surgery (FLS) certification. The survey also investigated the residents’ most practiced methods of training and preparation for laparoscopic surgeries. Simulation-based laparoscopic training was the least reported method for practice (17%), while watching educational videos was the most common approach (70%) (Table 1).

Senior residents – PGY 3 to 5 – reported higher levels of satisfaction with basic laparoscopic skills than junior residents, namely, depth perception, hand-to-hand coordination, tissue manipulation, and laparoscopic cutting (p= 0.001–0.031). Satisfaction with extra- and intra-corporeal knotting was equally low and not statistically significant between senior and junior residents. Senior residents reported higher monthly participation in laparoscopic cases than junior residents. However, it did not reach statistical significance (p= 0.098). Residents who reported participating in 5 or more laparoscopic cases per month – whether senior or junior residents – had a higher overall satisfaction rating of the training program, caseload, laparoscopic teaching, and intraoperative involvement (p= 0.001–0.029). They also reported higher satisfaction with their surgical skills (p= 0.003–0.037) except for extra- and intra-corporeal knotting. Prior simulation-based laparoscopic practice or obtaining formal laparoscopic training certification did not statistically affect the subjective responses regarding their satisfaction with the training or the subjectively reported satisfaction with their laparoscopic skills.

To objectively assess the residents’ laparoscopic skills, 14 trainees voluntarily participated in the FLS testing. Ten were senior residents, and four were junior residents. Five reported utilizing laparoscopic simulation-based practical training, and three had obtained prior laparoscopic training certification, namely FLS certification. A total of 90 task trials were recorded as some participants opted to take the optional second chance on the same task. The failure rate to complete or exceed the allowed time for each task was 36% (Table 2). Although senior residents tended to do better than junior residents, the difference in task failure rate was not statistically significant (31% versus 48%, p=0.16). Also, the differences in task failure rates between those who reported participating in five or more monthly laparoscopic cases and the rest were not statistically significant (34% versus 39%, p=0.65). On the other hand, those who reported utilizing simulation practice or obtained laparoscopic training certification completed the tasks in the allocated time 84% of the time compared to 47% of the time for the rest (p=0.003). In fact, those with prior FLS certification could pass all tasks either from the first or the second trials, although some tasks scored low. All four candidates who passed the FLS test - by obtaining a score higher than the FLS proficiency score of 270 - reported utilizing simulation practice or obtained laparoscopic training certification, while those who did not all failed, with the difference leaning towards statistical significance (p=0.069) (Table 2).

Table 2 FLS Task and Participant Results

Although senior residents obtained better task scores in the peg transfer (p=0.035) and extra-corporeal knotting (p=0. 04) the final score was not statistically different between junior and senior residents (p=0.2). Residents who reported participating in 5 or more laparoscopic cases monthly did not perform better, statistically, than those who reported less in all tasks. On the other hand, having prior simulation-based practice or laparoscopic training certification (n=7, 50%) had a statistically significant impact on most task scores as well as the total score (260.28 ± 53.8 versus 109.97 ± 73.91, p<0.001) (Table 2). This significance remained statistically relevant after controlling for PGY level as well as the reported monthly operative load (p=0.003).

Discussion

Our study evaluated the subjective satisfaction and self-appraisal of proficiency in laparoscopic skills among General Surgery (GS) residents in Saudi Arabia alongside an objective assessment using the Fundamentals of Laparoscopic Surgery (FLS) test. Although the study population was largely based in the capital, Riyadh, the city contains most GS trainees in the country. The findings reveal several key insights into this cohort’s current state of laparoscopic training.

Firstly, the subjective satisfaction with laparoscopic training among GS residents is limited. Only 37.1% of residents expressed overall satisfaction with their laparoscopic training, with notable dissatisfaction in their advanced skills, such as extra-corporeal and intra-corporeal knotting. These results align with previous studies conducted in Riyadh and Jeddah, where many surgical residents also reported dissatisfaction with their hands-on training in laparoscopic procedures.^13,14 Interestingly, despite the reported dissatisfaction, a significant proportion of residents (approximately 62%) were satisfied with the case volume encountered during their training. This discrepancy suggests that while residents are exposed to a reasonable number of laparoscopic cases, the quality or structure of the training may be insufficient to foster proficiency in more complex skills. Interestingly, residents who reported attending laparoscopic surgeries more frequently had higher reported satisfaction with their training and skills, regardless of their level of training. Nonetheless, this did not always translate to better results when tested objectively.

Although only fourteen residents participated in the objective evaluation part using the FLS test, the results provided a more nuanced understanding of the residents’ proficiency. The failure rate for task completion or time exceedance was 36%, indicating a substantial gap in competency for many residents. Notably, senior residents did not significantly outperform junior residents in most tasks. A critical finding was the significant impact of prior simulation-based practice or laparoscopic training certification on performance. Residents with such training or certification scored significantly higher and were more likely to achieve the passing grade. These two factors seem to have combined benefits. While prior laparoscopic training certification tended to shield candidates from failing the FLS tasks, only those who report using simulation-based practice achieved higher scores. This underscores the importance of structured simulation-based training programs in enhancing laparoscopic skills.

Interpreting the subjective data of proficiency remains plagued with doubts regarding the reliability of self-assessment. That is because self-assessment is notorious for being fallible. However, some studies suggest that self-assessment for surgical skills becomes more accurate with experience and can be considered.²⁴ To further test this assumption, we have utilized the available data from the subset of participants who completed the subjective online survey and took the objective FLS test. We plotted the subjective satisfaction results with laparoscopic skills and the objective scores obtained through FLS testing. Both scales were normalized to a percentile scale. Guided by Fraser et al, the FLS scores were capped at 350 divided by the five tasks, which is the highest reported cut-off to indicate laparoscopic task proficiency.²² The graphs showed a reasonable overlap with no statistically significant difference between the means of four out of the five domains. The only exception was the intracorporeal knotting skill, where participants tended to underestimate their objective performance (p=0.024). These results suggest that the survey findings for individuals who did not take part in the FLS testing can be considered a plausibly accurate reflection of reality (Figure 2).

Figure 2 Contrasting Subjective and Objective Skill Data pertaining to FLS-tasks. Note: Subjective data for all twelve participants were retrieved from the survey responses where the Likert scale of 1–5 is transformed to a percentile scale. Hand-eye coordination utilized the average of hand-to-hand coordination and depth perception while the circle cutting utilized the average of laparoscopic cutting and tissue manipulation. The objective data were obtained from the FLS test results of all 5 tasks where the scores were transformed to percentile scale with a score of 350 being the capped maximum as suggested by the literature. * Statistically significant difference (14% ± 23% versus 43% ± 39%, p= 0.024).

The results of our study underscore the importance of utilizing simulation-based practice and certified laparoscopic training in improving surgical performance among residents. The significant difference in success rates between those with and without such training highlights its crucial role in developing the necessary skills for intricate laparoscopic tasks. Prior training provides residents with a controlled environment to develop and refine their techniques, enhancing their ability to perform under pressure and within the specified time constraints. These findings go in line with growing evidence showing that dedicating time to training outside the operating room significantly improves educational outcomes. For instance, online video-based learning tools have become increasingly popular among trainees, thanks to advancements in video quality and accessibility.²⁵ Collectively, these tools allow trainees to study surgical steps and relevant anatomy in detail, enhancing their knowledge acquisition and procedural speed. They also address challenges typically associated with traditional teaching methods, such as limited exposure to surgeries in the operating room, providing a more comprehensive and effective learning experience. As surgical education continues to evolve in Saudi Arabia, integrating comprehensive simulation-based training and adopting laparoscopic certification programs should be prioritized to ensure that residents are well-prepared for the complexities of real-world surgical procedures.

Several limitations should be acknowledged in this study. Although the response rate to the online survey was reasonable (n=70), the sample size for the objective FLS assessment was relatively small (n=14), which may limit the generalizability of the findings. Additionally, the voluntary nature of participation in the FLS testing could introduce selection bias, as more motivated or confident residents may have opted in. Another limitation is the time gap between the subjective survey response and the objective test for those who chose to take the test. This gap was largely caused by logistical barriers related to the busy schedule of the residents and the travel distance to the exam location, which, in many instances, demanded rescheduling. Nonetheless, this gap would theoretically induce better results in the FLS scores due to acquired experience than if taken immediately after the survey. Hence, it would have induced bias towards better results.

Conclusion

This study highlights significant gaps in the satisfaction and proficiency of laparoscopic training among GS residents in Saudi Arabia. While residents are exposed to a reasonable volume of laparoscopic cases, the quality of training, particularly in advanced skills, requires enhancement. Simulation-based training and promotion of laparoscopic certification programs are crucial in improving laparoscopic proficiency. Addressing these gaps through structured training programs is essential to foster excellence and ensure patient safety in surgical practice.