Table Of ContentS P
CIENTIFIC APER
Virtual Reality Uterine Resectoscopic Simulator:
Face and Construct Validation and Comparative
Evaluation in an Educational Environment
Malcolm Gordon Munro, MD, David Paul Behling, MD
ABSTRACT Key Words: Resectoscope, Hysteroscope, Virtual reality,
Simulator.
Background and Objectives: Recognizing that resecto-
scopicsimulationmayhaveaneducationalrole,thispilot
study was designed to evaluate the face validity and ed-
ucational utility of a virtual reality uterine resectoscope
training system. INTRODUCTION
Methods: A pilot prospective comparative study of nov- Hysteroscopy is a surgical skill that provides the gyne-
iceandexperthysteroscopists’performanceonatargeting cologist and patient with the opportunity to experience
exerciseandmyomectomywiththevirtualloopelectrode. improved diagnostic accuracy, and clinically effective,
At baseline, expert and novice resectoscopists each per- minimallyinvasivesurgeryforthetreatmentofavariety
formed both exercises. Following instruction, novices ofdisordersincludingsymptomaticendometrialpolyps,
practiced each exercise a total of 9 times with the 10th submucosal leiomyomas, and selected Mu¨llerian fu-
recordedasthetrainingoutcome.Resultswerecompared sion/absorption defects. Unfortunately, since its intro-
both to baseline and to those of the experts. Data were duction into the medical literature by Pantaleone, from
analyzedwiththepairedtandWilcoxonranksumtestsas Italy, in 1869,1 hysteroscopy and hysteroscopically di-
appropriate. rected procedures have been reluctantly incorporated
into the practice of gynecology, despite the develop-
Results: At baseline, all experts touched 4 targets in a
ment of highly effective instruments for endoscopic
mean of 33 seconds with no perforations, compared to a
visualization and performance of intrauterine surgery.
mean of 2 for the 11 novices in a mean of 57 seconds
(P(cid:1)0.0034) with one perforation. In 3 minutes, the ex- Training of residents, fellows, and practicing gynecol-
perts removed a mean of 97.3% of the virtual myoma, ogists has been hampered by a number of obstacles,
compared to 66.1% for the novices (P(cid:1)0.0153). On the including a lack of suitably trained mentors, limited
10th“run,”novicestouchedameanof4targetsinamean access to appropriate equipment, restrictions in resi-
of23seconds,animprovementfrombaseline(P(cid:1)0.0004) denttraininghours,andtheabsenceofreadilyavailable
and improved to 89% on the myoma resection exercise systems to support training outside of the operating
(P(cid:1)0.0515) 36.3% over baseline. theater. A number of systems and simulators have been
proposed, but there still seems to be a relative paucity
Conclusion: Although this pilot study has a relatively
of postgraduate training programs with robust hystero-
small sample size and represents the results at one insti-
scopic education curricula. The existence of micropro-
tution, it demonstrates that virtual reality resectoscopic
cessors capable of managing large volumes of data
systems have the potential to measure and improve the
throughputhasprovidedtheopportunityforthedesign
technical skills of novices before they operate on human
of virtual reality (VR) surgical simulators that create a
patients.
near realistic operating environment. Such simulators
have been demonstrated to be effective in other surgi-
KaiserPermanenteLosAngelesMedicalCenter,andtheUniversityofCalifornia, cal disciplines and for other techniques, by reducing
LosAngeles,LosAngeles,California(allauthors). both the need for “in-OR” training time and activity
Presentedatthe38thGlobalCongressofMinimallyInvasiveGynecology,Orlando, associated with adverse events.2–5 The use of VR simu-
Florida,November19,2009.
lation can provide a realistic operating environment
Address correspondence to: Malcolm G. Munro, MD, Director of Gynecologic that provides neither surgical risk to a patient nor the
Services,DepartmentofObstetrics&Gynecology,4900SunsetBoulevard,Station
consumptionofvaluableoperatingroomresourcesand
3-B,LosAngeles,CA90027,USA.E-mail:[email protected]
time. If properly designed, such a system can provide
DOI:10.4293/108680811X13071180406312
the opportunity for objective measurement of selected
©2011byJSLS,JournaloftheSocietyofLaparoendoscopicSurgeons.Publishedby
theSocietyofLaparoendoscopicSurgeons,Inc. surgical performance outcomes.
142 JSLS(2011)15:142–146
Recently, a prototypical Virtual Reality Hysteroscopic resectionofaposteriorsubmucosalleiomyoma.Uterine
Simulator (VRHS) was developed by Immersion Medi- perforation is simulated if the trainee directs the hyst-
cal (CAE Montreal PQ, Canada), which provides an eroscope or electrode in a fashion that could traverse
opportunity to evaluate the utility of such a system for the myometrium.
training in hysteroscopic surgery (Figure 1). The de-
The pilot study had 2 hypotheses:
vice comprises a realistic resectoscope with an attach-
able “camera” and distal sensors, a pelvis with a me- 1. The device will show face validity in that experts will
chanical external cervical os, and actual electrosurgical complete the skill testing in less time than novice hyst-
foot pedals, all attached to a personal computer loaded eroscopists.
with proprietary software. Following assembly, the “re-
2. Novice hysteroscopists will show significant improve-
sectoscopic system” is inserted through a mechanical
ment in their times to successfully complete both skills
external os, thereby allowing the distally based sensors
followingaprescribedtrainingprogram(constructvalida-
to activate the software, in a fashion that provides a
tion).
realistic, interactive screen image. Virtual inflow and
outflow can be regulated, and the operator-controlled
MATERIALS AND METHODS
motions of the element and the distal optic are trans-
mitted to the screen to simulate an actual intrauterine
The Kaiser Permanente Regional Research Board ap-
environment with a loop electrode that can be manip-
proved the study. All procedures were performed in the
ulated and activated by the “surgeon.” Activation of the
SimulationCenteroftheKaiserFoundationHospital’sLos
foot pedals allows the surgeon to cut strips of virtual
Angeles Medical Center.
“tissue” from the target leiomyoma.
Thestudywasdesignedin2phases.Phase1wastheface
The device has 2 integrated software-based skill devel-
validation component that was designed to compare a
opment and testing exercises that include an introduc-
groupofnovicehysteroscopistswithagroupofsurgeons
tory“ManipulationModule”(Figure2),whichrequires
with expertise in hysteroscopy and resectoscopic proce-
the operator to sequentially use the electrode to touch
dures.Phase2wasdesignedtomeasureconstructvalidity,
4 randomly located targets within the endometrial cav-
byusingthedatafromthenovicesenrolledinPhase1as
ity; and a “Myoma Resection Module” (Figure 3),
abaselineforcomparingoutcomesfollowingaprescribed
which allows the trainee to perform resectoscopic loop
trainingprogramusingtheVHRS.Thepost“training”data
were also compared with the data from the 3 “experts”
obtained in Phase 1.
Theinvestigatorsidentified14subjectsfromtheattending
staff and residents of a community Obstetrics & Gynecol-
ogy residency education program, 3 “expert” hysterosco-
pists and 11 who were defined as “novices.” Each of the
hysteroscopists had more than a decade of extensive ex-
perience with diagnostic and operative hysteroscopy, as
well as resectoscopic surgery. All participated actively in
resident surgical teaching, but none had any exposure to
theVRHS.Novicehysteroscopistshadnoexperiencewith
hysteroscopy and comprised both medical students and
first-year residents.
For Phase 1, all of the subjects were oriented to the
VHRS in a single session that included a test “run” to
familiarize themselves with each of the 2 exercises.
Figure 1. Virtual Reality Resectoscope Simulator. A realistic
Then,eachsubjectperformedarecorded“baseline”run
resectoscope and camera (upper left) is passed through a me-
througheachexercise,firstwiththeManipulationMod-
chanicalcervix(upperright)withthecomputermonitordisplay-
ingthevirtualimage(lowerleft).Theoperator(center)conducts ule and then the Myoma Resection Module. The exer-
theexerciseusingrealisticfootpedals(lowerright)to“activate” cisetimes,targetoutcomes,andmeasurableerrorswere
theelectrosurgicalgenerator. recorded using the internal recording system and com-
JSLS(2011)15:142–146 143
Virtual Reality Uterine Resectoscopic Simulator: Face and Construct Validation and Comparative Evaluation in an Educational
Environment,MunroMGetal.
Figure 2. Targeting Exercise. The operator is randomly directed to touch 1 of 4 targets. The target stays red (left) until successfully
touched(right).
Figure3.MyomaResectionExercise.Theposteriormyomaisdisplayed(A)withtheextendedloopelectrode.Theoperatoractivates
the loop electrode, which turns orange and resects myoma tissue by moving the operating element (B). This continues until either
uterineperforation,3minuteshaveelapsed,orthesurgeonhascompletedtheresection(C).
pared. A maximum time of one minute was allowed for formal power analysis was performed to determine
theManipulationModule,while3minuteswereallotted sample size.
for the Myoma Resection Module.
RESULTS
ForPhase2,the11novicehysterscopistswereprovided
hysteroscopic education that included training in ma-
Phase 1
nipulation of the electrode, the lens, and other relevant
aspects of resectoscopic surgery. Two “runs” were su-
Manipulation Module (Baseline)
pervised and then the trainee was left alone, over a
All subjects participated in the manipulation test. In the
periodof2weeks,tocompleteatotalor9postbaseline
60-second exercise, each of the experts successfully
runs through both tests. Following completion of the
touched all 4 targets (Figure 4), without perforation,
ninth session, a final proctored run through each of the
withameanexercisetimeof33seconds(Figure5).For
2 exercises was performed, recorded, and then com-
the 11 novices, the median number of targets touched
paredtothebaselineforthetraineeand,collectively,to
was 2 (range, 0 to 4) (Figure 4), one perforation oc-
the expert scores obtained in Phase 1.
curred, and the mean exercise time was 57 seconds
Statistical comparisons were performed using paired t (Figure 5). These differences were significant.
tests for the Myoma Manipulation module, while the
Wilcoxon rank sum test was used to compare percent- Myoma Resection Module (Baseline)
ageresectionofmyomasfortheMyomaResectionMod- All 11 subjects completed the baseline of the myoma
ule. Because this was considered a pilot study, no resection module, with the experts removing a mean of
144 JSLS(2011)15:142–146
Phase 2
Manipulation Module (Run 10)
Sevenofthenovicesmadeitbackwithintheallottedtimefor
therecorded10thperformance;theother4didnotcomplete
theprogramintheprescribedtimeand,thereforewerenot
included in the analysis. The mean number of targets suc-
cessfullytouchedwas4(Figure4).Themeantimespenton
the Manipulation module was 23 seconds (Figure 5). The
differences from baseline were significant and were similar
tothoseofthe“experts”atbaseline.
Myoma Resection Module (Run 10)
For the 7 novices who completed the 10th run, baseline
Figure 4. Manipulation module results. Targets successfully
resectionvolumewas65.3%(range,30to92),whileatrun
touched.
10 it was 89% (range, 64 to 99), with one perforation
(Figure 6). These differences approached significance,
but there was a singular virtual perforation by one of the
novices during run 10. The differences in percentage re-
section between “novices” and “experts” were still signif-
icant, but the differences were greatly reduced.
DISCUSSION
Inthispilotstudy,theVHRSappearstohavefacevalidity
in that it is able to distinguish between novices and ex-
perts, by our definition. Furthermore, the results of this
studysupportthenotionthatthisvirtualrealitysimulation
system has construct validity, in that it was able to quan-
Figure5.Manipulationmoduleresults.Meantimetocompletion. titatively distinguish the experts from the novices, and
noviceswereable,withpractice,toimprovescoressothat
they approached those of the experts.
Although the results of the study suggest that the manip-
ulation module may be predictive of the performance of
myomectomy,furtherstudieswouldbenecessarytocon-
firm this impression. Furthermore, practice on such a
system might be expected to reduce the operating room
time spent by novices in the early part of the “learning
curve,” a process that adds to the cost of training and/or
thecostofcare.However,thisstudywasnotdesignedto
evaluate this outcome, and, consequently such a conclu-
sion cannot be made at this time.
Figure 6. Myoma resection. First versus tenth exercises by
novices. Simulator training using tissue simulations has been
shown to be effective at improving objective structured
assessing technique (OSAT) scores in residents at one
97.3% (range, 97% to 98%) of the myoma within the month, which deteriorates by 6 months.6 The process of
3-minuteexercisetime.Thenovicesremovedameanof setting up such time intensive laboratories is not condu-
66.1% of the virtual myoma (range, 12% to 92%) and civetorepetitivetrainingandpractice,therebyundermin-
one perforation occurred. The differences between ing the value of such systems. Virtual reality systems, at
“novices” and “experts” were significant (Figure 6). leasttheoretically,canbeavailabletobusyresidentsona
JSLS(2011)15:142–146 145
Virtual Reality Uterine Resectoscopic Simulator: Face and Construct Validation and Comparative Evaluation in an Educational
Environment,MunroMGetal.
daily or nightly basis, and, theoretically provide the op- trainingofnovicesinhysteroscopicsurgery.Furtherstud-
portunity for more sustained retention of learned skills. ies with larger sample sizes that include evaluation of
impactonbothresourceandclinicaloutcomes,including
Face validity simply implies that a given test appears to be
risks and complications, will be necessary to determine
able to measure a given skill. This VR system is 1 of 2
the ultimate value of these systems.
hysteroscopic VR systems that has been evaluated for face
validity.7 The other hysteroscopic VR simulator, HystSim
References:
(VirtaMed, Zurich, Switzerland), has also undergone con-
structvalidityevaluationandhasbeendemonstratedtohave 1. Pantaleone D. On endoscopic examination of the cavity of
valueindiagnostichysteroscopyandinmanipulationofthe thewomb[inItalian].MedicalPresCirc.1969;8:26–27.
instrument,butasurgicalinterventionsuchasresectoscopic
2. SeymourNE,GallagherAG,RomanSA,etal.Virtualreality
myomectomywasnottested.8Althoughitisanticipatedthat trainingimprovesoperatingroomperformance:resultsofaran-
training on the current system would reduce the additional domized, double-blinded study. Ann Surg. 2002;236:458–463;
time spent in the operating room involved with training discussion63–64.
residents at operative hysteroscopy and resectoscopy, a
3. Park J, MacRae H, Musselman LJ, et al. Randomized con-
well-designedstudyinvolvingoperatingroomperformance
trolled trial of virtual reality simulator training: transfer to live
wouldbenecessarytoprovethishypothesis.
patients.AmJSurg.2007;194:205–211.
Severalaspectsofresectoscopicsurgeryarenotassessedby 4. Aggarwal R, Ward J, Balasundaram I, et al. Proving the
thissystem.Thecompetenciesthatwerenottestedincluded effectiveness of virtual reality simulation for training in laparo-
resectoscope assembly, use of foreoblique (angled) lenses, scopicsurgery.AnnSurg.2007;246:771–779.
fluid management, and specimen removal. Many of these
5. Panait L, Bell RL, Roberts KE, Duffy AJ. Designing and
aspects could be added to the training provided by the VR
validating a customized virtual reality-based laparoscopic skills
systeminthecontextofacomprehensivehysteroscopyand
curriculum.JSurgEduc.2008;65:413–417.
resectoscopicsurgeryeducationprogram.Theimpactofthe
6. BurchardER,LockrowEG,ZahnCM,etal.Simulationtrain-
training aspect of the study is limited somewhat by the
ing improves resident performance in operative hysteroscopic
number of subjects who did not return for the follow-up
resectiontechniques.AmJObstetGynecol.2007;197:542;e1–4.
testing. While the sample size appeared to be adequate, it
was difficult to organize residents to return within a prede- 7. BajkaM,TuchschmidS,StreichM,etal.Evaluationofanew
termined period of time to complete the follow-up testing. virtual-reality training simulator for hysteroscopy. Surg Endosc.
Nonetheless, our data on the impact of training nearly 2009;23:2026–2033.
reached significance, particularly since one of the residents 8. Bajka M, Tuchschmid S, Fink D, Sze´kely G, H arders M.
who was performing well on the 7th through 9th sessions Establishing construct validity of a virtual-reality training simu-
perforatedtheuterusonthe10th. lator for hysteroscopy via a multimetric scoring system. Surg
Endosc.2010;24:79–88;Epub2009Jun24.
CONCLUSION
As a result of this pilot study, we feel that we have
demonstrated potential value for this system for the early
146 JSLS(2011)15:142–146
