The predictive value of aptitude assessment in laparoscopic surgery

19 Pages·2016·0.41 MB·English

medical education in review
The predictive value of aptitude assessment in
laparoscopic surgery: a meta-analysis
Kelvin H Kramp,1 Marc J van Det,2 Christiaan Hoff,1 Nic J G M Veeger,3,4 Henk O ten Cate Hoedemaker5 &
Jean-Pierre E N Pierie1,6
CONTEXT Current methods of assessing significantly with visual–spatial ability
candidates for medical specialties that involve (r = 0.32, 95% confidence interval [CI] 0.25–
laparoscopic skills suffer from a lack of instru- 0.39; p < 0.001), perceptual ability (r = 0.31,
ments to assess the ability to work in a mini- 95% CI 0.22–0.39; p < 0.001), psychomotor
mally invasive surgery environment. ability (r = 0.26, 95% CI 0.10–0.40; p = 0.003)
and simulator-based assessment of aptitude
OBJECTIVES A meta-analysis was conducted (r = 0.64, 95% CI 0.52–0.73; p < 0.001).
to investigate whether aptitude assessment can Three-dimensional dynamic visual–spatial abil-
be used to predict variability in the acquisition ity showed a significantly higher correlation
and performance of laparoscopic skills. than intrinsic static visual–spatial ability
(p = 0.024).
METHODS PubMed,PsycINFOandGoogle
ScholarweresearchedtoNovember2014for CONCLUSIONS In general, aptitude assess-
publishedandunpublishedstudiesreporting ments are associated with laparoscopic skill
themeasurementofaformofaptitudefor level. Simulator-based assessment of aptitude
laparoscopicskills.Thequalityofstudieswas appears to have the potential to represent a
assessedwithQUADAS-2.Summarycorrelations job sample and to enable the assessment of all
werecalculatedusingarandom-effectsmodel. forms of aptitude for laparoscopic surgery at
once. A laparoscopy aptitude test can be a
RESULTS Thirty-four studies were found to
valuable additional tool in the assessment of
be eligible for inclusion; six of these studies
candidates for medical specialties that require
used an operating room performance
laparoscopic skills.
measurement. Laparoscopic skills correlated
MedicalEducation2016:50:409–427
doi:10.1111/medu.12945
Discussideasarisingfromthearticleat
www.mededuc.comdiscuss.
1DepartmentofSurgery,MedicalCentreLeeuwarden, 6PostGraduateSchoolofMedicine,UniversityMedicalCenter
Leeuwarden,TheNetherlands GroningenandUniversityofGroningen,Groningen,the
2DepartmentofSurgery,HospitalGroupTwente,Almelo,The Netherlands
Netherlands
3DepartmentofEpidemiology,UniversityMedicalCentre Correspondence:KelvinHKramp,DepartmentofSurgery,Medical
Groningen,Groningen,TheNetherlands CentreLeeuwarden,LeeuwardenHenriDunantweg2,8934AD
4DepartmentofEpidemiology,LeeuwardenMedicalCenter, Leeuwarden,TheNetherlands.Tel:[phone];
Leeuwarden,theNetherlands E-mail:[email protected]
5DepartmentofSurgery,UniversityMedicalCenterGroningen,
Groningen,theNetherlands
ª2016JohnWiley &SonsLtd.MEDICAL EDUCATION 2016;50:409–427 409
K H Kramp et al
there is evidence that the ability to learn and per-
INTRODUCTION form laparoscopic surgery can be assessed with mea-
surements of these aptitudes, there are also reports
The assessment of candidates for training in a medi- that contradict such a correlation.10–14 To date,
cal discipline is a critical component of selection reviews that have aimed to reach an univocal con-
procedures in medical education. However, clusion on this topic have been mainly descriptive
although surgical techniques are becoming increas- in nature, have lacked a quantitative analysis or have
ingly difficult to master, no scientific methods are investigated a broad spectrum of surgical skills.4,8,14
currently used to evaluate the potential to acquire Therefore, a meta-analysis was conducted to evalu-
these surgical skills. This is especially worrisome in ate whether aptitude assessments can be used to
the field of laparoscopic surgery, which is coming to predict the ability to acquire and perform laparo-
represent the mainstay method in an increasing list scopic skills and to quantify how much of difference
of procedures in abdominal surgery, gynaecology between individuals can be predicted by aptitude
and urology. During laparoscopic surgery, there is assessment.
no direct visualisation of the operative field or
direct contact with intra-abdominal organs. Conse-
quently, laparoscopic surgery requires a set of skills METHODS
that differs from those used in conventional surgery.
The majority of trainees overcome the ergonomic Search strategy
difficulties associated with laparoscopic surgery dur-
ing laparoscopic skills training, but research has A systematic literature search in PubMed, PsycINFO
raised concerns about large individual differences and Google Scholar was conducted in November
during and after training that may be dependent on 2014 to find studies that measured laparoscopic
aptitude. For instance, the studies of both Schijven skills and aptitude (Fig. 1). The query used to iden-
et al.1 and Grantcharov and Funch-Jensen2 were able tify the available literature in PubMed was ‘((Space
to distinguish four groups that showed different pat- Perception[MeSH]) OR (Visual Perception[MeSH])
terns of improvement during laparoscopic skills OR (Psychomotor Performance[MeSH]) OR (Apti-
training: (i) those who were proficient at the begin- tude[Mesh])) AND (Laparoscopy[MeSH])’. The
ning; (ii) those who achieved proficiency through query used in PsycINFO was ‘Laparoscopy OR
training; (iii) those who showed improvement with- Laparoscopic’ and was limited to studies in human
out reaching proficiency, and (iv) those who showed subjects aged > 18 years. The queries used in Goo-
no improvement. A third study by Bosker et al.3 gle Scholar were the word ‘laparoscopy’ combined
showed that one of 13 participants (7.7%) seemed with the phrase ‘visual–spatial ability’, and the word
to have difficulties in learning to perform a laparo- ‘laparoscopy’ combined with the phrase ‘psychomo-
scopic sigmoid resection, although there were no tor ability’. To identify unpublished literature, dis-
factors that could have caused an increase in the sertation and thesis databases and conference
difficulty of procedures performed. abstract books were hand-searched using the key-
words ‘visual–spatial’, ‘spatial ability’, ‘visual percep-
There is growing evidence in the literature that tion’, ‘spatial perception’, ‘psychomotor’ and
some of these differences in ability to learn and per- ‘aptitude’ for additional relevant titles. Finally, the
form laparoscopic surgery can be explained in part reference lists of the studies retrieved were scanned
by aptitude. Aptitude for minimally invasive surgery for additional relevant studies and the names of key
(MIS) is often divided into three abilities that are authors were used as search terms in PubMed and
generally accepted to be of innate nature: visual– Google Scholar. The abstracts of studies with rele-
spatial ability; perceptual ability, and psychomotor vant titles were reviewed; only studies that assessed
ability.4 Visual–spatial ability refers to the ability to aptitude among subjects and measured laparoscopic
mentally visualise or manipulate objects; perceptual skill level were considered eligible for inclusion.
ability refers to the ability to interpret two-dimen-
sional representations of three-dimensional objects, Data extraction and quality assessment
and psychomotor ability refers to the ability to per-
form motor movements that involve eye–hand coor- From the final list of studies, information was
dination, bimanual dexterity and short reaction extracted on: the specific aptitude tests used; the
times. The question of whether testing of these abil- reported correlations between the results of apti-
ities can be used in the selection of trainees is cur- tude tests and the performance level of participants;
rently a topic of vehement debate.5–9 Although numbers of participants; characteristics of partici-
410 ª2016JohnWiley &SonsLtd.MEDICAL EDUCATION 2016;50:409–427
Assessment of aptitude for laparoscopic surgery
Figure 1 Flowdiagramof search strategy. TheGoogle Scholar searchwas conducted withthe word ‘laparoscopy’ combined
withthe phrase‘visual spatialability’a and‘laparoscopy’ combinedwith thephrase ‘psychomotor ability’b.Abstract books
were sourced from theEuropean Association forEndoscopicSurgery, the Society ofAmerican Gastrointestinaland
Endoscopic Surgeons,andthe Associationof Laparoscopic Surgeons ofGreatBritain and Ireland.Thesesand dissertations
were found byusing keyauthor namesas searchterms in GoogleScholar
pants; methods used to measure laparoscopic skills; for bias. Because the goal of this meta-analysis was
parts of the learning curve in which laparoscopic to investigate the predictive validity of aptitude tests,
skills were measured; publication status, and country which can be seen as index tests, the QUADAS-2
of origin. was considered to be a suitable quality assessment
tool.
The quality of the included studies was assessed
using the modified form of the QUADAS-2 (QUality Meta-analysis
Assessment of Diagnostic Accuracy Studies).16 QUA-
DAS-2 is a quality assessment system for diagnostic The collected correlations were coded such that
studies. It involves four domains of assessment that positive correlations indicate a proportional rela-
should be tailored to fit the study in question: sub- tionship and negative correlations an inverse pro-
ject selection; index test; reference test, and flow portional relationship between aptitude test scores
and timing. The first three of these domains are and laparoscopic performance metrics. For studies
evaluated based on aspects related to the risk for that did not report the relationship between apti-
bias and concerns related to the applicability of the tude scores and laparoscopic skills with Pearson or
study results, whereas the domain of flow and tim- Spearman correlation coefficients, the reported
ing is only assessed based on aspects related to risk results were converted into correlation coefficients
ª2016JohnWiley &SonsLtd.MEDICAL EDUCATION 2016;50:409–427 411
K H Kramp et al
using the formulae shown in Appendix S1. Fisher of variance that can be attributed to variation
Z–r transformation was used to translate the Pear- among studies. Heterogeneity was defined by per-
son, Spearman and converted correlations into centage as: low (25%); medium (50%), and high
effect sizes.17 (75%).18
If the actual value of a non-significant correlation Visual–spatial ability moderator analysis
was not reported, the lead authors were contacted
for additional data. If data could not be obtained Whenstatisticallysignificantheterogeneityis
from the authors, we used either of two different observed,amoderatoranalysiscanbeperformedto
strategies to address the missing correlation: (i) investigatewhethervariationsinresultsamongstud-
there is no relationship between the two variables, iesarecausedbydifferencesinstudymethodology.
or (ii) the sample size is too small to achieve a Inthismeta-analysis,amoderatoranalysiswascon-
level of significance. The first option can mathe- ductedtoevaluatethedifferentfactorsthatmight
matically be considered as a correlation of 0. Non- haveinfluencedtherelationshipbetweenvisual–spa-
significant correlations with an unreported value tialabilityandlaparoscopicskills.Moderatorswere
were therefore coded as r = 0 in dataset DS . The setas:(i)arecentlypublished2 9 2classificationof
0
second option was evaluated by substituting the visual–spatialability;(ii)methodofmeasuring
maximum achievable correlation coefficient (the laparoscopicskills;(iii)participantcharacteristics,
critical value of the Pearson correlation coefficient and(iv)componentsofthelearningcurve.Theran-
based on the number of participants) for the dom-effectsmodelwasusedtocalculatethesummary
unreported non-significant correlations in dataset, correlationforeachsubgroupwithinthemoderator
DataSet with r = Critical Value (DS ). analysis.Apooleds2wasusedtoestimatetheimpreci-
cv
sionofsubgroupsummarycorrelationsandsub-
Some studies used multiple groups of participants groupswerecomparedwithheterogeneityQ
with different characteristics (e.g. medical students, accordingtotheproceduredescribedbyBorenstein
trainees, consultants, etc.). In these cases, the corre- et al.17HeterogeneityQ-valueswithp-valuesof< 0.05
lations for each group of participants were calcu- (two-tailed)wereconsideredstatisticallysignificantin
lated into mean ‘participant group’ effect sizes. This themoderatoranalysis.
was done by computing the mean value of the cor-
relations between aptitude test outcomes and 2 9 2 classification of visual–spatial ability
laparoscopic skill level reported in the study for a
specific group of participants. It is currently accepted that visual–spatial ability is
not a uniform ability.19,20 Uttal et al.19 recently pro-
A correction was applied to the variance to compen- posed a classification of visual–spatial ability on the
sate for the partial independence between correla- basis of two fundamental properties of visual–spatial
tions because of commonalities in the study setting ability tests. The first distinction is whether a visual–
in which the different correlations were measured spatial ability test utilises intrinsic or extrinsic infor-
within a participant group.17 As no correlations that mation. Intrinsic information contains the charac-
could be used to correct the partial interdepen- teristics that define an object. Extrinsic information
dence between the reported correlations were iden- is information that comes from relationships
tified in the literature, r = 0.5 was used as a between groups of objects or the relationship of an
x
compromise between the two extremes (Formula 6, object to a framework. The second distinction refers
Appendix S1). to whether a visual–spatial ability test requires a sta-
tic or dynamic mental process to complete. Static
After estimating mean participant group effect sizes visual–spatial ability tests contain fixed objects,
and their variance, summary correlations were cal- whereas dynamic visual–spatial ability tests require
culated for the different forms of aptitude. Because the mental visualisation of a spatial change in an
a high degree of variety (heterogeneity) in method- object or perspective. These two properties of
ology among the included studies was observed, visual–spatial ability tests can be used to classify
the random-effects model was used to calculate the them in four categories: intrinsic static; intrinsic
mean correlations. Heterogeneity tests (Cochrane dynamic; extrinsic static, and extrinsic dynamic.
Q) were performed to assess the variety among Uttal et al.19 suggested that a visual–spatial ability
studies and a p-value of < 0.10 was considered to test score is an indication of visual–spatial ability in
indicate differences of statistical significance. The one of the four categories that cannot plainly be
I2 statistic was calculated to estimate the percentage generalised to other categories. For example, artists
412 ª2016JohnWiley &SonsLtd.MEDICAL EDUCATION 2016;50:409–427
Assessment of aptitude for laparoscopic surgery
seem to use their visual object ability (intrinsic sta- pseudo confidence intervals (CIs). Quantitative
tic) in their profession, whereas engineers appear to assessment of publication bias was performed with
depend more on visual–spatial translational ability the Egger and Begg tests with p-values of < 0.05
(intrinsic dynamic and extrinsic dynamic).21,22 (two-tailed) considered as indicative of signifi-
cance.26,27
Other moderators
A further three moderators were used to evaluate RESULTS
heterogeneity among studies in visual–spatial ability:
measurement method; participant characteristics, Search results
and part of the learning curve in which the perfor-
mance level was measured. The results of the literature search are shown in
Fig. 1. Eight studies were excluded because they
The association between visual–spatial ability and used a methodology or statistical analysis incompati-
laparoscopic skills may differ on the basis of the ble with the research question of the meta-analysis
method used to measure skill level. To investigate or because of technical difficulties during aptitude
whether the method of measuring performance testing (see Appendix S2 online for details).28–35
influences the correlation with visual–spatial ability, Some studies evaluated the correlation between a
correlations were categorised according to whether simulator-based assessment and a subsequent perfor-
they were derived from assessments using: (i) a mance on a simulator or performance in the OR.
video trainer; (ii) a virtual reality simulator; (iii) These studies were analysed separately from those
laparoscopic camera navigation, or (iv) laparoscopic using other aptitude measurements. In total, 34
surgery on an animal or human. studies were eligible for inclusion on the basis of
the present criteria and could be used for further
Whether the study sample represents the character- analysis; six of these studies included an OR
istics of the population of interest can be of key performance measurement. A limited series of
importance in the validation of prognostic tools. In studies reported cut-off scores for the classification
the moderator participant characteristics, the correla- of candidates and their corresponding sensitivity
tions were therefore categorised according to and specificity.1,36,37
whether they referred to: (i) non-medical students;
(ii) medical students; (iii) novice trainees, or (iv) Quality assessment
trainees who had received training in laparoscopy
and consultants. The quality of the 34 included studies is shown in
Appendix S3. Seven studies reported correlations
A typical learning curve has been documented in that were not at risk of bias and six studies used an
laparoscopic skills training on video trainers, virtual OR performance measurement and therefore did
reality simulators and in the operating room not raise concerns regarding applicability.13,37–48
(OR).23,24 The learning curve starts with a baseline
performance. After the first performance there are Visual–spatial ability and laparoscopic skills
improvements with each repetition. The improve-
ments decrease in size as experience accrues accord- Twenty-seven studies containing 36 participant
ing to the learning rate. As the improvements groups were included in the analysis of visual–spatial
become increasingly smaller with each repetition, a ability (Table 1). An overview of the aptitude tests
learning plateau is reached.25 The strength of the used to measure visual–spatial ability is shown in
association between visual–spatial ability test scores Appendix S4. In five groups of participants, laparo-
and laparoscopic skills may differ between learning scopic skills were measured in the OR.13,48 The
phases. In the moderator learning curve, correlations mean correlation in DS (dataset in which unre-
0
were therefore categorised according to whether ported non-significant correlations are coded as 0)
they referred to: (i) baseline performance; (ii) was 0.32 (95% CI 0.25–0.39; p < 0.001), Q was statis-
learning rate, or (iii) learning plateau. tically significant (74.55; p < 0.001) and I2 was 53%,
indicating above-moderate heterogeneity (Fig. 2).
Publication bias When inclusion was limited to studies that used an
OR performance to measure laparoscopic skills, the
To visually assess the sample of studies for publica- mean correlation in DS increased to 0.50 (95% CI
0
tion bias, a funnel plot was created with 95% 0.07–0.77; p = 0.024).
ª2016JohnWiley &SonsLtd.MEDICAL EDUCATION 2016;50:409–427 413
K H Kramp et al
Table1 Overviewofstudiesmeasuringthepredictivepowerofvisual–spatialability
Levelof
Study Year Country n training VSA* MM BL LR LP Sign Total Correlation† PS‡ Remarks
1 Risucci 2000 USA 39 20beginning 1,2b VT NA NA NA 3 12 0.41–0.71 p
etal.68 PGY1
19PGY3–5
(Np>30)
2 Eyal& 2001 USA 27 Undergraduates 2a, LCN NA NA NA 4 4 0.39–0.58 p
Tendick42 2b,4
3 Risucci 2001 USA 94 23PGY>3 1,2b VT + NA + 10 12 0.21–0.51 p Partialcorrelation
etal.69 71attending correctedfor
surgeons acquired
knowledge
duringcourse
alsosignificant
4 Haluck 2002 USA 25 Nosimulator 2a, LCN NA NA NA NA 3 0.30–0.39 p
etal.70 experience 2b,3
5a Keehner 2004a USA 48 Lowexperience 2b OR NA NA NA 1 1 0.39 p Performanceon
etal.13 (median liveanimals
Np=13)
5b Keehner 2004b USA 45 Highexperience 2b OR NA NA NA 0 1 0.02 p Performanceon
etal.13 (median liveanimals
Np=302)
6 Schijven 2004 Netherlands 28 Hospital 2b VRS NA NA NA 0 1 0.40 p Kendalltau
etal.1 residentsand correlation
final-year
interns
7 McClusky 2005 USA 11 Year4medical 2a, VRS NA NA NA 0 3 (cid:1) p
etal.12 students 2b,3
8 Stefanidis 2006 USA 20 Year1surgical 1,2a, VT, + NA NA 5 30 0.44–0.64 p
etal.36 residents 2b,3 VRS,
(median LCN
Np=0)
9a Hedman 2006a Sweden 54 Medical 2b VRS + NA NA 8 15 0.28–0.40 p
etal.43 students
(n=0)
9b Hedman 2006b Sweden 25 Medical 2b VRS + NA NA 8 15 0.43–0.49 p Correlation
etal.43 students significantwhen
(n=0) compensated
forBASIQ-
generaltest
scores
10 Keehner 2006 USA 22 Non-medical 5 LCN + NA + 3 4 0.19–0.46 p Correlation
etal.38 students correctedfor
general
intelligencewas
significant
414 ª2016JohnWiley &SonsLtd.MEDICAL EDUCATION 2016;50:409–427
Assessment of aptitude for laparoscopic surgery
Table1 (Continued)
Levelof
Study Year Country n training VSA* MM BL LR LP Sign Total Correlation† PS‡ Remarks
11 Andalib 2006 Canada 32 Medicaland 2a,2b, VRS NA + + 3 6 0.36–0.46 a
etal.71 dentalstudents 3,5
(Ns=0)
12 Birbas 2006 Greece 21 Minimal 5 VRS NA NA + 1 NA 0.72 a
etal.73 experience
13 Hassan 2007 Germany 16 NA 2b VRS NA NA NA NA NA NA p MannWhitney
etal.72 Utest
14 Enochsson 2008 Sweden 9 Gynaecological 2b VRS NA NA NA 4 4 0.72–0.82 a
etal.74 consultants
15 Rosenthal 2010 Switzerland 56 Novice(Np=0) 2b LCN NA NA NA 4 3 0.28–0.45 p
etal.31 toexpert
(Np>100)
16 Sliwinski75 2010 Netherlands 7 Surgicaland 1, VRS (cid:1) NA + 2 20 0.78–0.88 t
gynaecologic 2a,2b
trainees
17 Kolozsvari 2011 Canada 32 Medicaland 2a, VT (cid:1) (cid:1) (cid:1) 0 9 (cid:1) p Onlystudyto
etal.14 dentalstudents 2b,3 measure
(Ns=0) surgicalinterest
18 Jungmann 2011 Germany 40 Medicalstudents 2b VRS NA NA NA 2 3 0.38–0.56 p
etal.44 (Ns=0)
19 Ahlborg 2011 Sweden 13 Gynaecological 2b VRS NA NA NA 7 15 0.57–0.64 p
etal.45 consultants
20 Schlickum 2011 Sweden 25 Medical 2b VRS NA NA NA 1 1 0.45 p
etal.76 students
21 Luursema 2012 Netherlands 23 Technical 1, VRS (cid:1) NA – 0 3 (cid:1) p
etal.11 medicine 2a,5
students
22a Ahlborg 2012a Sweden 28 Gynaecological 2b VRS + NA NA 1 2 0.40 p
etal.39 trainees
(Ns=0,
Np=10)
22b Ahlborg 2012b Sweden 19 Gynaecological 2b VRS NA NA NA 1 2 (cid:1) p
etal.39 traineeswith
VRStraining
23a Nugent46 2012a Ireland 40 Pre-clinical 2a, VRS + NA NA 9 12 0.34–0.48 d
medical 2b,3
students
Years1–3
23b Nugent46 2012b Ireland 20 12PGY1basic 2a, VRS NA NA NA 3 12 0.45–0.59 d
surgicaltrainees 2b,3
8PGY2basic
surgicaltrainees
23c Nugent46 2012c Ireland 8 Highersurgical 2a, VRS NA NA NA 3 12 0.75–0.80 d
trainees 2b,3
Years1–3
ª2016JohnWiley &SonsLtd.MEDICAL EDUCATION 2016;50:409–427 415
K H Kramp et al
Table1 (Continued)
Levelof
Study Year Country n training VSA* MM BL LR LP Sign Total Correlation† PS‡ Remarks
23d Nugent46 2012d Ireland 12 Highersurgical 2a, VRS NA NA NA 0 12 (cid:1) d
trainees 2b,3
Years4–6
23e Nugent46 2012e Ireland 26 Pre-clinical 0 VRS NA NA NA 3 4 0.54–0.94 d
medical
students
Years1–3
24 Nugent46 2012 Ireland 67 Generaland 0 VRS, NA NA NA 0 2 (cid:1) d
plasticsurgery bench
trainees models
25 Nugent 2012 Ireland 10 Surgicaltrainees 0 VRS + NA NA 6 13 0.67–0.78 p
etal.51 (Nbl>20,
Nal<5)
26a Ahlborg 2013 Sweden 28 Gynaecological 2b OR + NA NA 0 1 0.33 p Performance
etal.48 trainees onhumans
(Ns=0,
Np<10)
26b Ahlborg 2013 Sweden 7 Gynaecological 2b OR + NA NA 1 1 0.98 p Performance
etal.48 traineesno onhumans
VRStraining
26c Ahlborg 2013 Sweden 13 Gynaecological 2b OR NA NA NA 0 1 0.13 p Performance
etal.48 traineeswith onhumans
VRStraining
27 Groenier 2014 Netherlands 53 Technical 1,2b VRS NA NA NA 0 1 (cid:1) p
etal.10 medicine
students
(cid:3)indicatessignificant/notsignificant
BL=baselineperformance;LCN=laparoscopiccameranavigation;LP=learningplateau;LR=learningrate;MM=measurement
method;NA=notaddressed;n=numberofparticipants;Nal=numberofadvancedlaparoscopicproceduresperformed;Nbl=number
ofbasiclaparoscopicproceduresperformed;Np=numberofproceduresperformed;Ns=numberofsimulatortasksperformed;
OR=animal/humanoperatingroomperformance;PGY=postgraduateyear;PS=publicationstatus;VSA=visual–spatialability;
VRS=virtualrealitysimulator;VT=videotrainer
*0=compositescorestaticanddynamic;1=intrinsicstatic;2a=intrinsicdynamictwo-dimensional;2b=intrinsicdynamicthree-dimen-
sional;3=extrinsicstatic;4=extrinsicdynamic;5=compositescoreintrinsicdynamicandextrinsicdynamic
†Correlation:minimum–maximumsignificantcorrelationfoundinthestudy
‡a=abstract;d=dissertation;t=thesis;p=publishedinpeer-reviewedjournal
Perceptual ability and laparoscopic skills mean correlation for perceptual ability in DS was
0
0.31 (95% CI 0.22–0.39; p < 0.001); Q was 21.30
In all of the studies included, perceptual ability was (p = 0.128), indicating no heterogeneity among
assessed with the Pictorial Surface Orientation test studies (Fig. 2).
(PicSOr). The PicSOr was developed to measure the
ability to recognise the 3-D orientation of a virtual Psychomotor ability and laparoscopic skills
object from a 2-D screen.12,49 Seven studies contain-
ing 15 participant groups were included in the cor- Four studies containing eight participant groups
relation analysis of perceptual ability (Table 2). The were included in the correlation analysis of
416 ª2016JohnWiley &SonsLtd.MEDICAL EDUCATION 2016;50:409–427
Assessment of aptitude for laparoscopic surgery
k n r 95%LCI95%UCI
Visual-spa(cid:2)al ability DS0 36 332 0.32 0.25 0.39
DScv 0.38 0.30 0.45
Perceptual ability DS0 0.31 0.22 0.39
15 33
DScv 0.37 0.27 0.45
Psychomotor ability DS0 8 57 0.26 0.10 0.40
DScv 0.43 0.26 0.58
Simulator performance metrics DS0 0.64 0.52 0.73
11 66
DScv 0.69 0.60 0.76
0.10 0.30 0.50 0.70 0.90
Figure 2 Forestplot ofthe summary correlations of thedifferent aptitude measurements with95% confidence intervals.
DS = datasetwith non-reported non-significant correlations coded as0;DS = datasetwith non-reported non-significant
0 cv
correlations coded asthecritical Pearsoncorrelation coefficient ata = 0.05 anddf = n (cid:1) 2; k = number ofparticipant
2
groups; n = number ofcorrelations; r = summary correlation; LCI = lowerconfidenceinterval; UCI = upperconfidence
interval
psychomotor ability (Table 3). An overview of the whether the heterogeneity among studies was
encountered aptitude tests for psychomotor ability caused by differences in methodology. The results
is shown in Appendix S5. Of the seven different psy- of the moderator analysis for visual–spatial ability
chomotor ability tests used, only the finger tap test are shown in Table 5. Three studies were excluded
and the grooved pegboard test showed significant from the analysis of the 2 9 2 classification of
correlations with laparoscopy performance.36,46 The visual–spatial ability because they used a composite
mean correlation for psychomotor ability in DS was measure of static and dynamic visual–spatial ability
0
0.26 (95% CI 0.10–0.40; p < 0.003); Q was 9.85 tests.46,51 The subgroup extrinsic dynamic contained
(p = 0.197), indicating no statistically significant only one study.38 The subgroups intrinsic dynamic
heterogeneity among studies (Fig. 2). and extrinsic dynamic were thus combined into the
subgroup dynamic visual–spatial ability. Close inspec-
Simulator-based assessment tion of this subgroup showed a subdivision of 2-D
visual–spatial ability tests, often of low complexity.
Nine studies containing 11 participant groups were The subgroup dynamic visual–spatial ability was conse-
included in the analysis of the predictive validity of quently divided into 2-D dynamic and 3-D dynamic to
simulator-based assessment of aptitude (Table 4). In create an adjusted 2 9 2 classification.
five of the nine studies, laparoscopic skills training
parameters were correlated with OR performance The subgroup intrinsic static showed no statistically
measurements.37,40,41,47,50 The mean correlation for significant correlation in DS (dataset with unre-
0
simulated MIS performance in DS was 0.64 (95% ported non-significant correlations coded as 0) and
0
CI 0.52–0.73; p < 0.001); Q was 13.78 (p = 0.183), DS (dataset with unreported non-significant corre-
cv
indicating no statistically significant heterogeneity lations coded as maximum achievable correlation)
among studies (Fig. 2). When the studies included (p = 0.069 and p = 0.100, respectively) and the sub-
were limited to those with a correlation between group extrinsic static showed no significant correla-
simulator performance and a subsequent OR perfor- tion only in DS (p = 0.075). A significant
0
mance, the mean correlation in DS decreased to difference was observed between subgroups
0
0.61 (95% CI 0.42–0.75; p < 0.001). (p = 0.024). The unknown size of non-significant
correlations led to a substantial difference between
Visual–spatial ability moderator analysis subgroups in DS and DS in the subgroup extrinsic
0 cv
static (DS : r = 0.14, 95% CI (cid:1)0.01 to 0.28; DS :
0 cv
2 9 2 classification of visual–spatial ability r = 0.34, 95% CI 0.19–0.48). Consequently, only the
subgroups 3-D dynamic and intrinsic static were mutu-
There was significant heterogeneity in the summary ally compared. Comparison of the 3-D dynamic and
correlation of visual–spatial ability (p < 0.001). A intrinsic static subgroups showed a statistically signifi-
moderator analysis was performed to investigate cant difference (p = 0.024).
ª2016JohnWiley &SonsLtd.MEDICAL EDUCATION 2016;50:409–427 417
K H Kramp et al
Table2 Overviewofstudiesmeasuringthepredictivepowerofperceptualability
Study Year Country n Leveloftraining MM BL LR LP Sign Total Correlation* PS†
1 Haluck 2002 USA 25 Nosimulatorexperience LCN NA NA NA 1 1 0.59 p
etal.70
2a Gallagher 2003 Ireland, 48 Laparoscopicnovices VT NA NA NA 1 1 0.50 p
etal.49 UK
2b Gallagher 2003 Ireland, 32 Laparoscopicnovices VT NA NA NA 1 1 0.50 p
etal.49 UK
2c Gallagher 2003 Ireland, 34 Laparoscopicnovicesand VT NA NA NA 1 1 0.42 p
etal.49 UK experiencedsurgeons
2d Gallagher 2003 Ireland, 18 Experiencedlaparoscopic VT NA NA NA 1 1 0.54 p
etal.49 UK surgeons
3 McClusky 2005 USA 11 Year4medicalstudents VRS NA NA NA 1 1 0.76 p
etal.12
4 Stefanidis 2006 USA 20 Year1surgicalresidents VT,VRS, (cid:1) NA NA 0 5 NS p
etal.36 (medianNp=0) LCN
5 Kolozsvari 2011 Canada 32 Medicalanddental VT + (cid:1) (cid:1) 1 3 0.38 p
etal.14 students(Ns=0)
6a Nugent46 2012a Ireland 40 Pre-clinicalmedical VRS + NA NA 1 3 0.49 d
studentsYears1–3
6b Nugent46 2012b Ireland 20 12PGY1basicsurgical VRS NA NA NA 1 3 0.52 d
trainees
8PGY2basicsurgical
trainees
6c Nugent46 2012c Ireland 8 Highersurgicaltrainees VRS NA NA NA 1 3 0.80 d
Years1–3
6d Nugent46 2012d Ireland 12 Highersurgicaltrainees VRS NA NA NA 2 3 0.70–0.73 d
Years4–6
6e Nugent46 2012e Ireland 26 Pre-clinicalmedical VRS NA NA NA 1 4 0.56 d
studentsYears1–3
7 Nugent46 2012 Ireland 67 Generalandplastic VRSBench NA NA NA 1 2 0.31 d
surgerytrainees models
(cid:3)indicatessignificant/notsignificant
BL=baselineperformance;LCN=laparoscopiccameranavigation;LP=learningplateau;LR=learningrate;MM=measurement
method;NA=notaddressed;n=numberofparticipants;Nal=numberofadvancedlaparoscopicproceduresperformed;Nbl=number
ofbasiclaparoscopicproceduresperformed;Np=numberofproceduresperformed;Ns=numberofsimulatortasksperformed;
PGY=postgraduateyear;PS=publicationstatus;VRS=virtualrealitysimulator;VSA=visual–spatialability;VT=videotrainer
*Correlation:minimum–maximumsignificantcorrelationfoundinthestudy
†a=abstract;d=dissertation;t=thesis;p=publishedinpeer-reviewedjournal
Other moderators learning curve and in the subgroup trained participants
in the moderator participant characteristics.
In the moderators measurement method, participant
characteristics and learning curve, no significant differ- Publication bias
ence was observed between subgroups (p = 0.553,
p = 0.271 and p = 0.507, respectively) (Table 5). Visual inspection of the funnel plot of visual–spatial
Interestingly, a significant correlation was observed ability showed an asymmetrical distribution of the
in the subgroup learning plateau in the moderator participant group effect sizes in DS (Fig. 3a). The
0
418 ª2016JohnWiley &SonsLtd.MEDICAL EDUCATION 2016;50:409–427